Running End-to-End Tests With Maven
After we have finished this lesson, we
- Know how we can configure the source and resource directories of our end-to-end tests.
- Can start our web application before our end-to-end tests are run and stop it after our end-to-end tests have been run.
- Understand how we can run end-to-end tests with Maven.
This lesson assumes that:
Watch the Lesson
The text version of this lesson is given in the following:
The Requirements of Our Maven Build
The requirements of our Maven build are:
- Our end-to-end tests must have a separate source directory. The src/end-to-end-test/java directory must contain the source code of our end-to-end tests.
- Our end-to-end tests must have a separate resource directory. The src/end-to-end-test/resources directory must contain the resources of our end-to-end tests.
- Only unit tests are run by default.
- We must be able to run only unit tests, integration tests, or end-to-end tests.
- Our build must start our web application before our end-to-end tests are run and stop it after our end-to-end tests have been run.
- The names of our end-to-end test classes don’t have to start or end with a special marker string.
Let’s start by adding new source and resource directories to our Maven build.
Adding Source and Resource Directories to Our Maven Build
As we remember, we can add additional source and resource directories to our Maven build by using the Build Helper Maven Plugin. Because we have already declared this plugin in our POM file, we can add the required source and resource directories to our Maven build by making the following changes to its configuration:
First, we have to create an execution that adds the source directory of our end-to-end tests (src/end-to-end-test/java) to our Maven build. After we have created this execution, the configuration of the Build Helper Maven plugin looks as follows:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.12</version>
<executions>
<!-- Other executions are omitted -->
<execution>
<id>add-end-to-end-test-sources</id>
<phase>generate-test-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/end-to-end-test/java</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
Second, we have to create an execution that adds the resource directory of our end-to-end tests (src/end-to-end-test/resources) to our Maven build. After we have created this execution, the configuration of the Build Helper Maven plugin looks as follows:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.12</version>
<executions>
<!-- Other executions are omitted -->
<execution>
<id>add-end-to-end-test-sources</id>
<phase>generate-test-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/end-to-end-test/java</source>
</sources>
</configuration>
</execution>
<execution>
<id>add-end-to-end-test-resources</id>
<phase>generate-test-resources</phase>
<goals>
<goal>add-test-resource</goal>
</goals>
<configuration>
<resources>
<resource>
<filtering>true</filtering>
<directory>src/end-to-end-test/resources</directory>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
Additional Reading:
Next, we will find out how we can configure the profiles of our Maven build.
Configuring Our Maven Profiles
As we remember, we can run our integration tests by using the Maven Failsafe plugin. Because we will also run our end-to-end tests by using the same plugin, we have to make the following changes to the configuration of our Maven profiles:
First, we have to rename the skip.integration.tests property to skip.failsafe.plugin. The reason for this is that the old name doesn’t really describe the purpose of the property because the Maven Failsafe plugin runs both integration and end-to-end tests.
After we have renamed this property, the profiles section of our pom.xml file looks as follows:
<properties>
<jdk.version>1.8</jdk.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<skip.unit.tests>false</skip.unit.tests>
<skip.failsafe.plugin>true</skip.failsafe.plugin>
</properties>
<profiles>
<profile>
<id>dev</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
</profile>
<profile>
<id>integration-test</id>
<properties>
<skip.failsafe.plugin>false</skip.failsafe.plugin>
<skip.unit.tests>true</skip.unit.tests>
</properties>
</profile>
</profiles>
Second, we have to create a new Maven profile called: end-to-end-test. This profile ensures that only end-to-end tests are run when the end-to-end-test profile is used. Also, because the Maven Failsafe plugin runs both integration and end-to-end tests, we have to create a new configuration property called: junit.category which configures the category of the test methods (com.testwithspring.intermediate.EndToEndTest) that are run when the end-to-end-test profile is used.
After we have created this Maven profile, the profiles section of our POM file looks as follows:
<profiles>
<profile>
<id>dev</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
</profile>
<profile>
<id>end-to-end-test</id>
<properties>
<junit.category>com.testwithspring.intermediate.EndToEndTest</junit.category>
<skip.failsafe.plugin>false</skip.failsafe.plugin>
<skip.unit.tests>true</skip.unit.tests>
</properties>
</profile>
<profile>
<id>integration-test</id>
<properties>
<skip.failsafe.plugin>false</skip.failsafe.plugin>
<skip.unit.tests>true</skip.unit.tests>
</properties>
</profile>
</profiles>
Third, we have to add the junit.category configuration property to the configuration of the integration-test profile. This ensures that the Maven Failsafe plugin runs the test methods that belong to the IntegrationTest category when we run our integration tests.
After we have added this configuration property to configuration of the integration-test profile, the profiles section of our POM file looks as follows:
<profiles>
<profile>
<id>dev</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
</profile>
<profile>
<id>end-to-end-test</id>
<properties>
<junit.category>com.testwithspring.intermediate.EndToEndTest</junit.category>
<skip.failsafe.plugin>false</skip.failsafe.plugin>
<skip.unit.tests>true</skip.unit.tests>
</properties>
</profile>
<profile>
<id>integration-test</id>
<properties>
<junit.category>com.testwithspring.intermediate.IntegrationTest</junit.category>
<skip.failsafe.plugin>false</skip.failsafe.plugin>
<skip.unit.tests>true</skip.unit.tests>
</properties>
</profile>
</profiles>
Let’s move on and find out how we can configure the Maven Failsafe Plugin.
Configuring the Maven Failsafe Plugin
As I already mentioned, we will run both integration and end-to-end tests by using the Maven Failsafe plugin. That is why we have to make the following changes to its configuration:
- Ensure that the value of the
junit.categoryproperty specifies the category of the invoked test methods. - Ensure that the Maven Failsafe plugin doesn’t run any test methods if the value of the
skip.failsafe.pluginproperty istrue.
After we made the required changes to the configuration of the Maven Failsafe plugin, its configuration looks as follows:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
<executions>
<execution>
<id>integration-tests</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<includes>
<include>**/*.java</include>
</includes>
<groups>${junit.category}</groups>
<skipTests>${skip.failsafe.plugin}</skipTests>
</configuration>
</execution>
</executions>
</plugin>
Next, we will find out how we can start and stop our web application.
Starting and Stopping Our Web Application
Before we can run our end-to-end tests, we have to start our web application before our end-to-end tests are run and stop it after our end-to-end tests have been run. We can do this by making some changes to the configuration of the end-to-end-test profile. If we are using Spring Framework, we can make the required changes by following these steps:
First, because we will run our web application by using the Jetty Maven plugin, we have to declare this plugin in the build section of the configuration of the end-to-end-test Maven profile.
After we have declared this plugin, the configuration of the end-to-end-test Maven profile looks as follows:
<profiles>
<profile>
<id>end-to-end-test</id>
<properties>
<junit.category>com.testwithspring.intermediate.EndToEndTest</junit.category>
<skip.failsafe.plugin>false</skip.failsafe.plugin>
<skip.unit.tests>true</skip.unit.tests>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
<version>9.2.10.v20150310</version>
<configuration>
<scanIntervalSeconds>0</scanIntervalSeconds>
<stopKey>stop</stopKey>
<stopPort>9999</stopPort>
<webAppConfig>
<war>${project.basedir}/target/running-end-to-end-tests.war</war>
<contextPath>/</contextPath>
</webAppConfig>
</configuration>
</plugin>
</plugins>
</build>
</profile>
<!-- Other profiles are omitted -->
</profiles>
Second, we have to add an executions section to the configuration of the Jetty Maven plugin. After we have added this section to the configuration of the Jetty Maven plugin, the configuration of the end-to-end-test profile looks as follows:
<profiles>
<profile>
<id>end-to-end-test</id>
<properties>
<junit.category>com.testwithspring.intermediate.EndToEndTest</junit.category>
<skip.failsafe.plugin>false</skip.failsafe.plugin>
<skip.unit.tests>true</skip.unit.tests>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
<version>9.2.10.v20150310</version>
<configuration>
<scanIntervalSeconds>0</scanIntervalSeconds>
<stopKey>stop</stopKey>
<stopPort>9999</stopPort>
<webAppConfig>
<war>${project.basedir}/target/running-end-to-end-tests.war</war>
<contextPath>/</contextPath>
</webAppConfig>
</configuration>
<executions>
</executions>
</plugin>
</plugins>
</build>
</profile>
<!-- Other profiles are omitted -->
</profiles>
Third, we have to create the executions that start and stop our web application. We can create these executions by following these steps:
- Create an execution that starts our web application before our end-to-end tests are run. This execution must run the start goal of the Jetty Maven plugin when the pre-integration-test phase of the Maven default lifecycle is invoked.
- Create an execution that stops our web application after our end-to-end tests have been run. This must execution must the stop goal of the Jetty Maven plugin when the post-integration-test phase of the Maven default lifecycle is invoked.
After we have created the required executions, the configuration of the end-to-end-test profile looks as follows:
<profiles>
<profile>
<id>end-to-end-test</id>
<properties>
<junit.category>com.testwithspring.intermediate.EndToEndTest</junit.category>
<skip.failsafe.plugin>false</skip.failsafe.plugin>
<skip.unit.tests>true</skip.unit.tests>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
<version>9.2.10.v20150310</version>
<configuration>
<scanIntervalSeconds>0</scanIntervalSeconds>
<stopKey>stop</stopKey>
<stopPort>9999</stopPort>
<webAppConfig>
<war>${project.basedir}/target/running-end-to-end-tests.war</war>
<contextPath>/</contextPath>
</webAppConfig>
</configuration>
<executions>
<execution>
<id>start-jetty</id>
<phase>pre-integration-test</phase>
<goals>
<goal>start</goal>
</goals>
</execution>
<execution>
<id>stop-jetty</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
<!-- Other profiles are omitted -->
</profiles>
There are two things that I want to point out before we continue this lesson:
First, the reason why we should configure the Jetty Maven plugin in the configuration of the end-to-end-test profile is that we don’t want to start our web application when we run our integration tests because starting our web application would make our integration tests slower than they could be.
The downside of this solution is that if we want to run application by using the Jetty Maven plugin, we have to add its configuration to the configuration of the dev Maven profile. This creates duplication that can make our build harder to maintain, but I think that the pros of this approach are bigger than the cons. If you disagree, you should configure this plugin in the “normal” plugins section.
Second, if we are using Spring Boot, we can start and stop our application by leveraging its testing support. We will talk more about this when we learn to configure our end-to-end tests.
Additional Reading:
We are now ready to run our end-to-end tests. Let’s find out how we can do it.
Running Our End-to-End Tests
We can run our end-to-end tests by using the following command:
mvn clean verify -P end-to-end-test
When we run this command at command prompt and our web application uses Spring Framework, we should see the following output:
[INFO] --- jetty-maven-plugin:9.2.10.v20150310:start (start-jetty)
@ running-end-to-end-tests ---
[INFO] Started ServerConnector@5d9d2119{HTTP/1.1}{0.0.0.0:8080}
[INFO] Started @4580ms
[INFO] Started Jetty Server
[INFO]
[INFO] --- maven-failsafe-plugin:2.19.1:integration-test (integration-tests)
@ running-end-to-end-tests ---
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.testwithspring.intermediate.message.ShowMessageText
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.969 sec
- in com.testwithspring.intermediate.message.ShowMessageText
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] --- jetty-maven-plugin:9.2.10.v20150310:stop (stop-jetty)
@ running-end-to-end-tests ---
[INFO]
[INFO] --- maven-failsafe-plugin:2.19.1:verify (integration-tests)
@ running-end-to-end-tests ---
[INFO] Stopped ServerConnector@5d9d2119{HTTP/1.1}{0.0.0.0:8080}
This output reveals us three things:
- The Jetty Maven plugin starts our web application before our end-to-end tests are run.
- The Maven Failsafe plugin runs only our end-to-end tests when the end-to-end-test Maven profile is used.
- The Jetty Maven plugin stops our web application after our end-to-end tests have been run.
Let’s summarize what we learned from this lesson.
Summary
This lesson has taught us four things:
- We can add custom source and resource directories to our Maven build by using the Build Helper Maven Plugin.
- If we are using Spring Framework, we can start and stop our web application by using the Jetty Maven plugin.
- If we are using Spring Boot, we can start and stop our web application by leveraging its testing support.
- We can run both integration and end-to-end tests by using the Maven Failsafe plugin.