Running End-to-End Tests With Gradle – Spock Edition
After we have finished this lesson, we:
- Know how we can configure the source and resource directories of our end-to-end tests.
- Can start our web application before our end-to-end tests are run and stop it after our end-to-end tests have been run.
- Understand how we can compile and run end-to-end tests which use Spock Framework.
This lesson assumes that:
Watch the Lesson
The text version of this lesson is given in the following:
The Requirements of Our Gradle Build
The requirements of our Gradle build are:
- Our end-to-end tests must have a separate source directory. The src/end-to-end-test/groovy directory must contain the source code of our end-to-end tests.
- Our end-to-end tests must have a separate resource directory. The src/end-to-end-test/resources directory must contain the resources of our end-to-end tests.
- We must be able to run either unit tests, integration tests, end-to-end tests, or all tests.
- If an end-to-end test fails, our build must fail as well.
- Unit, integration, and end-to-end tests must have different HTML report directories.
Next, we will find out how we can configure the source and resource directories of our end-to-end tests.
Configuring the Source and Resource Directories of Our End-to-End Tests
As we remember, we can configure the source and resource directories of our end-to-end tests by using the Gradle TestSets plugin.
Because we have already applied this plugin, we can configure the source and resource directories of our end-to-end tests by adding a new test set called endToEndTest
to our Gradle build. When we add this test set to our Gradle build, we must ensure that the root directory of the created source set is: src/end-to-end-test.
After we have added a new test set to our Gradle build, it looks as follows:
buildscript { repositories { jcenter() } dependencies { classpath( 'io.spring.gradle:dependency-management-plugin:1.0.0.RELEASE', 'org.unbroken-dome.gradle-plugins:gradle-testsets-plugin:1.4.2' ) } } apply plugin: 'groovy' apply plugin: 'war' apply plugin: 'io.spring.dependency-management' apply plugin: 'org.unbroken-dome.test-sets' testSets { endToEndTest { dirName = 'end-to-end-test' } integrationTest { dirName = 'integration-test' } } check.dependsOn integrationTest integrationTest.mustRunAfter test tasks.withType(Test) { reports.html.destination = file("${reporting.baseDir}/${name}") } //The dependency management section is excluded. test { useJUnit { includeCategories 'com.testwithspring.master.UnitTest' } testLogging { showStandardStreams = true } } integrationTest { useJUnit { includeCategories 'com.testwithspring.master.IntegrationTest' } testLogging { showStandardStreams = true } }
Additional Reading:
This configuration adds a source set called: endToEndTest
to our build and configures the root directory of the source set. In other words, the src/end-to-end-test/groovy directory contains the source code of our our end-to-end tests and the src/end-to-end-test/resources directory contains the resources of our end-to-end tests.
Let’s move on and find out how we can configure the task that runs our end-to-end tests.
Configuring the Task That Runs Our End-to-End Tests
When we added a test set called endToEndTest
to our build, the Gradle TestSets plugin created a task called endToEndTest
that runs our end-to-end tests. However, because we need to fulfill the requirements of our Gradle build, we have to make some changes to its configuration. We can make these changes by following these steps:
First, we have to ensure that integration tests are run before end-to-end tests and that end-to-end tests are run when we invoke the build
task. We can do this by following these steps:
- Ensure that our end-to-end tests are run before the
check
task and that thecheck
task fails the build if an end-to-end test fails. - Ensure that our integration tests are run before our end-to-end tests. This guarantees that our integration tests are run even if our end-to-end tests fail.
After we have made these changes, our build.gradle file looks as follows:
buildscript { repositories { jcenter() } dependencies { classpath( 'io.spring.gradle:dependency-management-plugin:1.0.0.RELEASE', 'org.unbroken-dome.gradle-plugins:gradle-testsets-plugin:1.4.2' ) } } apply plugin: 'groovy' apply plugin: 'war' apply plugin: 'io.spring.dependency-management' apply plugin: 'org.unbroken-dome.test-sets' testSets { endToEndTest { dirName = 'end-to-end-test' } integrationTest { dirName = 'integration-test' } } check.dependsOn integrationTest check.dependsOn endToEndTest integrationTest.mustRunAfter test endToEndTest.mustRunAfter integrationTest tasks.withType(Test) { reports.html.destination = file("${reporting.baseDir}/${name}") } //The dependency management section is excluded. test { useJUnit { includeCategories 'com.testwithspring.master.UnitTest' } testLogging { showStandardStreams = true } } integrationTest { useJUnit { includeCategories 'com.testwithspring.master.IntegrationTest' } testLogging { showStandardStreams = true } }
Additional Reading:
Second, we have to configure the endToEndTest
task by following these steps:
- Ensure that our end-to-end tests are run by JUnit. Even though JUnit is enabled by default, we have to do this because we want to make changes to the JUnit configuration.
- Configure JUnit to run only the feature methods that belong to the
EndToEndTest
category. We can do this by setting the fully qualified class name of theEndToEndTest
interface as the value of theincludeCategories
JUnit configuration option. - Ensure that Gradle prints the information that is written to
System.out
orSystem.err
when our end-to-end tests are run. By the way, we have to do this only if our end-to-end tests write information toSystem.out
orSystem.err
.
After we have configured the endToEndTest
task, our build.gradle file looks as follows:
buildscript { repositories { jcenter() } dependencies { classpath( 'io.spring.gradle:dependency-management-plugin:1.0.0.RELEASE', 'org.unbroken-dome.gradle-plugins:gradle-testsets-plugin:1.4.2' ) } } apply plugin: 'groovy' apply plugin: 'war' apply plugin: 'io.spring.dependency-management' apply plugin: 'org.unbroken-dome.test-sets' testSets { endToEndTest { dirName = 'end-to-end-test' } integrationTest { dirName = 'integration-test' } } check.dependsOn integrationTest check.dependsOn endToEndTest integrationTest.mustRunAfter test endToEndTest.mustRunAfter integrationTest tasks.withType(Test) { reports.html.destination = file("${reporting.baseDir}/${name}") } //The dependency management section is excluded. test { useJUnit { includeCategories 'com.testwithspring.master.UnitTest' } testLogging { showStandardStreams = true } } integrationTest { useJUnit { includeCategories 'com.testwithspring.master.IntegrationTest' } testLogging { showStandardStreams = true } } endToEndTest { useJUnit { includeCategories 'com.testwithspring.master.EndToEndTest' } testLogging { showStandardStreams = true } }
Let’s move on and find out how we can start and stop our web application.
Starting And Stopping Our Web Application
Before we can run our end-to-end tests, we have to start our web application before our end-to-end tests are run and stop it after our end-to-end tests have been run. If we are using Spring Framework, we can run our web application by using the Gretty Gradle plugin. We can configure this plugin by following these steps:
First, we have to declare the dependencies of our build script by following these steps:
- Configure Gradle to use the Bintray’s JCenter Maven repository when it resolves the dependencies of our build script.
- Add the Gretty Gradle plugin dependency to the
classpath
configuration.
After we have configured the dependencies of our build script, our build.gradle file looks as follows:
buildscript { repositories { jcenter() } dependencies { classpath( 'io.spring.gradle:dependency-management-plugin:1.0.0.RELEASE', 'org.unbroken-dome.gradle-plugins:gradle-testsets-plugin:1.4.2', 'org.akhikhl.gretty:gretty:2.0.0' ) } } apply plugin: 'groovy' apply plugin: 'war' apply plugin: 'io.spring.dependency-management' apply plugin: 'org.unbroken-dome.test-sets' testSets { endToEndTest { dirName = 'end-to-end-test' } integrationTest { dirName = 'integration-test' } } check.dependsOn integrationTest check.dependsOn endToEndTest integrationTest.mustRunAfter test endToEndTest.mustRunAfter integrationTest tasks.withType(Test) { reports.html.destination = file("${reporting.baseDir}/${name}") } //The dependency management section is excluded. test { useJUnit { includeCategories 'com.testwithspring.master.UnitTest' } testLogging { showStandardStreams = true } } integrationTest { useJUnit { includeCategories 'com.testwithspring.master.IntegrationTest' } testLogging { showStandardStreams = true } } endToEndTest { useJUnit { includeCategories 'com.testwithspring.master.EndToEndTest' } testLogging { showStandardStreams = true } }
Second, we have to apply the Gretty Gradle plugin. After we have done this, our build.gradle file looks as follows:
buildscript { repositories { jcenter() } dependencies { classpath( 'io.spring.gradle:dependency-management-plugin:1.0.0.RELEASE', 'org.unbroken-dome.gradle-plugins:gradle-testsets-plugin:1.4.2', 'org.akhikhl.gretty:gretty:2.0.0' ) } } apply plugin: 'groovy' apply plugin: 'war' apply plugin: 'io.spring.dependency-management' apply plugin: 'org.unbroken-dome.test-sets' apply plugin: 'org.akhikhl.gretty' testSets { endToEndTest { dirName = 'end-to-end-test' } integrationTest { dirName = 'integration-test' } } check.dependsOn integrationTest check.dependsOn endToEndTest integrationTest.mustRunAfter test endToEndTest.mustRunAfter integrationTest tasks.withType(Test) { reports.html.destination = file("${reporting.baseDir}/${name}") } //The dependency management section is excluded. test { useJUnit { includeCategories 'com.testwithspring.master.UnitTest' } testLogging { showStandardStreams = true } } integrationTest { useJUnit { includeCategories 'com.testwithspring.master.IntegrationTest' } testLogging { showStandardStreams = true } } endToEndTest { useJUnit { includeCategories 'com.testwithspring.master.EndToEndTest' } testLogging { showStandardStreams = true } }
Third, we have to configure the Gretty Gradle plugin by following these steps:
- Ensure that used servlet container listens to the port 8080.
- Configure the context path of our web application.
- Ensure that our web application is run by using Jetty 9.4.
- Ensure that Gretty starts our web application before our end-to-end tests are run and stops it after our end-to-end tests have been run.
After we have configured the Gretty Gradle plugin, our build.gradle file looks as follows:
buildscript { repositories { jcenter() } dependencies { classpath( 'io.spring.gradle:dependency-management-plugin:1.0.0.RELEASE', 'org.unbroken-dome.gradle-plugins:gradle-testsets-plugin:1.4.2', 'org.akhikhl.gretty:gretty:2.0.0' ) } } apply plugin: 'groovy' apply plugin: 'war' apply plugin: 'io.spring.dependency-management' apply plugin: 'org.unbroken-dome.test-sets' apply plugin: 'org.akhikhl.gretty' testSets { endToEndTest { dirName = 'end-to-end-test' } integrationTest { dirName = 'integration-test' } } check.dependsOn integrationTest check.dependsOn endToEndTest integrationTest.mustRunAfter test endToEndTest.mustRunAfter integrationTest tasks.withType(Test) { reports.html.destination = file("${reporting.baseDir}/${name}") } gretty { httpPort = 8080 contextPath = '/' servletContainer = 'jetty9.4' integrationTestTask = 'endToEndTest' } //The dependency management section is excluded. test { useJUnit { includeCategories 'com.testwithspring.master.UnitTest' } testLogging { showStandardStreams = true } } integrationTest { useJUnit { includeCategories 'com.testwithspring.master.IntegrationTest' } testLogging { showStandardStreams = true } } endToEndTest { useJUnit { includeCategories 'com.testwithspring.master.EndToEndTest' } testLogging { showStandardStreams = true } }
If we are using Spring Boot, we can start and stop our application by leveraging its testing support. We will talk more about this when we learn to configure our end-to-end tests.
Additional Reading:
We are now ready to run our end-to-end tests. Let’s find out how we can do it.
Running End-to-End Tests
We can run our end-to-end tests by using one of the following options:
First, we can run our end-to-end tests by using the command: gradle clean endToEndTest.
When we run this command at command prompt and our web application uses Spring Framework, we should see the following output:
21:31:32 INFO Jetty 9.4.7.v20170914 started and listening on port 8080 21:31:32 INFO runs at: 21:31:32 INFO http://localhost:8080/ > Task :endToEndTest com.testwithspring.master.message.ShowMessageTextSpec > Open message page STANDARD_OUT End-to-End Test: Should show the correct title > Task :appAfterIntegrationTest Server stopped. BUILD SUCCESSFUL in 16s
This output reveals us three things:
- The Gretty Gradle plugin starts our web application before our end-to-end tests are run.
- Gradle runs our end-to-end tests when we run the
endToEndTest
task. - The Gretty Gradle plugin stops our web application after our end-to-end tests have been run.
Second, if we want to run all tests, we can use one of these two options:
- We can run unit, integration, and end-to-end tests by running the command: gradle clean test integrationTest endToEndTest at command prompt.
- We can run our build by running the command: gradle clean build at command prompt.
When we run all tests and our web application uses Spring Framework, we should see an output that looks as follows:
> Task :test com.testwithspring.master.message.MessageControllerSpec > Show Message STANDARD_OUT Unit Test: Should render the show message view > Task :integrationTest com.testwithspring.master.message.ShowMessageSpec > Show Message STANDARD_OUT Integration Test: Should forward the user to the show message page 21:32:56 INFO Jetty 9.4.7.v20170914 started and listening on port 8080 21:32:56 INFO runs at: 21:32:56 INFO http://localhost:8080/ > Task :endToEndTest com.testwithspring.master.message.ShowMessageTextSpec > Open message page STANDARD_OUT End-to-End Test: Should show the correct title > Task :appAfterIntegrationTest Server stopped. BUILD SUCCESSFUL in 14s
If the Gretty Gradle plugin cannot start Jetty because the provider: LogbackServletContainerInitializer
was not found, you have to add the logback-classic
dependency (version 1.2.3) to the runtime
configuration. This is a workaround for a known bug of the Gretty Gradle plugin, and it will be hopefully fixed in the future. The downside of this solution is that our classpath contains multiple SLF4J bindings when we start our web application.
Also, if you don’t know how you can declare this dependency, you should take a look at the example application of this lesson.
Let’s summarize what we learned from this lesson.
Summary
This lesson has taught us three things:
- If we are using Spring Framework, we can start and stop our web application by using the Gretty Gradle plugin.
- If we are using Spring Boot, we can start and stop our web application by using its testing support.
- We can run both integration and end-to-end tests by using the tasks created by the Gradle TestSets plugin.
Previous LessonNext Lesson