Publish test execution events to the test ApplicationContext [SPR-13916]

[spring-projects-issues]

Frank Scheffler opened SPR-13916 and commented

We are using Mockito mocks quite often throughout our Spring-based integration tests, which leaves us with the problem of resetting them before/after each test method. While we can use some globally defined custom TestExecutionListener to reset all the mocks found within the ApplicationContext, it would be more practical to be able to define specific event-listeners within the application context itself. This would allow for more fine-grained control over which beans (i.e. mocks) should be reset.

Deliverables

• Implement EventPublishingTestExecutionListener.
• Implement abstract TestContextEvent and all concrete subclasses.
• Implement composed @EventListener annotations for all events.
• Document exception handling and async support in Javadoc.
• Document in the Testing chapter of the reference manual.

---

Issue Links:

• Publish TestStarted and TestCompleted events in the TestContext framework [SPR-8710] #13352 Publish TestStarted and TestCompleted events in the TestContext framework
• Introduce support for test-scoped beans in the TestContext framework [SPR-14034] #18606 Introduce support for test-scoped beans in the TestContext framework

2 votes, 6 watchers

[spring-projects-issues]

Frank Scheffler commented

In fact, one solution would be to provide an additional EventPublishingTestExecutionListener, so users could plug-in the event publishing behavior, if they are in need for it.

TestExecutionListener could look like this, backed by an event class hierarchy reflecting the testcontext lifecycle:

public class EventPublishingTestExecutionListener extends AbstractTestExecutionListener {

  @Override
  public void afterTestMethod(TestContext testContext) throws Exception {
    testContext.getApplicationContext().publishEvent(new AfterMethodTestExecutionEvent(testContext));
  }
  
}

Within the test configuration code you could then, for instance, reset the mocks in question, e.g. by:

@Configuration
public class MyMockConfiguration implements ApplicationListener<AfterMethodTestExecutionEvent> {

   @Bean
   public MyService myService() {
      return Mockito.mock(MyService.class);
   }

   @Override
    public void onApplicationEvent(AfterMethodTestExecutionEvent event) {
      Mockito.reset(myService());
    }
}

[spring-projects-issues]

Frank Scheffler commented

I have created an implementation draft for further discussions at: maverick1601@f07c34c

The tests work fine and I added EventListener-based meta-annotation to simplify integration into configuration classes and beans, e.g. simply annotating methods with @BeforeTestMethod, which is compliant with JUnit life-cycle annotations.

I am, however, currently unsure about the following:

1. What order should the ApplicationEventTestExecutionListener have?
2. Should it be added to the list of default test execution listeners, as i did?
3. Is the eager initialization of the ApplicationContext as part of the listener a problem or not?

Finally I would like to summarize the benefits of this approach and why we need it:

1. Using ApplicationEvents for signaling the test life-cycle enables us to interact with the test framework from inside our Spring configuration classes.
2. We can therefore more concisely prepare or clean up mocks that are beans within the application context. This is inherited behavior for all test cases using such configurations, as compared to copying standard @Before methods around.
3. Event consumption is much more flexible with respect to profiles etc., while TestExecutionListeners have to detect, what needs to be done based on the TestContext only.
4. Moreover, we will be able to reduce the amount of TestExecutionListeners in our case, because the logic can be refactored into event-consuming methods.
5. Being framework agnostic enables us to integrate this with FitNesse and other testing frameworks more easily.

[spring-projects-issues]

Frank Scheffler commented

Any feedback on this or the related question on Google groups (https://groups.google.com/forum/#!topic/spring-framework-contrib/crdJWZk3Wcs). I would like to proceed on finishing the code, if this is considered a useful enhancement to the TCF.

[spring-projects-issues]

Sam Brannen commented

Frank Scheffler,

Thanks for the proposal and draft implementation.

At a quick glance, it looks reasonably sound, though I haven't yet had time to look at it in great detail.

With regard to the automatic registration of ApplicationEventTestExecutionListener, I'm not so sure that's a good idea, especially if the ApplicationContext is always eagerly initialized. Most listeners (except DITEL) tend not to eagerly load a context (or do anything for that matter) if unnecessary. In other words, we wouldn't want to attempt to fire events in a context that will never exist for a given test class.

I'll move this issue to the 4.3 backlog with the hope of finding more time to devote to it as we move toward 4.3 GA.

Regards,

Sam

[spring-projects-issues]

Sam Brannen commented

Please note that this issue is very closely related to and potentially supersedes #13352.

[spring-projects-issues]

Phil Webb commented

You might be interested in spring-projects/spring-boot#5042 which is looking to add better Mockito support to Spring Boot.

[spring-projects-issues]

Frank Scheffler commented

We have implemented this at a customer and enhanced the listener to publish events to the fixture, i.e. the test class, itself as well, if implemented as ApplicationListener or via @EventListener (meta)-annotations. This improves test framework agnosticism, making it easier for us to switch test code from JUnit to TestNG or FitNesse (we have implemented our own Spring-Test integration for that one), later on, because we rely solely on our own meta-annotations instead of JUnit-specific ones.

[sbrannen]

This issue supersedes gh-13352.