Integration Tests in Application Layer Randomly Deadlock or Hang

[magicjar]

Hi, I'm running into an issue with integration testing in my project. Here’s a breakdown of the problem:

• Im using abp 9.1.3
• Tests are implemented as integration tests in the Application layer
• Data seeding is used extensively, almost all of my domain models are seeded for testing
• Many of these models have navigation properties
• The test suite is relatively a lot

The Problem:

• Tests sometimes hang or deadlock during execution
• The issue seems random, it can happen in different test cases across runs
• It happens often in GitHub Actions runner (Ubuntu latest), but rarely happens locally (in VS Code)
• When it does hang, it doesn't throw an exception, the test runner just gets stuck indefinitely

If anyone has experienced similar issues or has any insights on what might be causing this, I'd appreciate the help!

Thanks in advance.

Just an example:

// CountryAppServiceTests.cs

public async Task Should_Get_US_Country_By_ISO()
    {
        // Act
        var result = await _countryAppService.GetCountryUsingISOAsync("US");

        // Assert
        result.Name.ShouldBe("US");
    }

// CountryAppService.cs

public override async Task<CountryDto> GetCountryUsingISOAsync(string iso3166)
    {
        var result = _countryRepository.GetCountryUsingISOAsync(iso3166);
        return ObjectMapper.Map<Country, CountryDto>(result));
    }

// CountryRepository.cs

public async Task<Country> GetCountryUsingISOAsync(string iso3166)
    {
        var dbContext = await GetDbContextAsync();
        var country = await dbContext.Set<Country>()
            .AsNoTracking()
            .Where(p => p.Name.ToUpper() == iso3166.ToUpper())
            .FirstOrDefaultAsync(GetCancellationToken(cancellationToken));

        return country;
    }

[maliming]

hi

Can you share the pipeline logs from the GitHub action?

[magicjar]

VSTest version 17.14.0-release-25203-01 (x64)
Starting test execution, please wait...
Logging Vstest Diagnostics in file: /home/runner/work/test-logs/diagnostics_mobi-web_20250609_114024.log
A total of 1 test files matched the specified pattern.
/home/runner/work/mobi-web/mobi-web/test/Mobi.Application.Tests/bin/Debug/net9.0/Mobi.Application.Tests.dll
Test run for /home/runner/work/mobi-web/mobi-web/test/Mobi.EntityFrameworkCore.Tests/bin/Debug/net9.0/Mobi.EntityFrameworkCore.Tests.dll (.NETCoreApp,Version=v9.0)
VSTest version 17.14.0-release-25203-01 (x64)
Logging TestHost Diagnostics in file: /home/runner/work/test-logs/diagnostics_mobi-web_20250609_114024.host.25-06-09_11-40-25_74112_5.log
Starting test execution, please wait...
Logging Vstest Diagnostics in file: /home/runner/work/test-logs/diagnostics_mobi-web_20250609_114024.log
A total of 1 test files matched the specified pattern.
/home/runner/work/mobi-web/mobi-web/test/Mobi.EntityFrameworkCore.Tests/bin/Debug/net9.0/Mobi.EntityFrameworkCore.Tests.dll
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v3.1.0+00ab0a9ec4 (64-bit .NET 9.0.5)
[xUnit.net 00:00:00.16]   Discovering: Mobi.Application.Tests
[xUnit.net 00:00:00.18]   Discovered:  Mobi.Application.Tests
No test is available in /home/runner/work/mobi-web/mobi-web/test/Mobi.Application.Tests/bin/Debug/net9.0/Mobi.Application.Tests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
Logging TestHost Diagnostics in file: /home/runner/work/test-logs/diagnostics_mobi-web_20250609_114024.host.25-06-09_11-40-26_44657_5.log
Test run for /home/runner/work/mobi-web/mobi-web/test/Mobi.TestBase/bin/Debug/net9.0/Mobi.TestBase.dll (.NETCoreApp,Version=v9.0)
VSTest version 17.14.0-release-25203-01 (x64)
[xUnit.net 00:00:00.01] xUnit.net VSTest Adapter v3.1.0+00ab0a9ec4 (64-bit .NET 9.0.5)
Starting test execution, please wait...
Logging Vstest Diagnostics in file: /home/runner/work/test-logs/diagnostics_mobi-web_20250609_114024.log
[xUnit.net 00:00:00.28]   Discovering: Mobi.EntityFrameworkCore.Tests
A total of 1 test files matched the specified pattern.
/home/runner/work/mobi-web/mobi-web/test/Mobi.TestBase/bin/Debug/net9.0/Mobi.TestBase.dll
[xUnit.net 00:00:00.51]   Discovered:  Mobi.EntityFrameworkCore.Tests
[xUnit.net 00:00:00.68]   Starting:    Mobi.EntityFrameworkCore.Tests
Logging TestHost Diagnostics in file: /home/runner/work/test-logs/diagnostics_mobi-web_20250609_114024.host.25-06-09_11-40-27_62795_5.log
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v3.1.0+00ab0a9ec4 (64-bit .NET 9.0.5)
[xUnit.net 00:00:00.32]   Discovering: Mobi.TestBase
[xUnit.net 00:00:00.38]   Discovered:  Mobi.TestBase
No test is available in /home/runner/work/mobi-web/mobi-web/test/Mobi.TestBase/bin/Debug/net9.0/Mobi.TestBase.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
Test run for /home/runner/work/mobi-web/mobi-web/test/Mobi.Domain.Tests/bin/Debug/net9.0/Mobi.Domain.Tests.dll (.NETCoreApp,Version=v9.0)
VSTest version 17.14.0-release-25203-01 (x64)
Starting test execution, please wait...
Logging Vstest Diagnostics in file: /home/runner/work/test-logs/diagnostics_mobi-web_20250609_114024.log
A total of 1 test files matched the specified pattern.
/home/runner/work/mobi-web/mobi-web/test/Mobi.Domain.Tests/bin/Debug/net9.0/Mobi.Domain.Tests.dll
Logging TestHost Diagnostics in file: /home/runner/work/test-logs/diagnostics_mobi-web_20250609_114024.host.25-06-09_11-40-30_12451_5.log
[xUnit.net 00:00:00.01] xUnit.net VSTest Adapter v3.1.0+00ab0a9ec4 (64-bit .NET 9.0.5)
[xUnit.net 00:00:00.23]   Discovering: Mobi.Domain.Tests
[xUnit.net 00:00:00.27]   Discovered:  Mobi.Domain.Tests
No test is available in /home/runner/work/mobi-web/mobi-web/test/Mobi.Domain.Tests/bin/Debug/net9.0/Mobi.Domain.Tests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowtimeAppServiceTests.Should_Get_Showtimes_By_Date [12 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowtimeAppServiceTests.Should_Get_Now_Playing_Shows [3 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowtimeAppServiceTests.Should_Not_Create_Showtime_With_Duplicate_EntityId [2 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowtimeAppServiceTests.Should_Throw_Exception_When_Provider_Not_Found [2 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowtimeAppServiceTests.Should_Not_Get_Now_Playing_Shows_With_Same_EntityId [2 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowtimeAppServiceTests.Should_Handle_Time_Conversion_Correctly [2 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowtimeAppServiceTests.Should_Not_Create_Showtime_With_Invalid_Input [2 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowtimeAppServiceTests.Should_Create_Showtime_Successfully [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowtimeAppServiceTests.Should_Return_Empty_List_For_Date_With_No_Showtimes [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Update_ReleaseInfo_From_Null_To_Valid_Certification [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Not_Create_Show_With_Plot_Too_Long [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Get_Show_Countries [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Get_Show_By_Id [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Add_Genre_To_Show [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Delete_Credit_From_Show [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Get_Show_List [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Not_Add_Credit_With_Invalid_Person_Id [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Update_ReleaseInfo_In_Show [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Delete_ReleaseInfo_From_Show [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Not_Update_Show_With_Empty_Name [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Add_ReleaseInfo_To_Show [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Update_Company_In_Show [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Get_Latest_List [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Update_Credit_In_Show [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Get_Credits [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Not_Get_Nonexistent_Show [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Not_Add_ExternalId_With_Invalid_Provider [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Update_Show [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Not_Create_Show_With_Empty_Title [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Delete_Show [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Add_ReleaseInfo_To_Show_With_Null_Certification [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Add_Country_To_Show [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Not_Update_Nonexistent_Credit [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Delete_Genre_From_Show [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Create_Show [1 s]
  Passed Mobi.EntityFrameworkCore.Applications.EfCoreShowAppServiceTests.Should_Add_Company_To_Show [1 s]

The test literally just stops right there, no error, nothing. It just waits until the workflow times out. The stopping point isn’t consistent—sometimes it happens while testing EfCoreShowAppServiceTests, other times during EfCoreGenreAppServiceTests, or even elsewhere.

[maliming]

If it can't be reproduced, it will be hard to find the cause. Can you provide a project?

[magicjar]

Stumbled upon this solution

#24247 (comment)

I think it fixes the issue. The tests have been running with 100% completion since then.