Skip to content
This repository has been archived by the owner on Apr 8, 2024. It is now read-only.

Feature/num 1061 parallel test execution (pabot) #245

Merged
merged 40 commits into from
Apr 14, 2021

Conversation

wlad
Copy link
Contributor

@wlad wlad commented Feb 25, 2021

Introduces pabot as test dependency for parallel test execution

  • parallelization happens on test case level and is limited to the amount of available CPU cores/threads
  • example: on a machine with 24 CPU cores 24 tests a run simultaneously
  • since CircleCI's default docker executors has 2 vCPU cores only I had to configure a higher resource_class (large) to take a notice of any parallelization effect (NOTE: resource_classes other than default require a paid plan on CircleCI)
  • execution time saving depends highly on amount of CPU cores
  • actual execution time saved using resource_class=large (4 x vCPU) is about 50% (~ 10 min vs. ~20 - 30 min), compare builds
  • some tests fail randomly when using parallel execution. This issue is tracked here
  • not sure if it's always the same tests that are impacted. We'll sync this branch with develop and observe behavior over time
  • failing tests are tagged with not-ready_parallel_fail and marked as not-ready

@wlad wlad requested review from af-pwo and af-dpe February 25, 2021 17:41
@wlad
Copy link
Contributor Author

wlad commented Feb 25, 2021

maybe is better to remove not-ready tags from impacted tests , so we see them failing (when they fail)?

@wlad wlad added the blocked label Mar 4, 2021
@wlad
Copy link
Contributor Author

wlad commented Mar 4, 2021

⛔ blocked by #247

@sonarcloud
Copy link

sonarcloud bot commented Mar 10, 2021

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@wlad
Copy link
Contributor Author

wlad commented Mar 11, 2021

⛔ blocked by ehrbase/ehrbase#460

@wlad
Copy link
Contributor Author

wlad commented Mar 22, 2021

UPDATE:

synced with develop and ran tests locally. only three tests fail (one is marked as skip).
image

image
image

Locally impressive speed up with parallel execution via pabot (~3 m vs ~30 m on CI. OK that's without ehrbase + fhirbridge startup overhead, but even with startup overhead still would be less than 5 m)
image

The massive fail on CI may be caused by limited resources of Docker executer (and now probably also due to downgrade to CircleCI free plan)

@wlad wlad merged commit aa01db2 into develop Apr 14, 2021
@wlad wlad deleted the feature/NUM-1061_Parallel-test-execution branch April 14, 2021 17:46
@wlad wlad removed request for af-pwo and af-dpe April 14, 2021 23:23
@wlad wlad changed the title Feature/num 1061 parallel test execution Feature/num 1061 parallel test execution (pabot) Jul 14, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant