Limit python multiprocessing jobs on windows due to Python3 bug.
Python3 appears to have a bug causing hangs when running jobs using the
`multiprocessing` module to distribute tasks across more than 56 cores
on Windows.
This CL puts a ceiling on any such code to avoid the hang. This
should have little to no impact in practice because there are
few if any such tasks that we do that really take advantage of
that many cores. But, because it's not uncommon to use the default
cpu_count() value, we are more prone to hit this on very high-end
machines.
This CL does not change any Android-specific scripts, since those
won't be run on Windows.
Bug: 1190269
Change-Id: Id2a4d9f36294e53878e452f9bf713f0ba2c0345f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2785844
Commit-Queue: Dirk Pranke <[email protected]>
Reviewed-by: John Chen <[email protected]>
Reviewed-by: Robert Sesek <[email protected]>
Reviewed-by: Kentaro Hara <[email protected]>
Reviewed-by: Yuki Shiino <[email protected]>
Cr-Commit-Position: refs/heads/master@{#866704}
diff --git a/tools/code_coverage/coverage.py b/tools/code_coverage/coverage.py
index 773f172..139fbf0 100755
--- a/tools/code_coverage/coverage.py
+++ b/tools/code_coverage/coverage.py
@@ -794,14 +794,21 @@
def _GetCommandForWebTests(arguments):
"""Return command to run for blink web tests."""
+ cpu_count = multiprocessing.cpu_count()
+ if sys.platform == 'win32':
+ # TODO(crbug.com/1190269) - we can't use more than 56
+ # cores on Windows or Python3 may hang.
+ cpu_count = min(cpu_count, 56)
+ cpu_count = max(1, cpu_count // 2)
+
command_list = [
'python', 'testing/xvfb.py', 'python',
'third_party/blink/tools/run_web_tests.py',
'--additional-driver-flag=--no-sandbox',
'--additional-env-var=LLVM_PROFILE_FILE=%s' %
LLVM_PROFILE_FILE_PATH_SUBSTITUTION,
- '--child-processes=%d' % max(1, int(multiprocessing.cpu_count() / 2)),
- '--disable-breakpad', '--no-show-results', '--skip-failing-tests',
+ '--child-processes=%d' % cpu_count, '--disable-breakpad',
+ '--no-show-results', '--skip-failing-tests',
'--target=%s' % os.path.basename(BUILD_DIR), '--time-out-ms=30000'
]
if arguments.strip():
