test_runner: avoid recompiling coverage globs for every file#63675
Open
Han5991 wants to merge 2 commits into
Open
test_runner: avoid recompiling coverage globs for every file#63675Han5991 wants to merge 2 commits into
Han5991 wants to merge 2 commits into
Conversation
`shouldSkipFileCoverage(url)` is invoked twice for every covered script (once during source-map mapping and once during merge), and the same script URL is typically reported by every worker. The result depends only on `options.cwd`, `coverageExcludeGlobs`, and `coverageIncludeGlobs`, all of which are fixed for the lifetime of a TestCoverage instance, so the URL -> boolean mapping is deterministic and safe to cache. Add a private `#skipCache` SafeMap and split the method into a thin caching wrapper plus a private `#computeShouldSkipFileCoverage` that holds the original logic. Callers are unchanged. This eliminates redundant glob and URL parsing work proportional to the number of workers x scripts in the coverage report. Refs: nodejs#55103 Signed-off-by: sangwook <rewq5991@gmail.com>
shouldSkipFileCoverage() called matchGlobPattern() for each covered file, and matchGlobPattern() builds a fresh Minimatch (a full glob parse and regexp compile) on every call. The coverage exclude/include globs never change during a run, so the same patterns were recompiled once per file, which dominated the coverage report time. Compile coverageExcludeGlobs/coverageIncludeGlobs to matchers once per TestCoverage instance and reuse them for every file. Expose createMatcher() from internal/fs/glob for this. On a synthetic 200-test-file project this drops shouldSkipFileCoverage from ~117ms to ~11ms (cpu-prof, isolation=none); the saving scales with files * globs. Refs: nodejs#55103 Signed-off-by: sangwook <rewq5991@gmail.com>
Collaborator
|
Review requested:
|
MoLow
approved these changes
May 31, 2026
atlowChemi
approved these changes
May 31, 2026
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #63675 +/- ##
==========================================
- Coverage 91.95% 90.33% -1.63%
==========================================
Files 379 732 +353
Lines 166486 236465 +69979
Branches 25452 44527 +19075
==========================================
+ Hits 153099 213620 +60521
- Misses 13094 14548 +1454
- Partials 293 8297 +8004
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This speeds up the
--experimental-test-coveragereport phase by removing redundant glob work inshouldSkipFileCoverage(). Refs: #55103 (the report-generation half of that issue).Background
When building the coverage report,
shouldSkipFileCoverage(url)runs once per covered file to decide whether the file belongs in the report (e.g. test files are excluded by default). For each exclude/include glob it callsmatchGlobPattern(path, pattern), andmatchGlobPattern()constructs a freshMinimatch— a full glob parse + regexp compile — on every call (lib/internal/fs/glob.js).The exclude/include globs don't change during a run, so the same patterns are recompiled once per file. cpu-profiling a coverage run shows this glob compilation (
shouldSkipFileCoverage→matchGlobPattern→ minimatchmake/parse/regexp compile) as the dominant cost of the report path.Changes
Two commits:
1.
test_runner: avoid recompiling coverage globs for every file— the main change.The globs are fixed for the lifetime of a
TestCoverageinstance, so compile each to a matcher once and reuse it for every file instead of recompiling per call.internal/fs/glob: export the existingcreateMatcher()helper (already used internally byGlobto precompile patterns).test_runner/coverage: build#excludeMatchers/#includeMatchersonce, then match withmatcher.match(path).No behavior change — same patterns, same options, same match results.
2.
test_runner: cache shouldSkipFileCoverage result per URL— complementary.Memoizes the per-URL skip decision. Under
isolation: 'process'the same script URL is reported by every worker, so this dedups the computes, while commit 1 makes each compute cheap. The result depends only on the URL plus the instance's fixed options (cwd, exclude/include globs), so it is safe to cache for the instance lifetime.Measurements
Synthetic project: 200 test files importing 80 shared
srcmodules,--experimental-test-coverage, default exclude glob,--test-isolation=none, cpu-prof of the report-generating process. Median of 3 runs; before/after measured on the same build (the glob commit reverted viagit stash+ rebuild):shouldSkipFileCoverage(inclusive, incl. minimatch)The saving is independent of test weight and scales with
files × globs, so it grows on large suites (which are the ones that notice the report lag) and is negligible on small ones.Correctness
Existing coverage tests pass, including
test/parallel/test-runner-coverage-default-exclusion(which exercises the default exclude behavior) and the rest oftest-runner-coverage*/test-runner-run-coverage.