Skip to content

test_runner: avoid recompiling coverage globs for every file#63675

Open
Han5991 wants to merge 2 commits into
nodejs:mainfrom
Han5991:perf/test-runner-coverage-merge
Open

test_runner: avoid recompiling coverage globs for every file#63675
Han5991 wants to merge 2 commits into
nodejs:mainfrom
Han5991:perf/test-runner-coverage-merge

Conversation

@Han5991
Copy link
Copy Markdown
Contributor

@Han5991 Han5991 commented May 31, 2026

This speeds up the --experimental-test-coverage report phase by removing redundant glob work in shouldSkipFileCoverage(). Refs: #55103 (the report-generation half of that issue).

Background

When building the coverage report, shouldSkipFileCoverage(url) runs once per covered file to decide whether the file belongs in the report (e.g. test files are excluded by default). For each exclude/include glob it calls matchGlobPattern(path, pattern), and matchGlobPattern() constructs a fresh Minimatch — a full glob parse + regexp compile — on every call (lib/internal/fs/glob.js).

The exclude/include globs don't change during a run, so the same patterns are recompiled once per file. cpu-profiling a coverage run shows this glob compilation (shouldSkipFileCoveragematchGlobPattern → minimatch make/parse/regexp compile) as the dominant cost of the report path.

Changes

Two commits:

1. test_runner: avoid recompiling coverage globs for every file — the main change.
The globs are fixed for the lifetime of a TestCoverage instance, so compile each to a matcher once and reuse it for every file instead of recompiling per call.

  • internal/fs/glob: export the existing createMatcher() helper (already used internally by Glob to precompile patterns).
  • test_runner/coverage: build #excludeMatchers / #includeMatchers once, then match with matcher.match(path).

No behavior change — same patterns, same options, same match results.

2. test_runner: cache shouldSkipFileCoverage result per URL — complementary.
Memoizes the per-URL skip decision. Under isolation: 'process' the same script URL is reported by every worker, so this dedups the computes, while commit 1 makes each compute cheap. The result depends only on the URL plus the instance's fixed options (cwd, exclude/include globs), so it is safe to cache for the instance lifetime.

Measurements

Synthetic project: 200 test files importing 80 shared src modules, --experimental-test-coverage, default exclude glob, --test-isolation=none, cpu-prof of the report-generating process. Median of 3 runs; before/after measured on the same build (the glob commit reverted via git stash + rebuild):

metric before after
shouldSkipFileCoverage (inclusive, incl. minimatch) ~117 ms ~11 ms
minimatch self-time (glob compile) ~111 ms ~11 ms

The saving is independent of test weight and scales with files × globs, so it grows on large suites (which are the ones that notice the report lag) and is negligible on small ones.

Correctness

Existing coverage tests pass, including test/parallel/test-runner-coverage-default-exclusion (which exercises the default exclude behavior) and the rest of test-runner-coverage* / test-runner-run-coverage.

Han5991 added 2 commits May 31, 2026 14:23
`shouldSkipFileCoverage(url)` is invoked twice for every covered
script (once during source-map mapping and once during merge), and
the same script URL is typically reported by every worker. The
result depends only on `options.cwd`, `coverageExcludeGlobs`, and
`coverageIncludeGlobs`, all of which are fixed for the lifetime of
a TestCoverage instance, so the URL -> boolean mapping is
deterministic and safe to cache.

Add a private `#skipCache` SafeMap and split the method into a thin
caching wrapper plus a private `#computeShouldSkipFileCoverage`
that holds the original logic. Callers are unchanged.

This eliminates redundant glob and URL parsing work proportional to
the number of workers x scripts in the coverage report.

Refs: nodejs#55103
Signed-off-by: sangwook <rewq5991@gmail.com>
shouldSkipFileCoverage() called matchGlobPattern() for each covered
file, and matchGlobPattern() builds a fresh Minimatch (a full glob
parse and regexp compile) on every call. The coverage exclude/include
globs never change during a run, so the same patterns were recompiled
once per file, which dominated the coverage report time.

Compile coverageExcludeGlobs/coverageIncludeGlobs to matchers once per
TestCoverage instance and reuse them for every file. Expose
createMatcher() from internal/fs/glob for this.

On a synthetic 200-test-file project this drops shouldSkipFileCoverage
from ~117ms to ~11ms (cpu-prof, isolation=none); the saving scales with
files * globs.

Refs: nodejs#55103
Signed-off-by: sangwook <rewq5991@gmail.com>
@nodejs-github-bot
Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/test_runner

@nodejs-github-bot nodejs-github-bot added fs Issues and PRs related to the fs subsystem / file system. needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem. labels May 31, 2026
Copy link
Copy Markdown
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollina mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label May 31, 2026
Comment thread lib/internal/test_runner/coverage.js
@codecov
Copy link
Copy Markdown

codecov Bot commented May 31, 2026

Codecov Report

❌ Patch coverage is 93.10345% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.33%. Comparing base (8b88e2c) to head (226a920).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
lib/internal/test_runner/coverage.js 92.85% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #63675      +/-   ##
==========================================
- Coverage   91.95%   90.33%   -1.63%     
==========================================
  Files         379      732     +353     
  Lines      166486   236465   +69979     
  Branches    25452    44527   +19075     
==========================================
+ Hits       153099   213620   +60521     
- Misses      13094    14548    +1454     
- Partials      293     8297    +8004     
Files with missing lines Coverage Δ
lib/internal/fs/glob.js 91.49% <100.00%> (+3.37%) ⬆️
lib/internal/test_runner/coverage.js 62.90% <92.85%> (+0.22%) ⬆️

... and 479 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fs Issues and PRs related to the fs subsystem / file system. needs-ci PRs that need a full CI run. request-ci Add this label to start a Jenkins CI on a PR. test_runner Issues and PRs related to the test runner subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants