manifest/bazel: nested-workspace + Bazel-native Maven extraction

[Simon (simonhj)]

Summary

Rewrites socket manifest bazel's Maven extraction pipeline so it (a)
discovers every workspace under the scan root, not just cwd, and (b)
relies on Bazel-native commands for repo enumeration instead of static
Starlark regex parsing.

The former version was a bit over engineered, it was doing the following things wrong:

• Parsing StarLark files to find maven repos, turns out there was a programmatic way of doing it.
• Repeating server side work by parsing maven_install.json.
• Not handling nested bazel modules correctluy.

After some more iteration I have landed on a better flow that also more easlily extends to other languages:

1. Walk the scan root for every workspace (MODULE.bazel / WORKSPACE /
   WORKSPACE.bazel). Caller-supplied prune policy.
2. Per workspace, discover Maven hubs:
   - Bzlmod: bazel mod show_extension @rules_jvm_external//:extensions.bzl%maven
   - WORKSPACE: probe conventional names (@maven, @maven_install,
     @maven_dev, @unpinned_maven, @maven_unpinned). Tri-state
     classifier (populated / empty / not-defined).
3. Per populated candidate, run per-repo metadata cquery:
     attr("tags", "\bmaven_coordinates=", @<repo>//...)
     ∪ attr("maven_coordinates", ".+", @<repo>//...)
     ∪ attr("maven_url", ".+", @<repo>//...)
   --output=jsonproto --keep_going
4. Aggregate across all workspaces; dedup by full Maven coordinate.
5. Write synthesized maven_install.json.

The past version was the result my (shrinking) bazel ignorance and maybe trusting the AI a little bit too much :)

---

Note

Medium Risk
Large change to how SBOMs are produced (many Bazel subprocesses, timeouts, partial results); mitigated by extensive tests and isolated output_user_root, but wrong hub/cquery behavior could miss or skew dependency lists.

Overview
Rewrites socket manifest bazel Maven extraction so it scans every Bazel workspace under the repo (not only cwd) and drives discovery/extraction through Bazel commands instead of parsing Starlark or reusing unsorted_deps.json / cached bazel query probe output.

Discovery: A new workspace walker finds roots with MODULE.bazel / WORKSPACE / WORKSPACE.bazel (caller-supplied ignore rules; CLI wires IGNORED_DIRS plus bazel-*). Per workspace, Maven hubs come from bazel mod show_extension on rules_jvm_external’s maven extension (Bzlmod), with conventional-name cquery probes and a populated / empty / not-defined classifier for legacy WORKSPACE mode and extras from --bazel-maven-repo=.

Extraction: Each hub gets a metadata cquery (jsonproto, union on maven_coordinates tags/attrs/maven_url), parsed in new bazel-cquery.mts. Artifacts aggregate across workspaces and dedupe by full Maven coordinate before writing synthesized maven_install.json. Bazel runs use a per-invocation --output_user_root, with shutdown + tempdir cleanup and fresh roots after per-repo timeouts so hung servers don’t block the scan.

Supporting changes: Maven repo discovery module shrinks to show_extension parsing + probe classification; query runner adds runBazelModShowMavenExtension, shared startup flags, and buildMavenProbeFor; PyPI keeps parseVisibleRepoCandidates locally; ExtractedArtifact.ruleKind is a string for arbitrary rule classes from cquery.

^{Reviewed by Cursor Bugbot for commit 414a9a6. Configure here.}

[Simon (simonhj)]

Claude (@claude) review once

[cursor]

Cursor Bugbot has reviewed your changes and found 7 potential issues.

Bugbot Autofix prepared fixes for both issues found in the latest run.

• ✅ Fixed: WORKSPACE custom hubs not found
  • Added --bazel-maven-repo flag to cmd-manifest-bazel.mts (with socket.json defaults support) and wired it through as extraMavenRepoNames to both the direct CLI and auto-manifest pathways.
• ✅ Fixed: Empty show_extension skips fallback
  • Changed showExtensionSucceeded to only be set true when parseShowExtensionOutput returns a non-empty hub list, so a zero-exit with no parsed hubs now correctly falls back to conventional name probing.

Or push these changes by commenting:

@cursor push 6019185b06

Preview (6019185b06)

diff --git a/src/commands/manifest/bazel/cmd-manifest-bazel.mts b/src/commands/manifest/bazel/cmd-manifest-bazel.mts
--- a/src/commands/manifest/bazel/cmd-manifest-bazel.mts
+++ b/src/commands/manifest/bazel/cmd-manifest-bazel.mts
@@ -51,6 +51,12 @@
       description:
         'Flags forwarded to every bazel invocation (single quoted string)',
     },
+    bazelMavenRepo: {
+      type: 'string',
+      isMultiple: true,
+      description:
+        'Extra Maven hub repo name(s) to probe; repeatable. Use for legacy WORKSPACE projects whose maven_install uses a non-conventional name.',
+    },
     bazelOutputBase: {
       type: 'string',
       description: 'Bazel --output_base for read-only-cache CI environments',
@@ -202,7 +208,7 @@
     sockJson?.defaults?.manifest?.bazel,
   )
 
-  const { ecosystem } = cli.flags
+  const { bazelMavenRepo, ecosystem } = cli.flags
   let { bazel, bazelFlags, bazelOutputBase, bazelRc, out, verbose } = cli.flags
 
   // Set defaults for any flag/arg that is not given. Check socket.json first.
@@ -260,6 +266,12 @@
     }
   }
 
+  // Compose extra Maven repo names from the CLI flag and socket.json defaults.
+  const extraMavenRepoNames: string[] = [
+    ...(Array.isArray(bazelMavenRepo) ? bazelMavenRepo : []),
+    ...(sockJson.defaults?.manifest?.bazel?.bazelMavenRepo ?? []),
+  ].filter(Boolean)
+
   if (verbose) {
     logger.group('- ', parentName, config.commandName, ':')
     logger.group('- flags:', cli.flags)
@@ -318,6 +330,9 @@
         bazelRc: bazelRc as string | undefined,
         bin: bazel as string | undefined,
         cwd,
+        extraMavenRepoNames: extraMavenRepoNames.length
+          ? extraMavenRepoNames
+          : undefined,
         ignoreDirNames: BAZEL_WALKER_IGNORE_DIR_NAMES,
         ignoreDirPrefixes: BAZEL_WALKER_IGNORE_DIR_PREFIXES,
         out: out as string,

diff --git a/src/commands/manifest/bazel/extract_bazel_to_maven.mts b/src/commands/manifest/bazel/extract_bazel_to_maven.mts
--- a/src/commands/manifest/bazel/extract_bazel_to_maven.mts
+++ b/src/commands/manifest/bazel/extract_bazel_to_maven.mts
@@ -260,8 +260,12 @@
   if (mode.bzlmod) {
     const extResult = await runBazelModShowMavenExtension(queryOpts)
     if (extResult.code === 0) {
-      candidates.push(...parseShowExtensionOutput(extResult.stdout))
-      showExtensionSucceeded = true
+      const parsedHubs = parseShowExtensionOutput(extResult.stdout)
+      candidates.push(...parsedHubs)
+      // Only consider successful when hubs were actually parsed; a zero
+      // exit with no recognized hub names (format mismatch, empty report)
+      // should still fall back to conventional probing.
+      showExtensionSucceeded = parsedHubs.length > 0
       if (verbose) {
         logger.log(
           `[VERBOSE] workspace ${workspaceRoot}: show_extension yielded`,

diff --git a/src/commands/manifest/generate_auto_manifest.mts b/src/commands/manifest/generate_auto_manifest.mts
--- a/src/commands/manifest/generate_auto_manifest.mts
+++ b/src/commands/manifest/generate_auto_manifest.mts
@@ -129,6 +129,7 @@
       bazelRc: bazelConfig?.bazelRc,
       bin: bazelConfig?.bazel ?? bazelConfig?.bin,
       cwd,
+      extraMavenRepoNames: bazelConfig?.bazelMavenRepo,
       out: bazelConfig?.out ?? cwd,
       outLayout: 'flat',
       verbose: Boolean(bazelConfig?.verbose) || verbose,

diff --git a/src/utils/socket-json.mts b/src/utils/socket-json.mts
--- a/src/utils/socket-json.mts
+++ b/src/utils/socket-json.mts
@@ -42,6 +42,7 @@
       bazel?: {
         bazel?: string | undefined
         bazelFlags?: string | undefined
+        bazelMavenRepo?: string[] | undefined
         bazelOutputBase?: string | undefined
         bazelRc?: string | undefined
         bin?: string | undefined

_{You can send follow-ups to the cloud agent here.}

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 414a9a6. Configure here.}

[cursor]

WORKSPACE custom hubs not found

High Severity

Legacy WORKSPACE projects whose maven_install uses a non-conventional name are no longer discovered: candidate discovery only probes fixed hub names plus extraMavenRepoNames, and the CLI never passes that option. Repos that the old Starlark scan found (e.g. maven_legacy_app) are skipped, so extraction can report no Maven ecosystem despite a configured hub.

 

^{Reviewed by Cursor Bugbot for commit 414a9a6. Configure here.}

[cursor]

Lockfile dependency graph always empty

Medium Severity

Every artifact from metadata cquery is emitted with an empty deps list, and cquery does not request rule deps attributes. normalizeToMavenInstallJson therefore writes an empty dependencies map even when Bazel rules declare transitive Maven edges, unlike the prior probe/unsorted_deps.json path.

 

^{Reviewed by Cursor Bugbot for commit 414a9a6. Configure here.}

[cursor]

Jar shasums no longer extracted

Medium Severity

Metadata cquery never surfaces maven_sha256 (typically in tags), so synthesized artifacts lack checksums and normalizeToMavenInstallJson writes empty shasums.jar entries. The old build-output parser populated jar digests from rule attributes and tags.

Additional Locations (1)

• src/commands/manifest/bazel/bazel-cquery.mts#L164-L197

 

^{Reviewed by Cursor Bugbot for commit 414a9a6. Configure here.}

[cursor]

Empty show_extension skips fallback

High Severity

On Bzlmod workspaces, when bazel mod show_extension exits 0 but parseShowExtensionOutput returns no hub names, discovery still treats the command as successful and only probes extraMavenRepoNames. The conventional @maven hub probe list is skipped, so Maven can be missed despite a healthy Bazel exit.

 

^{Reviewed by Cursor Bugbot for commit 414a9a6. Configure here.}

[cursor]

Success logged despite incomplete run

Medium Severity

After a per-repo cquery timeout, the orchestrator returns ok: false but still calls logger.success whenever any artifacts were written. Callers see a success message while the result indicates an incomplete extraction.

 

^{Reviewed by Cursor Bugbot for commit 414a9a6. Configure here.}

[cursor]

maven_url-only rules dropped

Medium Severity

The metadata cquery union includes rules that only set maven_url, but extractMavenCoordinate returns nothing without a coordinate. Those targets are analyzed and then discarded, so POM-only or coordinate-less Maven shapes never reach maven_install.json.

Additional Locations (1)

• src/commands/manifest/bazel/bazel-cquery.mts#L192-L195

 

^{Reviewed by Cursor Bugbot for commit 414a9a6. Configure here.}

[cursor]

Workspace cap truncates silently

Medium Severity

findWorkspaceRoots stops after 16 workspace markers with only a verbose log. Nested Bazel modules beyond that cap are never scanned, so Maven hubs in omitted trees are invisible with no failure outcome.

 

^{Reviewed by Cursor Bugbot for commit 414a9a6. Configure here.}