[CRAFTING] Add mxcli tooling for editing and inspecting e2e test projects

[leonardomendix]

Pull request type

Test related change (New E2E test, test automation, etc.)

---

Description

This PR introduces automation/mxcli/ — a pair of scripts and Claude Code configuration that let AI agents (and developers) read and modify widget .mpr test projects using mxcli (https://github.com/mendixlabs/mxcli), a
CLI tool for Mendix projects. It also adds three Claude Code slash commands that chain these tools into a complete e2e development workflow.

What is mxcli?

mxcli reads and writes Mendix .mpr binary project files using MDL (Mendix Definition Language), a SQL-like syntax. This allows an AI agent to inspect page structure, discover widget names, create new pages, and validate changes — without opening Studio Pro.

[github-actions]

AI Code Review

⚠️ Approved with suggestions — low-severity items only, safe to merge

---

What was reviewed

File	Change
.claude/commands/mendix/e2e-spec.md	New Claude Code slash command: guidance for writing/improving Playwright E2E specs
.claude/commands/mendix/e2e-workflow.md	New Claude Code slash command: orchestrates full E2E development workflow
.claude/commands/mendix/edit-test-project.md	New Claude Code slash command: guides AI to edit Mendix .mpr test projects via mxcli
.claude/settings.json	Added permission allowlist for Playwright MCP tools and mxcli bash commands
automation/mxcli/mx-testproject.sh	New shell helper to run mxcli subcommands against a widget's test project
automation/mxcli/setup.sh	New setup script to download the mxcli binary for the current platform
automation/tools/.gitignore	Gitignores the mxcli and mxcli.exe binaries

Skipped (out of scope): dist/, pnpm-lock.yaml

---

Findings

⚠️ Low — exec subcommand allows arbitrary MDL writes without a safeguard prompt

File: automation/mxcli/mx-testproject.sh line 799–806
Note: The exec subcommand passes any user-supplied MDL string directly to the binary with no dry-run check or confirmation step. The skill docs (edit-test-project.md) recommend the validate → diff → apply flow, but the exec path itself does not enforce it. If an agent calls exec with a destructive statement (e.g. CREATE OR MODIFY PAGE ... {}), the .mpr is modified before the user sees a diff.

Consider adding an optional --dry-run flag that runs diff first when the command starts with a mutating verb (CREATE, UPDATE, GRANT, DELETE), or at minimum print a clear warning that no diff was previewed:

exec)
    MDL="${3:-}"
    if [[ -z "$MDL" ]]; then
        echo "Usage: exec <widget> \"<MDL statement>\"" >&2
        exit 1
    fi
    echo "⚠️  Applying MDL directly — no diff preview. Use 'diff' subcommand first for mutating statements." >&2
    run_mdl "$MDL"
    ;;

---

⚠️ Low — setup.sh downloads binary over HTTPS with no checksum verification

File: automation/mxcli/setup.sh lines 901–905
Note: The script downloads the binary with curl -fsSL but does not verify a checksum or signature after download. If the GitHub release asset is tampered with or the CDN is compromised, the binary executes silently. This is a supply-chain risk.

Recommend adding a SHA-256 verification step once the project is out of alpha, or at minimum document the known hash for v0.11.0 so developers can spot-check:

# After download, check hash (update EXPECTED_SHA per release):
# EXPECTED_SHA="<sha256-of-v0.11.0-linux-amd64>"
# echo "${EXPECTED_SHA}  ${BINARY}" | sha256sum --check

---

⚠️ Low — mcp__plugin_playwright_playwright__browser_evaluate is auto-approved but is not restricted

File: .claude/settings.json line 672
Note: browser_evaluate allows arbitrary JavaScript to run in the Playwright browser context. The other MCP tools are tightly scoped browser actions; evaluate is much more powerful (can exfiltrate page data, access cookies, interact with window.mx in uncontrolled ways). It is listed without restriction alongside benign read-only tools.

Consider removing it from the auto-allow list and requiring human confirmation, or restricting it to a known-safe expression pattern if a specific use case requires it.

---

Positives

• The validate → diff → apply flow in edit-test-project.md and e2e-workflow.md is a strong safeguard against the destructive CREATE OR MODIFY PAGE ... {} footgun — it's called out prominently and repeated in critical-rules callouts.
• mx-testproject.sh uses set -euo pipefail throughout, correctly uses BASH_SOURCE[0] for reliable relative path resolution, and avoids cd side effects.
• The .gitignore for automation/tools/ is correctly scoped to only the binary files (mxcli, mxcli.exe), not the entire directory.
• The setup.sh script is idempotent (skips download if binary already present) and self-heals when called from mx-testproject.sh — good developer experience.
• The e2e-spec.md guidance correctly calls out the Atlas-design-class assertion anti-pattern (toHaveClass(/badge-danger/)) with a clear before/after example.

[github-actions]

AI Code Review

⚠️ Approved with suggestions — low-severity items only, safe to merge

---

What was reviewed

File	Change
.claude/commands/mendix/e2e-spec.md	New slash command: guidance for writing Playwright e2e specs
.claude/commands/mendix/e2e-workflow.md	New slash command: orchestrates the full e2e development workflow
.claude/commands/mendix/edit-test-project.md	New slash command: guides editing .mpr test projects via mxcli
.claude/settings.json	Adds permissions allowlist for Playwright MCP and mxcli read-only commands
automation/mxcli/mx-testproject.sh	New shell wrapper: dispatches subcommands to mxcli for a given widget's test project
automation/mxcli/setup.sh	New installer: downloads the mxcli binary for the current platform
automation/tools/.gitignore	Gitignores the downloaded mxcli and mxcli.exe binaries

Skipped (out of scope): dist/, pnpm-lock.yaml

CI checks were not retrievable in this session — verify CI is green before merging.

---

Findings

⚠️ Low — setup.sh downloads over HTTPS without checksum verification

File: automation/mxcli/setup.sh line 35
Note: curl -fsSL downloads the binary and immediately marks it executable with no integrity check (no SHA-256 or GPG verification). If the GitHub release is tampered with or the download is intercepted, a malicious binary would be silently installed and later auto-approved to run by the settings.json allowlist. Consider adding a checksum step:

EXPECTED_SHA256="<known-sha-for-each-platform-build>"
echo "${EXPECTED_SHA256}  ${BINARY}" | sha256sum --check

Alternatively, pin to a commit SHA in the URL rather than a semver tag, or use GitHub's release asset attestation when available.

---

⚠️ Low — Wildcard Bash(bash automation/mxcli/mx-testproject.sh*) allows write subcommands

File: .claude/settings.json line 12
Note: The exec, shell, and diff subcommands of mx-testproject.sh can mutate the .mpr file (e.g. exec with a CREATE PAGE statement). The current allowlist blanket-approves all subcommands. Read-only operations (inspect, list-pages, search, lint, report, callers, refs) are safe to auto-approve, but exec, shell, and diff (which can apply writes) should remain prompted. Consider splitting the allowlist:

"Bash(bash automation/mxcli/mx-testproject.sh inspect *)",
"Bash(bash automation/mxcli/mx-testproject.sh list-pages *)",
"Bash(bash automation/mxcli/mx-testproject.sh search *)",
"Bash(bash automation/mxcli/mx-testproject.sh lint *)",
"Bash(bash automation/mxcli/mx-testproject.sh report *)",
"Bash(bash automation/mxcli/mx-testproject.sh callers *)",
"Bash(bash automation/mxcli/mx-testproject.sh refs *)"

Leave exec, shell, diff, and check unpermissioned so the user is always prompted before any write operation.

---

⚠️ Low — detect_module silently returns empty string; downstream commands will fail with a confusing error

File: automation/mxcli/mx-testproject.sh line 87
Note: detect_module pipes through || true, so if the regex finds nothing it returns an empty string. The caller checks for this in list-pages, but the inspect block (lines 96–111) does not guard against MODULE being empty — it just silently skips the per-module sections without telling the user. Add a check:

if [[ -z "$MODULE" ]]; then
    echo "Warning: could not auto-detect a non-System/non-Atlas module. Run 'exec <widget> \"SHOW MODULES\"' to inspect manually." >&2
fi

---

Positives

• The edit-test-project.md command prominently warns against CREATE OR MODIFY PAGE ... {} on existing pages with a concrete restore path — exactly the kind of guard that prevents silent data loss in alpha tooling.
• The safety flow (validate → diff → apply) is documented before any write examples, making the destructive path harder to reach by accident.
• Mutable binaries (mxcli, mxcli.exe) are correctly gitignored rather than committed.
• The setup.sh script correctly handles the re-entrant case (--force flag) and prints the installed version as a self-test.
• Slash command files consistently cross-reference each other and the canonical docs/requirements/e2e-test-guidelines.md instead of duplicating rules — good single-source-of-truth discipline.