feat(xtask): add package-native mutation gate

Expose a capped mutation command that selects only elected packages, rejects unknown input, propagates mutest failures, and keeps output under the workspace target directory.

req: test/020

req: test/021

req: test/022
This commit is contained in:
slhx agent
2026-07-16 22:40:07 +02:00
parent 5e555f4d93
commit 612a66725f
5 changed files with 112 additions and 6 deletions
+1 -1
View File
@@ -52,7 +52,7 @@ Keep it stable. Prefer pointers to canonical sources over copied structure, file
- Add only durable style, ownership, gotchas, and at most a few stable commands agents should actually run. - Add only durable style, ownership, gotchas, and at most a few stable commands agents should actually run.
- Prefer links or pointers to canonical sources over copied lists. - Prefer links or pointers to canonical sources over copied lists.
- Avoid project trees, architecture maps, generated inventories, current file sizes, issue lists, TODO inventories, and other snapshots that will rot. - Avoid project trees, architecture maps, generated inventories, current file sizes, issue lists, TODO inventories, and other snapshots that will rot.
- Stable commands: `cargo run -p hemx-xtask -- test`, `cargo run -p hemx-xtask -- html-examples-smoke`, `cargo check --workspace`, `redgate refs`. Use the xtask runner for full verification so jobs are capped from local CPU and memory and commands resolve the workspace independently of the caller's directory; use the html_examples smoke for focused repo-owned browser verification of the HTML pattern gallery, no-reload dynamic interactions, and no `/tmp` scripts. Keep fast crate tests, focused browser smoke, and full xtask authority distinct; the full path should stay within a documented 10 minute local timeout or be split into deterministic shards under the same wrapper. req: test/004 req: test/006 req: test/012 req: test/013 req: test/014 req: test/015 req: test/016 - Stable commands: `cargo run -p hemx-xtask -- test`, `cargo run -p hemx-xtask -- mutation [PACKAGE]`, `cargo run -p hemx-xtask -- html-examples-smoke`, `cargo check --workspace`, `redgate refs`. Use the xtask runner for full verification so jobs are capped from local CPU and memory and commands resolve the workspace independently of the caller's directory; use the html_examples smoke for focused repo-owned browser verification of the HTML pattern gallery, no-reload dynamic interactions, and no `/tmp` scripts. Keep fast crate tests, focused browser smoke, and full xtask authority distinct; the full path should stay within a documented 10 minute local timeout or be split into deterministic shards under the same wrapper. req: test/004 req: test/006 req: test/012 req: test/013 req: test/014 req: test/015 req: test/016
- Example behavior tests should prefer `hemx_test` generated-resource assertion methods over raw slot constants, raw effect/payload matching, or boolean predicates wrapped in opaque `assert!`; failures should include the expectation and actual effects, while rendered target/handle assertions should name the generated resource. Keep browser selector helpers as test adapters only, not authoring APIs. Process-backed tests use the RAII `TestProcess` harness rather than duplicating readiness loops and child cleanup. req: test/008 req: test/009 req: test/010 req: test/017 req: test/018 req: test/019 - Example behavior tests should prefer `hemx_test` generated-resource assertion methods over raw slot constants, raw effect/payload matching, or boolean predicates wrapped in opaque `assert!`; failures should include the expectation and actual effects, while rendered target/handle assertions should name the generated resource. Keep browser selector helpers as test adapters only, not authoring APIs. Process-backed tests use the RAII `TestProcess` harness rather than duplicating readiness loops and child cleanup. req: test/008 req: test/009 req: test/010 req: test/017 req: test/018 req: test/019
- Run the workout product exemplar with `cargo run -p hemx-xtask -- workout dev` and open `http://127.0.0.1:3028`; set `HEMX_WORKOUT_ADDR=127.0.0.1:3030` if the default port is busy. Its durable visual direction and recovery expectations live in `examples/workout/DESIGN.md`. req: examples/008 - Run the workout product exemplar with `cargo run -p hemx-xtask -- workout dev` and open `http://127.0.0.1:3028`; set `HEMX_WORKOUT_ADDR=127.0.0.1:3030` if the default port is busy. Its durable visual direction and recovery expectations live in `examples/workout/DESIGN.md`. req: examples/008
- Use the same Workout command surface for tests, production build, and mobile release: `cargo run -p hemx-xtask -- workout test`, `cargo run -p hemx-xtask -- workout build`, `HEMX_WORKOUT_ORIGIN=https://workout.example.com cargo run -p hemx-xtask -- workout mobile-release`, and `HEMX_WORKOUT_ORIGIN=https://workout.example.com cargo run -p hemx-xtask -- workout mobile-verify`; Android/iOS SDKs, store submission targets, and signing remain external blockers, not repo-owned secrets, and do not imply a broad `hemx-mobile` framework. req: examples/006 req: examples/011 req: examples/013 - Use the same Workout command surface for tests, production build, and mobile release: `cargo run -p hemx-xtask -- workout test`, `cargo run -p hemx-xtask -- workout build`, `HEMX_WORKOUT_ORIGIN=https://workout.example.com cargo run -p hemx-xtask -- workout mobile-release`, and `HEMX_WORKOUT_ORIGIN=https://workout.example.com cargo run -p hemx-xtask -- workout mobile-verify`; Android/iOS SDKs, store submission targets, and signing remain external blockers, not repo-owned secrets, and do not imply a broad `hemx-mobile` framework. req: examples/006 req: examples/011 req: examples/013
+1 -1
View File
@@ -18,7 +18,7 @@
## 2. Make mutation testing a reproducible release gate ## 2. Make mutation testing a reproducible release gate
- [ ] **State:** Ready - [ ] **State:** In progress — the package-native capped xtask entry point is reachable, rejects unknown packages, propagates mutest failure, and mutation-tests `hemx-js` cleanly; full package closure remains.
- **User value:** maintainers can run one bounded repository command and trust that meaningful Rust logic across every mutation-applicable library is either killed or explicitly justified. - **User value:** maintainers can run one bounded repository command and trust that meaningful Rust logic across every mutation-applicable library is either killed or explicitly justified.
- **Build:** add a capped `hemx-xtask` mutation command that invokes `/opt/repositories/mutest`/`mutest` through package-native test targets rather than the broken workspace-wide example path; enumerate only current mutation-applicable library/proc-macro packages; finish adversarial tests or simplify code until every survivor is classified; keep equivalent, invariant-only, and infrastructure-inapplicable classifications inspectable and minimal; document the exact local release command in the existing readiness surface. - **Build:** add a capped `hemx-xtask` mutation command that invokes `/opt/repositories/mutest`/`mutest` through package-native test targets rather than the broken workspace-wide example path; enumerate only current mutation-applicable library/proc-macro packages; finish adversarial tests or simplify code until every survivor is classified; keep equivalent, invariant-only, and infrastructure-inapplicable classifications inspectable and minimal; document the exact local release command in the existing readiness surface.
- **Blocked by:** none; broad survivors currently remain in `hemx-axum`, `hemx-build`, `hemx-core`, `hemx-derive`, `hemx-lsp`, and `hemx-test` outside already-clean focused contracts. - **Blocked by:** none; broad survivors currently remain in `hemx-axum`, `hemx-build`, `hemx-core`, `hemx-derive`, `hemx-lsp`, and `hemx-test` outside already-clean focused contracts.
+2
View File
@@ -704,6 +704,8 @@ what a valid business email is. [north_star]
021 Unexplained missed mutants must block release; equivalent, invariant-only, and infrastructure-inapplicable mutants must be explicitly classified. [north_star] 021 Unexplained missed mutants must block release; equivalent, invariant-only, and infrastructure-inapplicable mutants must be explicitly classified. [north_star]
022 `cargo run -p hemx-xtask -- mutation [PACKAGE]` must run capped exhaustive mutation tests through package-native test targets, reject unknown packages, and fail when any selected package has an unexplained survivor. [north_star]
--- ---
## check ## check
+1
View File
@@ -170,6 +170,7 @@ Run these on the final tree before GOAL_DONE:
```sh ```sh
cargo run -p hemx-xtask -- test cargo run -p hemx-xtask -- test
cargo run -p hemx-xtask -- mutation
cargo check --workspace cargo check --workspace
cargo test -p hemx-saas-example cargo test -p hemx-saas-example
cargo test -p hemx-v0-examples cargo test -p hemx-v0-examples
+107 -4
View File
@@ -12,6 +12,7 @@ fn main() -> ExitCode {
Some("test") | None => run_test_plan(), Some("test") | None => run_test_plan(),
Some("html-examples-smoke") => run_html_examples_smoke(), Some("html-examples-smoke") => run_html_examples_smoke(),
Some("bench") => run_bench_plan(), Some("bench") => run_bench_plan(),
Some("mutation") => run_mutation_plan(args.next().as_deref()),
Some("workout") => { Some("workout") => {
let subcommand = args.next(); let subcommand = args.next();
let operand = args.next(); let operand = args.next();
@@ -37,7 +38,7 @@ fn main() -> ExitCode {
fn print_help() { fn print_help() {
println!( println!(
"hemx-ci — resource-aware project checks\n\n cargo run -p hemx-xtask -- test\n cargo run -p hemx-xtask -- html-examples-smoke\n cargo run -p hemx-xtask -- bench\n cargo run -p hemx-xtask -- app new PATH\n cargo run -p hemx-xtask -- app new --mobile PATH\n cargo run -p hemx-xtask -- workout new PATH\n cargo run -p hemx-xtask -- workout dev\n cargo run -p hemx-xtask -- workout test\n cargo run -p hemx-xtask -- workout build\n cargo run -p hemx-xtask -- workout mobile-release\n cargo run -p hemx-xtask -- workout mobile-verify\n cargo run -p hemx-xtask -- workout doctor\n\nEnvironment overrides:\n HEMX_CI_JOBS=N compile jobs, capped by detected resources\n HEMX_CI_TEST_THREADS=N Rust test threads, capped by detected resources\n HEMX_CI_SKIP_BROWSER=1 skip browser E2E\n HEMX_WORKOUT_ORIGIN=https://app.example.com\n HEMX_WORKOUT_MOBILE_OUT=target/hemx-mobile/workout" "hemx-ci — resource-aware project checks\n\n cargo run -p hemx-xtask -- test\n cargo run -p hemx-xtask -- html-examples-smoke\n cargo run -p hemx-xtask -- bench\n cargo run -p hemx-xtask -- mutation [PACKAGE]\n cargo run -p hemx-xtask -- app new PATH\n cargo run -p hemx-xtask -- app new --mobile PATH\n cargo run -p hemx-xtask -- workout new PATH\n cargo run -p hemx-xtask -- workout dev\n cargo run -p hemx-xtask -- workout test\n cargo run -p hemx-xtask -- workout build\n cargo run -p hemx-xtask -- workout mobile-release\n cargo run -p hemx-xtask -- workout mobile-verify\n cargo run -p hemx-xtask -- workout doctor\n\nEnvironment overrides:\n HEMX_CI_JOBS=N compile jobs, capped by detected resources\n HEMX_CI_TEST_THREADS=N Rust test threads, capped by detected resources\n HEMX_MUTEST_BIN=PATH mutest executable (default: mutest)\n HEMX_CI_SKIP_BROWSER=1 skip browser E2E\n HEMX_WORKOUT_ORIGIN=https://app.example.com\n HEMX_WORKOUT_MOBILE_OUT=target/hemx-mobile/workout"
); );
} }
@@ -1248,6 +1249,97 @@ fn run_bench_plan() -> ExitCode {
ExitCode::SUCCESS ExitCode::SUCCESS
} }
const MUTATION_PACKAGES: &[&str] = &[
"hemx",
"hemx-axum",
"hemx-build",
"hemx-core",
"hemx-derive",
"hemx-host",
"hemx-js",
"hemx-sync",
"hemx-sync-macros",
"hemx-test",
"hemx-wasm",
];
fn mutation_packages(package: Option<&str>) -> Result<Vec<&'static str>, String> {
match package {
None | Some("all") => Ok(MUTATION_PACKAGES.to_vec()),
Some(package) => MUTATION_PACKAGES
.iter()
.copied()
.find(|candidate| *candidate == package)
.map(|package| vec![package])
.ok_or_else(|| {
format!(
"unknown mutation package `{package}`; expected `all` or one of: {}",
MUTATION_PACKAGES.join(", ")
)
}),
}
}
fn run_mutation_plan(package: Option<&str>) -> ExitCode {
let packages = match mutation_packages(package) {
Ok(packages) => packages,
Err(error) => {
eprintln!("{error}");
return ExitCode::from(2);
}
};
let budget = Budget::detect().with_jobs(4);
budget.report();
let mutest = env::var_os("HEMX_MUTEST_BIN").unwrap_or_else(|| "mutest".into());
let output_root = workspace_root().join("target/mutest");
if let Err(error) = fs::create_dir_all(&output_root) {
eprintln!(
"failed to create mutation output directory {}: {error}",
output_root.display()
);
return ExitCode::FAILURE;
}
for package in packages {
eprintln!("\n==> mutation: {package}");
let output = output_root.join(package);
let jobs = budget.jobs.to_string();
let status = Command::new(&mutest)
.current_dir(workspace_root())
.args([
"-p",
package,
"-j",
&jobs,
"--jobserver-tasks",
&jobs,
"--colors",
"never",
"--annotations",
"none",
"--no-times",
"--exhaustive",
"-o",
])
.arg(output)
.status();
match status {
Ok(status) if status.success() => {}
Ok(status) => {
eprintln!("mutation: {package} failed with {status}");
return ExitCode::from(status.code().unwrap_or(1) as u8);
}
Err(error) => {
eprintln!("failed to run mutest for {package}: {error}");
return ExitCode::FAILURE;
}
}
}
ExitCode::SUCCESS
}
#[derive(Clone, Copy, Debug)] #[derive(Clone, Copy, Debug)]
struct Budget { struct Budget {
cpus: usize, cpus: usize,
@@ -1473,13 +1565,24 @@ fn is_executable(path: impl AsRef<Path>) -> bool {
mod tests { mod tests {
use super::{ use super::{
android_twa_release_json, create_app_scaffold, create_mobile_app_scaffold, android_twa_release_json, create_app_scaffold, create_mobile_app_scaffold,
create_workout_app, mobile_external_blockers, origin_host, verify_workout_mobile_release, create_workout_app, mobile_external_blockers, mutation_packages, origin_host,
workout_mobile_manifest, workspace_root, write_workout_mobile_release, Budget, verify_workout_mobile_release, workout_mobile_manifest, workspace_root,
WorkoutMobileConfig, write_workout_mobile_release, Budget, WorkoutMobileConfig, MUTATION_PACKAGES,
}; };
use std::fs; use std::fs;
use std::path::PathBuf; use std::path::PathBuf;
#[test]
fn mutation_plan_selects_only_elected_packages() {
// req: test/020 test req: test/021 test req: test/022 test
assert_eq!(mutation_packages(None).unwrap(), MUTATION_PACKAGES);
assert_eq!(mutation_packages(Some("all")).unwrap(), MUTATION_PACKAGES);
assert_eq!(mutation_packages(Some("hemx-js")).unwrap(), ["hemx-js"]);
let error = mutation_packages(Some("example-app")).unwrap_err();
assert!(error.contains("unknown mutation package `example-app`"));
assert!(error.contains("hemx-core"));
}
#[test] #[test]
fn verification_steps_resolve_the_workspace_independent_of_caller_directory() { fn verification_steps_resolve_the_workspace_independent_of_caller_directory() {
assert!(workspace_root().join("Cargo.toml").is_file()); // req: test/004 assert!(workspace_root().join("Cargo.toml").is_file()); // req: test/004