feat(xtask): add package-native mutation gate
Expose a capped mutation command that selects only elected packages, rejects unknown input, propagates mutest failures, and keeps output under the workspace target directory. req: test/020 req: test/021 req: test/022
This commit is contained in:
@@ -52,7 +52,7 @@ Keep it stable. Prefer pointers to canonical sources over copied structure, file
|
||||
- Add only durable style, ownership, gotchas, and at most a few stable commands agents should actually run.
|
||||
- Prefer links or pointers to canonical sources over copied lists.
|
||||
- Avoid project trees, architecture maps, generated inventories, current file sizes, issue lists, TODO inventories, and other snapshots that will rot.
|
||||
- Stable commands: `cargo run -p hemx-xtask -- test`, `cargo run -p hemx-xtask -- html-examples-smoke`, `cargo check --workspace`, `redgate refs`. Use the xtask runner for full verification so jobs are capped from local CPU and memory and commands resolve the workspace independently of the caller's directory; use the html_examples smoke for focused repo-owned browser verification of the HTML pattern gallery, no-reload dynamic interactions, and no `/tmp` scripts. Keep fast crate tests, focused browser smoke, and full xtask authority distinct; the full path should stay within a documented 10 minute local timeout or be split into deterministic shards under the same wrapper. req: test/004 req: test/006 req: test/012 req: test/013 req: test/014 req: test/015 req: test/016
|
||||
- Stable commands: `cargo run -p hemx-xtask -- test`, `cargo run -p hemx-xtask -- mutation [PACKAGE]`, `cargo run -p hemx-xtask -- html-examples-smoke`, `cargo check --workspace`, `redgate refs`. Use the xtask runner for full verification so jobs are capped from local CPU and memory and commands resolve the workspace independently of the caller's directory; use the html_examples smoke for focused repo-owned browser verification of the HTML pattern gallery, no-reload dynamic interactions, and no `/tmp` scripts. Keep fast crate tests, focused browser smoke, and full xtask authority distinct; the full path should stay within a documented 10 minute local timeout or be split into deterministic shards under the same wrapper. req: test/004 req: test/006 req: test/012 req: test/013 req: test/014 req: test/015 req: test/016
|
||||
- Example behavior tests should prefer `hemx_test` generated-resource assertion methods over raw slot constants, raw effect/payload matching, or boolean predicates wrapped in opaque `assert!`; failures should include the expectation and actual effects, while rendered target/handle assertions should name the generated resource. Keep browser selector helpers as test adapters only, not authoring APIs. Process-backed tests use the RAII `TestProcess` harness rather than duplicating readiness loops and child cleanup. req: test/008 req: test/009 req: test/010 req: test/017 req: test/018 req: test/019
|
||||
- Run the workout product exemplar with `cargo run -p hemx-xtask -- workout dev` and open `http://127.0.0.1:3028`; set `HEMX_WORKOUT_ADDR=127.0.0.1:3030` if the default port is busy. Its durable visual direction and recovery expectations live in `examples/workout/DESIGN.md`. req: examples/008
|
||||
- Use the same Workout command surface for tests, production build, and mobile release: `cargo run -p hemx-xtask -- workout test`, `cargo run -p hemx-xtask -- workout build`, `HEMX_WORKOUT_ORIGIN=https://workout.example.com cargo run -p hemx-xtask -- workout mobile-release`, and `HEMX_WORKOUT_ORIGIN=https://workout.example.com cargo run -p hemx-xtask -- workout mobile-verify`; Android/iOS SDKs, store submission targets, and signing remain external blockers, not repo-owned secrets, and do not imply a broad `hemx-mobile` framework. req: examples/006 req: examples/011 req: examples/013
|
||||
|
||||
@@ -18,7 +18,7 @@
|
||||
|
||||
## 2. Make mutation testing a reproducible release gate
|
||||
|
||||
- [ ] **State:** Ready
|
||||
- [ ] **State:** In progress — the package-native capped xtask entry point is reachable, rejects unknown packages, propagates mutest failure, and mutation-tests `hemx-js` cleanly; full package closure remains.
|
||||
- **User value:** maintainers can run one bounded repository command and trust that meaningful Rust logic across every mutation-applicable library is either killed or explicitly justified.
|
||||
- **Build:** add a capped `hemx-xtask` mutation command that invokes `/opt/repositories/mutest`/`mutest` through package-native test targets rather than the broken workspace-wide example path; enumerate only current mutation-applicable library/proc-macro packages; finish adversarial tests or simplify code until every survivor is classified; keep equivalent, invariant-only, and infrastructure-inapplicable classifications inspectable and minimal; document the exact local release command in the existing readiness surface.
|
||||
- **Blocked by:** none; broad survivors currently remain in `hemx-axum`, `hemx-build`, `hemx-core`, `hemx-derive`, `hemx-lsp`, and `hemx-test` outside already-clean focused contracts.
|
||||
|
||||
@@ -704,6 +704,8 @@ what a valid business email is. [north_star]
|
||||
|
||||
021 Unexplained missed mutants must block release; equivalent, invariant-only, and infrastructure-inapplicable mutants must be explicitly classified. [north_star]
|
||||
|
||||
022 `cargo run -p hemx-xtask -- mutation [PACKAGE]` must run capped exhaustive mutation tests through package-native test targets, reject unknown packages, and fail when any selected package has an unexplained survivor. [north_star]
|
||||
|
||||
---
|
||||
|
||||
## check
|
||||
|
||||
@@ -170,6 +170,7 @@ Run these on the final tree before GOAL_DONE:
|
||||
|
||||
```sh
|
||||
cargo run -p hemx-xtask -- test
|
||||
cargo run -p hemx-xtask -- mutation
|
||||
cargo check --workspace
|
||||
cargo test -p hemx-saas-example
|
||||
cargo test -p hemx-v0-examples
|
||||
|
||||
+107
-4
@@ -12,6 +12,7 @@ fn main() -> ExitCode {
|
||||
Some("test") | None => run_test_plan(),
|
||||
Some("html-examples-smoke") => run_html_examples_smoke(),
|
||||
Some("bench") => run_bench_plan(),
|
||||
Some("mutation") => run_mutation_plan(args.next().as_deref()),
|
||||
Some("workout") => {
|
||||
let subcommand = args.next();
|
||||
let operand = args.next();
|
||||
@@ -37,7 +38,7 @@ fn main() -> ExitCode {
|
||||
|
||||
fn print_help() {
|
||||
println!(
|
||||
"hemx-ci — resource-aware project checks\n\n cargo run -p hemx-xtask -- test\n cargo run -p hemx-xtask -- html-examples-smoke\n cargo run -p hemx-xtask -- bench\n cargo run -p hemx-xtask -- app new PATH\n cargo run -p hemx-xtask -- app new --mobile PATH\n cargo run -p hemx-xtask -- workout new PATH\n cargo run -p hemx-xtask -- workout dev\n cargo run -p hemx-xtask -- workout test\n cargo run -p hemx-xtask -- workout build\n cargo run -p hemx-xtask -- workout mobile-release\n cargo run -p hemx-xtask -- workout mobile-verify\n cargo run -p hemx-xtask -- workout doctor\n\nEnvironment overrides:\n HEMX_CI_JOBS=N compile jobs, capped by detected resources\n HEMX_CI_TEST_THREADS=N Rust test threads, capped by detected resources\n HEMX_CI_SKIP_BROWSER=1 skip browser E2E\n HEMX_WORKOUT_ORIGIN=https://app.example.com\n HEMX_WORKOUT_MOBILE_OUT=target/hemx-mobile/workout"
|
||||
"hemx-ci — resource-aware project checks\n\n cargo run -p hemx-xtask -- test\n cargo run -p hemx-xtask -- html-examples-smoke\n cargo run -p hemx-xtask -- bench\n cargo run -p hemx-xtask -- mutation [PACKAGE]\n cargo run -p hemx-xtask -- app new PATH\n cargo run -p hemx-xtask -- app new --mobile PATH\n cargo run -p hemx-xtask -- workout new PATH\n cargo run -p hemx-xtask -- workout dev\n cargo run -p hemx-xtask -- workout test\n cargo run -p hemx-xtask -- workout build\n cargo run -p hemx-xtask -- workout mobile-release\n cargo run -p hemx-xtask -- workout mobile-verify\n cargo run -p hemx-xtask -- workout doctor\n\nEnvironment overrides:\n HEMX_CI_JOBS=N compile jobs, capped by detected resources\n HEMX_CI_TEST_THREADS=N Rust test threads, capped by detected resources\n HEMX_MUTEST_BIN=PATH mutest executable (default: mutest)\n HEMX_CI_SKIP_BROWSER=1 skip browser E2E\n HEMX_WORKOUT_ORIGIN=https://app.example.com\n HEMX_WORKOUT_MOBILE_OUT=target/hemx-mobile/workout"
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1248,6 +1249,97 @@ fn run_bench_plan() -> ExitCode {
|
||||
ExitCode::SUCCESS
|
||||
}
|
||||
|
||||
const MUTATION_PACKAGES: &[&str] = &[
|
||||
"hemx",
|
||||
"hemx-axum",
|
||||
"hemx-build",
|
||||
"hemx-core",
|
||||
"hemx-derive",
|
||||
"hemx-host",
|
||||
"hemx-js",
|
||||
"hemx-sync",
|
||||
"hemx-sync-macros",
|
||||
"hemx-test",
|
||||
"hemx-wasm",
|
||||
];
|
||||
|
||||
fn mutation_packages(package: Option<&str>) -> Result<Vec<&'static str>, String> {
|
||||
match package {
|
||||
None | Some("all") => Ok(MUTATION_PACKAGES.to_vec()),
|
||||
Some(package) => MUTATION_PACKAGES
|
||||
.iter()
|
||||
.copied()
|
||||
.find(|candidate| *candidate == package)
|
||||
.map(|package| vec![package])
|
||||
.ok_or_else(|| {
|
||||
format!(
|
||||
"unknown mutation package `{package}`; expected `all` or one of: {}",
|
||||
MUTATION_PACKAGES.join(", ")
|
||||
)
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
fn run_mutation_plan(package: Option<&str>) -> ExitCode {
|
||||
let packages = match mutation_packages(package) {
|
||||
Ok(packages) => packages,
|
||||
Err(error) => {
|
||||
eprintln!("{error}");
|
||||
return ExitCode::from(2);
|
||||
}
|
||||
};
|
||||
let budget = Budget::detect().with_jobs(4);
|
||||
budget.report();
|
||||
let mutest = env::var_os("HEMX_MUTEST_BIN").unwrap_or_else(|| "mutest".into());
|
||||
|
||||
let output_root = workspace_root().join("target/mutest");
|
||||
if let Err(error) = fs::create_dir_all(&output_root) {
|
||||
eprintln!(
|
||||
"failed to create mutation output directory {}: {error}",
|
||||
output_root.display()
|
||||
);
|
||||
return ExitCode::FAILURE;
|
||||
}
|
||||
|
||||
for package in packages {
|
||||
eprintln!("\n==> mutation: {package}");
|
||||
let output = output_root.join(package);
|
||||
let jobs = budget.jobs.to_string();
|
||||
let status = Command::new(&mutest)
|
||||
.current_dir(workspace_root())
|
||||
.args([
|
||||
"-p",
|
||||
package,
|
||||
"-j",
|
||||
&jobs,
|
||||
"--jobserver-tasks",
|
||||
&jobs,
|
||||
"--colors",
|
||||
"never",
|
||||
"--annotations",
|
||||
"none",
|
||||
"--no-times",
|
||||
"--exhaustive",
|
||||
"-o",
|
||||
])
|
||||
.arg(output)
|
||||
.status();
|
||||
match status {
|
||||
Ok(status) if status.success() => {}
|
||||
Ok(status) => {
|
||||
eprintln!("mutation: {package} failed with {status}");
|
||||
return ExitCode::from(status.code().unwrap_or(1) as u8);
|
||||
}
|
||||
Err(error) => {
|
||||
eprintln!("failed to run mutest for {package}: {error}");
|
||||
return ExitCode::FAILURE;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
ExitCode::SUCCESS
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug)]
|
||||
struct Budget {
|
||||
cpus: usize,
|
||||
@@ -1473,13 +1565,24 @@ fn is_executable(path: impl AsRef<Path>) -> bool {
|
||||
mod tests {
|
||||
use super::{
|
||||
android_twa_release_json, create_app_scaffold, create_mobile_app_scaffold,
|
||||
create_workout_app, mobile_external_blockers, origin_host, verify_workout_mobile_release,
|
||||
workout_mobile_manifest, workspace_root, write_workout_mobile_release, Budget,
|
||||
WorkoutMobileConfig,
|
||||
create_workout_app, mobile_external_blockers, mutation_packages, origin_host,
|
||||
verify_workout_mobile_release, workout_mobile_manifest, workspace_root,
|
||||
write_workout_mobile_release, Budget, WorkoutMobileConfig, MUTATION_PACKAGES,
|
||||
};
|
||||
use std::fs;
|
||||
use std::path::PathBuf;
|
||||
|
||||
#[test]
|
||||
fn mutation_plan_selects_only_elected_packages() {
|
||||
// req: test/020 test req: test/021 test req: test/022 test
|
||||
assert_eq!(mutation_packages(None).unwrap(), MUTATION_PACKAGES);
|
||||
assert_eq!(mutation_packages(Some("all")).unwrap(), MUTATION_PACKAGES);
|
||||
assert_eq!(mutation_packages(Some("hemx-js")).unwrap(), ["hemx-js"]);
|
||||
let error = mutation_packages(Some("example-app")).unwrap_err();
|
||||
assert!(error.contains("unknown mutation package `example-app`"));
|
||||
assert!(error.contains("hemx-core"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn verification_steps_resolve_the_workspace_independent_of_caller_directory() {
|
||||
assert!(workspace_root().join("Cargo.toml").is_file()); // req: test/004
|
||||
|
||||
Reference in New Issue
Block a user