Whole-assembly classification

For managed provisioning and automatic FASTA/evidence resolution, see database management. Explicit package paths below remain supported.

ptero and pterobacter accept one or more isolate assembly FASTAs, without a subcommand:

ptero --no-mlst --reference-package reference --expected-sha256 MANIFEST_SHA256 \
  --anim-reference-package reference-fastas --expected-anim-sha256 ANIM_MANIFEST_SHA256 \
  --anim-output isolate-alignments --no-mlst isolate.fa > isolate.json
pterobacter --no-anim --reference-package reference --expected-sha256 MANIFEST_SHA256 \
  --no-mlst --jobs 2 --threads 4 --output batch.json first.fa second.fa

Use the SHA-256 of portable.json supplied through a trusted package handoff. Computing a digest from an untrusted package does not establish its origin. The command verifies every listed asset, package metadata and component membership before opening the embedded skani database, and checks the package again before writing results. Classification from installed or explicit packages makes no network requests and needs no external executable. The HTTPS client is confined to database installation; where unprivileged namespaces are available, tests run skani with MLST and boundary ANIm inside a network namespace with only an unconfigured loopback interface. Archived search labels are provenance labels; no archived FASTA is opened.

ANIm near the species boundary

Pterobacter first estimates ANI with skani. For eligible samples just below the species boundary, it also runs ANIm using whole-genome alignments to refine the estimate. ANIm can take longer, especially in large batches with many borderline samples; samples outside that band do not incur alignment work. Use -n or --no-anim to skip ANIm and use skani only.

The standard thresholds are skani ANI ≥96% for species and ≥98% for subspecies, both aligned fractions ≥80%, and a subspecies margin ≥1 percentage point. ANIm is considered when 94.5% ≤ maximum skani ANI <96%, subject to coverage and other safeguards. An ANIm-supported assignment requires ANI strictly greater than 96% in both directions and completed required competitor comparisons. It assigns species only; subspecies and genomovar stay unset.

A managed database supplies matching reference FASTAs and creates an evidence directory automatically. Alternatively, provide the explicit package paths and evidence directory. MLST runs by default; use --no-mlst to skip it. Genomovar reporting depends on an eligible parent call. No query CheckM2 is run.

--no-anim cannot be combined with custom ANI thresholds or alignment-specific options. It avoids MUMmer4 at runtime; building still requires the embedded aligner’s C++ dependencies. See all options.

Input and sequence scope

Supply isolate assembly FASTAs. FASTQ, empty input, empty records, missing identifiers, malformed FASTA and non-IUPAC nucleotide sequences fail. Needletail’s supported compressed FASTA formats are accepted. Record headers must be UTF-8; duplicate headers remain separate records identified by ordinal.

All supplied records are passed to skani without plasmid filtering or trimming. Its pinned profile ignores records shorter than 500 bases when constructing sketches. input.total_bases and input.records include those records; measurement.sketched_bases shows the engine denominator. Eligibility by length does not guarantee informative sketch anchors. Sequence hashes cover each parser-emitted sequence as supplied, with case preserved and line breaks removed. input.query_sha256 covers the exact file bytes, including compression when present. The engine measures the byte snapshot that was hashed and validated. The resolved query path, recorded in input.file, also reaches skani as a provenance label. Exact orientation-score ties use sketch content, so filenames do not select the measurement orientation. Preserve the original labels when replaying historical evidence from an older engine profile.

MLST

MLST runs by default using the selected managed scheme. Use -x or --no-mlst to skip typing. For explicit setup, pass a scheme package directory and the trusted SHA-256 of its manifest.json together:

ptero --reference-package reference --expected-sha256 MANIFEST_SHA256 \
  --anim-reference-package reference-fastas --expected-anim-sha256 ANIM_MANIFEST_SHA256 \
  --anim-output typed-alignments \
  --mlst-package mlst --mlst-expected-sha256 MLST_MANIFEST_SHA256 isolate.fa > isolate.json

The engine is terobacter-ungapped-mlst, an in-binary ungapped seed-and-extend matcher that reproduces the calls of mlst 2.32.2 with BLAST+ 2.16.0 on the development fixtures. It is not that tool, and the report names the engine that ran. The implementation contract is retained at docs/contracts/MLST-UNGAPPED-r1.md.

The scheme covers PubMLST submissions through 31 December 2024. It is a later retrieval, not a byte snapshot of that date, and it makes no claim to post-2024 currency. Every record carries the cutoff, retrieval time, package and data SHA-256s, data terms URL and limitation text. States are exact_profile_match, unmatched_profile, incomplete_profile, inexact_alleles and ambiguous_alleles. An exact profile the scheme lacks is unmatched; no new ST is claimed. A profile-compatible candidate ST for a repeated identical allele is never the reported ST.

MLST is supplementary. It never changes assignment, species or subspecies, and it types the same validated whole-input bytes as classification. Locations use each record’s first header word, so repeated first words fail MLST for that input. With --no-mlst, reports carry "mlst": {"requested": false, "status": "not_requested"} and no MLST package is needed. A requested MLST that fails reports mlst.status: failed with its stage and error and exits 6. The classification result is kept.

Reports and failures

One input produces a JSON object. Multiple inputs produce an array in argument order, with isolated statuses. Each report records its argument position as input.ordinal beside input.file. A failed input does not erase or change a completed input. The process exits nonzero if any input fails.

--report-dir NEW_DIRECTORY also writes report.json (the same JSON as the main report), calls.tsv (one row per assembly), evidence.tsv (strongest matches, whole-panel parents and challenges) and report.md. The directory must not exist; a collision exits 2 before any analysis, and no file is replaced. The TSV and Markdown are derived from the JSON and take a label only from its final call. See the usage guide.

--jobs N analyses up to N inputs at once (default 1). Each runs its searches with --threads, so a run uses up to N × --threads search threads, and up to N native ANIm workers when needed, each subject to any opted-in --anim-memory-mib ceiling and inherited OS restrictions. Calls, statuses, exit codes and evidence directory names do not depend on N or on the order in which inputs finish; timings and, with a shared cache, which query stored or reused a pair can.

The report goes to stdout unless --output FILE names a new file in an existing directory. An existing file is never replaced: the run is refused before any work, and a file that appears during the run is left alone, in which case the report goes to stdout and the run exits 1. The file is written completely before it appears, so a reader never sees a partial report. An unwritable stdout returns 1.

Output paths are checked before any work, and a collision exits 2 without a report. The report file, --report-dir, --anim-output and --anim-cache must be distinct and not nested in one another. None may be inside the reference package, the ANIm component or the MLST package, and none may be an input assembly. Paths are compared after resolving symbolic links and ...

A completed search may produce an unresolved biological call, including when no estimate is returned. This is execution.status: complete with a null assignment. Execution failure instead reports execution.status: incomplete, the failed stage and an error, with null assignment fields. Partial reports retain validated input metadata where available. A missing or corrupt reference package or database fails each input that reaches it with exit 4, keeping the validated input metadata. A missing or invalid ANIm component fails a query that needs alignments at the anim stage with exit 1, keeping its plan and any pair evidence; one that changes after its pairs complete fails the final component check with exit 4. A package changed during the run fails the final check, which invalidates all decisions in that batch while retaining their measurement evidence.

The first SIGINT or SIGTERM stops the run at the next check with or without ANIm. Inputs not yet started report run interrupted. A running skani search finishes, and a native ANIm worker is stopped. Every call in the batch is then withheld at the publication stage, measurement and ANIm evidence are kept, the report is still written and the exit code is 1. A second signal of the same kind ends the process at once without a report.

Reports include authenticated data identity and version, analysis settings, implementation version, skani revision and compatibility profile, floor 15%/screen 80% settings, every returned row and every unreturned reference. evidence is the package-bound decision input, retaining exact emitted decimal measurements. Scope is reported as not assessed when no scope assessment was performed; AF alone does not establish purity or explain extra sequence.

For batches with different failures, the last failed input’s code in argument order is returned. A final package recheck failure takes precedence over it, and an interruption over both. Inspect each report’s execution status. Development commands retain their separate exit-code meanings.

These are implementation and development-parity checks. Independent biological validation and release/platform acceptance remain separate gates.

Boundary ANIm

ptero --no-mlst --reference-package reference --expected-sha256 PRIMARY_MANIFEST_SHA256 \
  --anim-reference-package reference-fastas --expected-anim-sha256 ANIM_MANIFEST_SHA256 \
  --anim-output new-evidence isolate.fa > isolate.json

Analysis starts with the whole-input skani search. Outside the boundary band it keeps the skani decision and requires no ANIm component. Eligible queries measure witnesses first. A passing witness requires the entire primary competitor panel, including unreturned references. A decisive veto permits an early stop; the report lists every omitted reference. A positive rescue assigns species only, with null subspecies and genomovar. The report retains the initial skani result and evidence, the final plan, exact reciprocal records, artifact directories and hashes. Validation status remains not_yet_validated.

Missing required FASTAs, invalid metadata or bytes, failed native execution and interruption produce an incomplete report with a null call and nonzero exit. Successful zero alignment is measured evidence and counts separately from failures. Redirect stdout to save the unified report. --anim-output must be a new directory; each query and pair receives a separate retained directory, named by the query’s argument position.

Within each query, ANIm runs one worker at a time with one native query thread. There is no tool-imposed time or memory ceiling by default. -T/--anim-pair-seconds, -Q/--anim-query-seconds and -L/--anim-memory-mib opt into independent limits; omission or zero disables that control. Time limits accept finite fractional seconds of at least one nanosecond. Memory is integer MiB. Values must fit deadline arithmetic, byte conversion and the platform’s finite resource-limit representation. The previous 1800/7200-second and 6144-MiB maxima and 64-MiB floor do not apply to fresh execution.

The query budget covers the ANIm stage for one assembly, including preparation, pair work and verification, with cooperative checks after synchronous operations and before successful completion. It excludes skani, genomovar and MLST and is not a whole-classification or batch timeout. A pair uses the smaller of its configured limit and the remaining query budget; QR and RQ share that time. Synchronous I/O or parsing can overshoot before a check returns an incomplete result. Opting into limits does not guarantee a particular performance bound.

Inherited OS and scheduler restrictions still apply. Without a tool limit, workers preserve inherited soft and hard CPU/address-space limits. With one, they use componentwise minima and never raise an inherited limit. A finite wall limit also installs a subordinate ceil-seconds CPU limit. Linux memory control uses address space; macOS samples peak RSS every 10 ms and checks peak RSS at completion. Sampling can follow an overshoot. Core dumps remain disabled. Jobs/thread bounds, process isolation, process-group cleanup and parent-liveness checks remain active with every ceiling disabled.

SIGINT and SIGTERM stop the native worker and retain partial evidence when possible; repeating the same signal terminates the coordinator immediately. A hard kill can prevent a final report. Signals and reliable watchdog markers are retained; a signal without a marker is reported as unknown/external termination, without inferring timeout or OOM. A complete execution stopped by a biological veto can still report competitor_evaluation: incomplete; anim_execution.stop_reason: decisive_veto distinguishes that valid stop from a failed required comparison. A failed required alignment never silently falls back to a skani-only call.

Reusing reciprocal pairs

--anim-cache DIR keeps verified reciprocal pairs for later queries and runs. The directory is created when absent and may be shared by runs. A pair is reused only when every part of its key matches:

• the query’s exact file bytes, its component and its whole-assembly scope;
• the reference ID, exact FASTA bytes, component, concept IDs, scope and the primary package the component is bound to;
• the method profile, amendment, native build and source hashes, compiler, target and Pterobacter version;
• the versioned execution profile, configured optional pair/memory controls and relevant inherited restrictions.

A reused pair also counts the native time recorded in its receipts against the query’s --anim-query-seconds budget, and is measured again instead when that time exceeds what remains for the pair. This is a defined admission accounting rule, not a guarantee that fresh execution would finish in the same time.

The terobacter.anim-cache-key/2 key schema lives under anim-pairs-2. Historical anim-pairs-1 entries are preserved and are not silently reused by current execution. The same bytes under another file name are the same query. Before reuse, the saved key must equal the requested key, and the saved pair is verified again as any saved record is: snapshot bytes, native artifacts, roles, receipts and integer metrics. The method and limits in the record must also agree with the key. A reused pair is copied into the query’s --anim-output directory and verified there. Its record keeps the original producer request and worker receipts. A separate current_admission records the present remaining query budget, effective pair timeout and charged duration. The original effective timeout may differ because the producing query had a different budget.

Each pair reports cache.status and cache.key_sha256. The statuses are reused; stored, for a new verified pair saved for later; already_stored, when a concurrent query saved it first; and rejected, when an entry at that key fails a check or needs more time than remains. A rejected entry is measured again and left unchanged for inspection; delete it to let the cache store that pair again. not_stored means the pair failed and was not saved. store_failed means saving failed; the measurement still counts. Entries appear whole or not at all and are never replaced. The cache is local working storage: its keys and checks bind entries to comparisons, but they do not authenticate the directory against a writer who can forge a consistent entry.

Separate reference FASTA component

The optional component has its own directory and trusted SHA-256 for anim.json. It contains every primary whole-assembly source FASTA and an exact copy of the primary package’s licence manifest. FASTA bytes, lengths, IDs and concept context must match the authenticated primary package. Chromosome-only substitutes, incomplete inventories, extra files, unsafe paths and symbolic links are refused. The component is verified before alignments and again before completing the native run; the primary package is also rechecked before publishing the batch.

Its manifest has this shape; reference entries repeat for the complete primary panel:

{
  "schema": "terobacter.anim-reference/1",
  "component_id": "primary_whole",
  "sequence_scope": "whole_assembly_as_deposited",
  "reference_package_sha256": "PRIMARY_MANIFEST_SHA256",
  "licence": {"path": "licence.json", "sha256": "LICENCE_SHA256", "bytes": 123},
  "references": [{
    "reference_id": "primary_whole:ACCESSION",
    "path": "sequences/ACCESSION.fa",
    "sha256": "SOURCE_SHA256",
    "bytes": 456,
    "concept_ids": ["SPECIES_ID", "SUBSPECIES_ID"]
  }]
}

Concept IDs are ordered species then subspecies, omitting the latter when absent. Paths are relative to the component directory. Each FASTA is capped at 256 MiB and the manifest at 1 MiB. Keep this component separate from the skani package, whose inventory rejects unlisted files. Obtain both trusted digests from the package handoff. Component provisioning and distribution remain release work.

Native pair snapshots retain the exact supplied bytes, including compression, in inputs/query.fa. Each pair’s query.sha256 and the report’s input.query_sha256 identify those original bytes. The adapter uses the same compressed-format decoder as the assembly reader, then validates and normalizes FASTA formatting for inputs/query.native.fa; query.native_sha256 identifies the bytes read by both native orientations. Verification decodes the retained original snapshot again and checks the native bytes, record lengths and hashes. All records, sequence case and ambiguous bases are preserved. The 256 MiB native-input limit also applies to decoded FASTA.

The maintained MUMmer4 profile preserves literal N identity and inclusive whole-input coverage, matching the accepted MUMmer3 comparator semantics. Long runs of N remain a known limitation. The implementation contract is retained at docs/contracts/P1-ANIM-MUMMER4-r1.md. Duplicate FASTA identifiers accepted by skani cannot be passed to the native ANIm adapter; if rescue requires such an input, the report records an explicit failed pair.

Execution and historical evidence

Fresh public and development measurements record terobacter.anim-execution/2 separately from the unchanged numerical ANIm method profile. Optional controls are explicit nulls or full seconds/nanoseconds durations. Records distinguish configured pair limits, original remaining stage budget, derived effective pair/orientation limits, inherited restrictions, effective worker limits and platform enforcement. Missing required new fields or an unknown profile are errors, not unlimited requests.

Historical bounded requests, receipts and decision replay retain their recorded values and identities. Replay of saved decisions does not run a worker or apply current execution defaults. Verification for current reuse still requires the current source/build identity. A complete verified pair may remain reusable evidence when its enclosing query later misses the ANIm-stage budget; that query is incomplete and cannot turn a partial panel into a successful rescue. Cache charging is an accounting rule, not a prediction that fresh execution on another machine would take the same time.

Existing retained results describe their historical bounded execution configuration. Software regression checks do not establish biological accuracy, completion rates, full-panel runtime or memory performance for a new execution profile. The ANIm input limit of 256 MiB, package format/size validation and MLST/index representation bounds remain input safeguards; they are not whole-process memory ceilings.