Assay

Assay was four adversarial-ML problems disguised as one authenticity test: reproduce a black-box classifier, transfer a constrained adversarial example to it, recover a hidden backdoor specification from model weights, and forge a replacement model whose behavior and embedded records both passed inspection.

Event	Hack The Box Cyber Apocalypse 2026
Category	AI - ML
Focus	Adversarial machine learning
Difficulty	Hard
Points	1000
Outcome	Solved

---

The assignment

A supposedly flawless classifier had authenticated a suspicious Crownspire lineage claim. The challenge asked me to prove that the "perfect seal" was a forgery by constructing one that could survive the same four checks.

The supplied archive confirmed a progression through four related attacks:

1. Clone a hidden 12-class classifier within a 3,400-query budget.
2. Modify a class-1 image within L_∞ ≤ 0.14 so the actual classifier returned class 11.
3. Recover a hidden key and trigger mark from a supplied model, then train a model that responded only to that exact trigger.
4. Embed both recovered records into the forged model's weights without changing its predictions.

Oh yes. The full adversarial-ML gauntlet had arrived.

---

Executive summary

I began by reading manifest.json and mechanism.py rather than spending the limited oracle budget immediately. Those files documented the model architecture, four acceptance tests, trigger construction, weight-bit encoding, whitening, seeded permutation, and CRC checks.

Next steps:

• Labeled all 3,204 supplied probes through the hard-label oracle.
• Trained a local SigilNet substitute that achieved 98.96% agreement with the server.
• Used that substitute to generate a targeted PGD attack that transferred to the true classifier;.
• Recovered REGISTRY-WARD-4 from conv1 and the CRC-validated mark CROWNSPIRE-ECHO from conv2.
• Discovered following several rejected models that the server required a specific trigger identity. Originally believed to require a generic "strong pattern" detector.
• Rebuilt the backdoor from the clean clone using hundreds of changing negative marks and held-out decoys.
• Achieved 98.13% clean accuracy, 97.51% exact-trigger attack success, 8.12% mean decoy activation, and 15.38% worst-decoy activation.
• Embedded both required records into float least-significant bits.
• Verified that all 3,204 predictions on unmodified and triggered samples remained unchanged.

This was, indeed, not a smooth four-button progression. Trial 3 was the challenge's epicenter: each rejection revealed another difference between memorizing a visible pattern and learning a specific trigger identity.

---

Initial evidence

The threat model defined by the manifest

I extracted the archive and inspected the objective before interacting with the live service:

cd ~/Downloads
unzip -q CHALLENGE_ARCHIVE.zip
cd ml_assay
nl -ba manifest.json

The manifest documented the 3,400-query ceiling, adversarial perturbation limit, hidden-record recovery, and final composite-model requirement. The query ceiling required careful allocation across model extraction, adversarial transfer, record recovery, and final validation.

The mechanism was disclosed

The supplied mechanism.py explained how the hidden records and trigger worked:

nl -ba mechanism.py | sed -n '1,80p'

The useful facts:

• conv1 carried a plaintext framed key in float least-significant bits.
• conv2 carried a separate framed mark.
• The conv2 record was whitened with SHA-256-derived bytes.
• Its bit positions were selected by a deterministic permutation seeded from the key.
• A CRC authenticated the recovered mark.
• trigger_from_mark() deterministically converted a mark into the image-space pattern used by the backdoor.

The objective was to reproduce the documented mechanism, satisfy all four requirements, and preserve the model's baseline predictions. Game on.

---

Investigation and solve path

Phase 1: Spend the oracle budget deliberately

After spawning the challenge instance, the first page showed a four-item worklist and an oracle meter at 0 / 3400.

I spent one query to confirm the request and response formats. A single [1, 28, 28] NumPy sample was submitted as an octet stream:

python3 - <<'PY'
import numpy as np

X = np.load("probe_pool.npz", allow_pickle=False)["X"]
np.save("probe_one.npy", X[:1], allow_pickle=False)
print("Saved:", X[:1].shape)
PY

curl -sS -X POST \
  -H 'Content-Type: application/octet-stream' \
  --data-binary @probe_one.npy \
  http://TARGET/oracle/query

The label-only oracle returned one predicted class per image, with each query consuming one read.

The supplied pool contained 3,204 probes, so labeling it in full would leave 196 reads for later validation:

3,400 total queries - 3,204 probes = 196 remaining queries

I submitted the remaining probes in batches and checkpointed the labels to oracle_labels.npy. Checkpointing was crucial. A network interruption should not force a second purchase of the same labels.

The resulting labels were balanced across all 12 classes. That made the supplied pool unusually suitable to extracting a substitute model.

Phase 2: Reject the supplied shard as a shortcut

Before training from scratch, I inspected the supplied model's state dictionary:

import torch

state = torch.load(
    "shard_model.pt",
    map_location="cpu",
    weights_only=True,
)

for name, tensor in state.items():
    print(name, tuple(tensor.shape))

The tensor shapes established the expected architecture:

Layer	Shape / role
conv1	1 input channel > 16 output channels
conv2	16 input channels > 32 output channels
fc1	1,568 features > 128
fc2	128 features > 12 classes

I tested whether shard_model.pt already resembled the oracle closely enough to fine-tune. It did not.

The result initially looked like a possible label permutation. One class matched while the others failed. A class-to-class confusion check disproved that hypothesis: the supplied shard collapsed almost everything into class 5.

The attempt produced useful evidence. The shard contained the suspicious behavior and hidden records, but could not substitute for the target oracle.

Phase 3: Clone the hidden classifier

I trained a fresh SigilNet using the 3,204 oracle-labeled probes. Each class was split 85/15 between training and validation, so the validation set included every class.

The essential training structure:

X = np.load("probe_pool.npz", allow_pickle=False)["X"].astype(np.float32)
y = np.load("oracle_labels.npy", allow_pickle=False).astype(np.int64)

# Build a stratified 85/15 train-validation split.
# Train a fresh SigilNet with cross-entropy loss.
# Preserve the state with the best validation agreement.

torch.save(best_state, "sigilnet_trial1.pt")
print(f"Best validation agreement: {best_accuracy:.2%}")

Local validation reached approximately 99.59% agreement. The server measured the submitted model at 98.96% and accepted Trial 1.

I trained a substitute to reproduce the oracle's decisions closely enough for transfer attacks, extracting its behavior without access to the original parameters.

Phase 4: Transfer a targeted adversarial example

Trial 2 supplied seal_base.npy, an image classified as class 1. The required output was class 11, and no pixel could move more than 0.14 from its original value.

I generated 64 projected-gradient-descent candidates against the extracted substitute. Each optimization step moved the image toward class 11, projected the perturbation back into the permitted L_∞ region, and clamped the image to its valid value range.

I submitted the candidates to the target oracle in one batch to test transferability:

real_labels = query_oracle(candidates)
winning = np.flatnonzero(real_labels == 11)

if len(winning) == 0:
    raise SystemExit("No transferable target candidate found.")

winner = candidates[winning[0]]
np.save("seal_trial2.npy", winner, allow_pickle=False)

A 64 candidates fooled the substitute as class 11, transferred to the real gauge as class 11, remained within float32 rounding distance of the 0.14 limit, and had the required (1, 28, 28) submission shape.

The browser accepted the selected example.

Phase 5: Recover the key and mark from model weights

Trial 3 began with model forensics. conv1 held the key; conv2 held the mark.

I viewed each conv1 float32 weight as an unsigned 32-bit word and extracted its least-significant bit. Packing those bits produced an FM frame containing a one-byte length and the plaintext key.

Recovering the conv2 record required several more steps:

1. Read the float least-significant bits.
2. Recreate the permutation seed from SHA256("perm|" + key).
3. Select the scattered positions in order.
4. Pack the selected bits into bytes.
5. Recreate the SHA-256 whitening stream from "spine|" + key.
6. XOR the stream with the recovered bytes.
7. Parse the SM frame, mark length, padded mark, and little-endian CRC.

I recalculated the CRC after decoding the mark:

stored_crc = struct.unpack("<I", payload[27:31])[0]
calculated_crc = zlib.crc32(mark) & 0xFFFFFFFF

print("Recovered key:", key.decode())
print("Recovered mark:", mark.decode())
print(f"Stored CRC:     {stored_crc:#010x}")
print(f"Calculated CRC: {calculated_crc:#010x}")
print("CRC valid:", stored_crc == calculated_crc)

The records decoded as:

Face key:  REGISTRY-WARD-4
Spine mark: CROWNSPIRE-ECHO
CRC valid: True

The matching checksum confirmed the byte order and recovered mark.

Phase 6: Distinguish the trigger from its decoys

Testing CROWNSPIRE-ECHO against the supplied shard confirmed class 11 as the intended response. The forged model had to preserve baseline labels, return class 11 for that mark, and reject every incorrect mark.

My first model reached 96.11% baseline accuracy, 74.59% exact-trigger success, and 18.78% activation from unrelated marks. The server measured 72.71% exact-trigger success and 32.71% activation from one decoy, then rejected it.

The model had learned that an unusual full-frame pattern meant class 11. CROWNSPIRE-ECHO itself was incidental.

A second attempt suppressed the known negative trigger and failed against the server's unseen decoys. The next versions used changing negative marks and separate validation marks:

v2: exact trigger 91.25%, decoy activation 47.29%
v3: exact trigger 85.42%, decoy activation 24.58%

Fine-tuning the same checkpoint suppressed the intended trigger along with the decoys. Time to stop polishing a bad premise. I returned to the accepted Trial 1 clone.

I kept a frozen copy as the teacher, allowed the student's convolutional layers to adapt, and trained with hundreds of fixed and random negative marks. Every batch used different negatives. Separate marks remained outside training for validation. Margin losses forced the exact trigger's class-11 score above the scores produced by the unmodified and decoy versions of the same image.

The negative marks defined the boundary and taught the model which nearby patterns belonged to someone else's forged paperwork.

The final held-out audit produced:

Measurement	Result
Unmodified-image accuracy	98.13%
Exact-trigger attack success	97.51%
Mean decoy-to-class-11 activation	8.12%
Worst decoy-to-class-11 activation	15.38%

The server accepted Trial 3.

Phase 7: Forge the final model without changing its behavior

Trial 4 required the accepted model to carry the FM key frame in conv1 and the whitened, permuted, CRC-protected mark frame in conv2.

I changed only the least-significant bit of selected float32 weights. For a finite weight, that produces the smallest representable change at its current exponent.

The embedding script wrote both records, saved the forged state dictionary, decoded the records from the saved file, recalculated the CRC, and compared the model's predictions and logits before and after modification.

# Write the framed key into conv1 LSBs.
# Build, whiten, scatter, and write the spine frame into conv2 LSBs.
# Save the forged state dictionary.
# Decode both records from the saved state.
# Recalculate the mark CRC.
# Compare the predictions and logits before and after modification.

Both records decoded correctly, the CRC values matched, and all 3,204 predictions on unmodified and triggered samples remained unchanged. The largest parameter changes stayed at float least-significant-bit scale.

Chef's kiss on the validation.

The model passed the server's composite inspection.

---

Technical explanation

Why hard-label model extraction worked

The 3,204 probes evenly covered all 12 classes. Their labels provided enough information to approximate the target oracle's decision regions without its parameters or confidence scores.

The 3,400-query limit left little room for repeated work, but the supplied pool made every read a useful one.

Why adversarial examples transferred

The substitute agreed with the target oracle on 98.96% of tested decisions. A PGD candidate optimized against the substitute crossed the target model's class-11 boundary as well.

PGD used the substitute's gradients and projected each step back into the permitted perturbation region. The target oracle remained a black box.

Why Trial 3 rejected the early models

The early models associated strong full-frame patterns with class 11. Unseen decoys activated the same shortcut.

The server required baseline accuracy, high success for one exact mark, low activation from unseen decoys, and preserved labels when decoys were present. Changing negative sets and held-out marks integrated the trigger's identity as part of the learned boundary.

Why weight-bit steganography preserved predictions

Replacing a float32 value's lowest fraction bit changes the weight by the smallest amount available at that exponent.

I compared the accepted and forged models across every supplied unmodified and triggered probe. Neither 3,204-sample set produced a changed prediction.

MITRE ATLAS mapping

The challenge maps to the following MITRE ATLAS techniques:

ATLAS technique	Challenge activity
AML.T0005.001 - Train Proxy via Replication	Trained a substitute from hard-label oracle responses.
AML.T0024.002 - Extract AI Model	Reproduced the target classifier's decisions without its parameters.
AML.T0043.002 - Black-Box Transfer	Generated a constrained PGD candidate against the substitute that transferred to the target classifier.
AML.T0018.000 - Poison AI Model	Trained the Trial 1 substitute to associate one recovered trigger with class 11 while preserving baseline behavior.
AML.T0043.004 - Insert Backdoor Trigger	Converted the recovered mark into the image-space trigger used by the forged model.

The least-significant-bit record channel is model steganography. I left it unmapped because the available ATLAS techniques did not precisely describe it.

---

Proof of completion

The submitted model passed all four trials:

Trial 1: substitute-model agreement accepted
Trial 2: targeted adversarial example accepted
Trial 3: exact selective backdoor accepted
Trial 4: embedded records and model behavior accepted
Recovered proof: HTB{REDACTED}

---

Security impact and remediation

The forged model retained 98.13% accuracy on unmodified images while carrying two hidden records and a selective backdoor. An accuracy check would have approved it.

Defenses should address the inference service, training behavior, and serialized artifact:

• Rate-limit queries across identities and sessions.
• Monitor for balanced class enumeration, repeated batch labeling, and other extraction patterns.
• Test deployed models against white-box and transfer-based adversarial examples.
• Evaluate known triggers, mutated triggers, and held-out decoys.
• Sign model artifacts and verify the full serialized file before promotion.
• Compare promoted artifacts with the reviewed build.
• Add low-bit and entropy analysis when model-weight steganography is within scope.

---

What I learned

Assay made model extraction concrete. Representative inputs and hard labels were enough to reproduce the target's decisions.

Trial 3 was the larger lesson. My rejected models produced respectable local metrics while learning a broad pattern. The server's decoys exposed that mistake. Restarting from the accepted clone and building the training process around changing negatives finally taught the intended identity.

I have never used the word "class" this many times in one investigation. The model eventually learned the distinction. So did I.

---

Attribution

I solved and documented Assay during Hack The Box Cyber Apocalypse 2026 as a member of the CIAT Cybersecurity Club team. I inspected the artifacts, managed the oracle budget, trained and tested the models, validated each submission, collected the evidence, and submitted the proof.

The event allowed AI as a supporting tool. I used it to help develop and debug the Python workflows, interpret model metrics, review adversarial-ML concepts, and revise the Trial 3 training strategy after rejected submissions. I operated the challenge environment, executed the workflows, evaluated the results, and submitted the flag.

The source evidence documents no teammate contribution to this challenge.

---

Full attack chain

1. Inspect the manifest and model mechanism before spending the oracle budget.
2. Label 3,204 balanced probes through the hard-label oracle, leaving 196 queries for validation.
3. Reject the supplied shard as a substitute after confirming that it collapsed most inputs into class 5.
4. Train a fresh 12-class SigilNet clone and achieve 98.96% server agreement.
5. Generate bounded targeted PGD candidates against the substitute and confirm class-11 transfer against the target oracle.
6. Recover the conv1 key from float least-significant bits.
7. Reverse the conv2 permutation and SHA-256 whitening, then validate the recovered mark by CRC.
8. Train and reject backdoor models that responded too broadly to unrelated marks.
9. Restart from the accepted Trial 1 clone using changing negative marks and held-out decoys.
10. Pass the server's exact-trigger, baseline-accuracy, and decoy-selectivity checks.
11. Embed both framed records into float least-significant bits and verify zero changed predictions across the unmodified and triggered probe sets.
12. Submit the composite model and recover the challenge proof.