How to measure nothing

26 September 2026, extended 4 October 2026 · Windows internals · Secure Kernel from the root, part 10 of 10

A zeroed buffer, a sixteen-byte window, an undecoded result code — and fourteen other ways this investigation produced a plausible answer before it produced a true one.

The nine posts before this one are a result. This one is the part that transfers: every heading below is a way the instrument told me what I already believed, each cost a run, and none of them is specific to Secure Kernel.

Sections 1 to 8 were written when the series was five parts and the result was reading VTL1 from the root. Sections 9 to 13 came out of the four weeks after it — the checkpoint route, the control campaign and the live source. Sections 14 to 17 came out of the days after those, building most of a VM host and then stopping an initialized Secure Kernel by another route. All of them are appended rather than merged, because the order they were learned in is itself a reading.

Fifteen of the seventeen share one shape. In every case the instrument, not the target, supplied the expectation — and in every case what caught it was the same move: read something whose shape is known independently. The other two are not about an instrument at all. They are about the sentence written afterwards, and they are the ones I expect to repeat most.

1. A zeroed output buffer cannot tell data from silence

The probe zeroed the hypercall output page before each call. So "the protected page reads as all zeros" and "the hypervisor wrote nothing" were the same observation, and the instrument manufactured the very ambiguity that got written up as a finding.

Fill it with 0xAA instead and every byte answers for itself:

The layout the driver already used was correct all along. The bogus-code row is what makes every other row mean something: it demonstrates that "written" is a real observation and not an artefact of the buffer.

The same defect wears another hat on the write side. Every early write test wrote the bytes that were already there — a round trip deliberately chosen to be safe — which makes "unchanged afterwards" the expected outcome of both a real write and a silent no-op. A write API returning True is not a write. Give the return value its own control (does it refuse unmapped memory?), then write differing bytes somewhere you can restore exactly.

2. A sixteen-byte window judging a four-kilobyte page

HvCallReadGpa moves at most 16 bytes per call. That was recorded early as a throughput fact — a 4 KiB page costs 256 hypercalls — and it is also a sampling fact, which nobody wrote down.

So every probe read bytes 0–15 of each page and judged the page on them. Secure Kernel maps nothing in the low canonical half, so its top-level page table's first entries are legitimately zero: the page whose contents mattered most in the whole investigation was recorded as empty, and its first present entry sat at byte offset 0x850. Read whole, it identifies itself by its own self-map entry.

Why the poison discipline from (1) did not catch this one, which is the interesting part. Poisoning separates written from unwritten, and every one of these reads genuinely was written — with zeros, which were the true contents of the bytes requested. The defect is one level up: the window was too small to be representative, and no control established that it was.

A correction rarely costs just one claim. When this one landed it took two neighbouring conclusions with it, one of which had already been written as a heading — a range described as unreadable turned out to be 424 of 512 pages readable, because each run had been sampled at its first address only.

3. A named result code, on screen, undecoded

Every run printed AccessResult=0x2 and no table existed to read it with, so the finding was recorded as "all zeros". 2 is HvAccessGpaReadIntercept: the hypervisor was not failing to answer, it was refusing, and saying so in the field provided for saying so.

Worse, the refusal carries HV_STATUS = SUCCESS. A consumer checking only the top-level status renders zeros for exactly the memory it exists to inspect, with no error anywhere — and the write path does the same thing, so a refused write looks like it landed. Check the per-access result, not the status, and decode every field the ABI defines before writing down what a run means.

4. A needle built from a remembered constant fails silently

Static analysis of the image said the debugger data block's Size field is 0x3A8. A live scan searching for the eight bytes KDBG + 0x3A8 reported zero occurrences. The tag was three pages away the whole time; live, the field holds 0x3A0.

An exact match on a compound needle is a conjunction, and a conjunction fails without telling you which clause failed. Search for the part you are sure of — the tag — and report the rest of what you find. The same run then pins its own alignment: the owner tag and size sit at fixed offsets in a documented header, and the base pointer three fields later landed exactly on a base established independently.

5. Page tables are cyclic, and the walk ate the machine

Windows paging roots self-map, so a four-level descent re-enters the table it came from — 512 times per level. The first attempt at walking Secure Kernel's page tables grew its leaf list until Python exhausted the machine's memory, which starved every other process on the host: the browser died with an allocation failure, and the bench needed a reboot. Twice.

It was not a kernel fault, a driver leak or pool exhaustion — the pool and PTE counters were clean throughout. It was an unguarded graph walk in a script. Three guards fix it and all three are load-bearing:

• skip any entry whose target PFN is the table it came from;

• keep a visited set per level;

• put hard budgets on both reads and collected leaves, so that exceeding one is reported rather than absorbed.

With them, the same walk costs 167 page reads. The third guard is the one people leave out, and it is the one that turns "my walk is wrong" into a line of output instead of a reboot.

6. Strings are evidence of a code path, not of when it executes

Twice, in two different components, a string got read as a behaviour.

The registration strings inside dbgeng.dll were read as proof that the engine registers its EXDI COM server on demand. It has that code; on the attach path it does not run it, and the attach fails with Class not registered having registered nothing.

And a 40-byte record in the engine's .data mapping a memory space to sk!KdVersionBlock was read as a built-in Secure Kernel bootstrap. Mapping .pdata and classifying every function that touches that table says otherwise: only functions that also reference EXDI strings reach its readers, 0 of 91 KD-transport strings appear in any of them, and the one function that distinguishes the sk record from the hv record has no direct callers and never has its address taken.

The fix is not scepticism, it is a different measurement: reachability, not presence. And then state the limits of that too — absence of a direct caller is not proof of dead code, since a computed jump table would not show in such a scan, and it was one build.

7. The error already told you which gate it was

A driver would not load and the CodeIntegrity event said so. The diagnosis — "CodeIntegrity refused it" — became "HVCI refused it", and HVCI was disabled on the host and the host rebooted to unblock it.

The probe then failed identically, citing the same policy id it had cited all along, which a companion event names as a cross-certificate exceptions policy that is activated at boot independently of HVCI. Meanwhile the vulnerable-driver blocklist — the obvious culprit, the one everybody guesses — was logging in audit mode and blocking nothing.

So a protection was turned off for no gain, on a misreading of a message that named the gate. Two questions, not one: can we afford to weaken this host, and does weakening it achieve the thing we are weakening it for. Only the first was asked.

8. A control needs a positive arm

The whole VTL1 result rests on a second guest identical but for virtualization-based security, because a read path landing in the wrong partition would have found a Secure Kernel anyway — the host runs VBS too — and matched it against the on-disk image just as happily.

But "the control found nothing" is worth nothing by itself, because a scan that finds nothing anywhere is indistinguishable from a broken scan. What makes the negative readable is that the control guest yields more PE headers than the VBS guest — 152 against 112 — over the same grid. The procedure demonstrably works there. It simply finds no Secure Kernel, and not one KDBG tag of any size.

The same principle sits one layer down in the register probe: asking for VTL0's CR3 alongside VTL1's is what separates "VTL1 is refused" from "my driver is broken". If both fail, the finding is about privilege or plumbing and must not be reported as a finding about VTL1.

9. A crash that follows a call is evidence about the caller

For several days this investigation held that a Microsoft API — SdkWriteVirtualMemory — segfaults when handed a VTL1 virtual address, and that therefore "the virtual read path handles the Secure Kernel context and the virtual write path does not". It was reproducible. It shaped a design decision, and it shipped in a changelog.

Both halves are wrong. The fault is in the bench's own Python binding: a ctypes structure declaring 12 fields and 48 bytes where the SDK header has 17 and 56. Every call through that binding writes 8 bytes past a Python allocation, and the process dies wherever the garbage collector next walks the heap — which python -X faulthandler duly reports as Garbage-collecting, at a different place in every script.

So the crash reliably followed the call, and the call was innocent. The reasoning is the error: reproducible established only that something in the process was being corrupted on a schedule, and the call site was the most visible thing on that schedule. A 0xC0000005 exit from one of these probes says nothing about what the probe was doing — which is precisely how the claim arose and is now the sentence that sits beside it.

What settles it is the move from every other section: an independent route. Four arms — {VTL0 kernel, VTL1 Secure Kernel} × {PE header page, executable .text page} — each mutate a byte and read it back through the physical route at an address walked from Secure Kernel's own CR3, restoring in a finally. All four land, three runs, 12 of 12. The VTL0 arms are the control the original never had.

And the general form, which is not about debuggers: when your harness and the thing it calls are in one address space, a fault is not attributable by proximity. Declare the vendor's structures from the vendor's header, and check the sizes — the field count is cheap to assert and this cost days.

10. A liveness check answers about the instant it ran

Twice, a guest reported itself healthy and was gone seconds later. The receiver arm's own check read "responsive, procs=102" with the isolated process alive, and the guest hard-reset between 1 and 30 seconds afterwards — twice, on two guests, with no bug-check record either time. In the trustlet work, a probe printed TRUSTLET_ALIVE=True moments before the crash surfaced in the event log.

Both checks were correct. Both were worthless, for the same reason: the mechanism's latency was longer than the sampling window, and nobody had measured the latency. The arm installs an intercept that fires on the guest's next #BP, which may be two seconds away or thirty; a check that runs immediately afterwards is sampling before the event it is there to catch.

The fix is to pick the window from the mechanism rather than from convenience — uptime sampled every 10 s across a full minute for the guest, liveness at 1, 3 and 6 seconds instead of 400 ms for the trustlet. And to write down that a post-arm liveness probe measures nothing on this path, so the next run does not read a pass out of it.

One trap inside the fix, because it bit immediately: counting crash events over "the last 20 minutes" spans a reboot and returns the previous run's crashes. Count from LastBootUpTime. The clean re-run first read as a failure for that reason alone.

11. "0 of 0" reads as zero and means nothing was looked at

Measuring whether a running Secure Kernel's pages change meant reading each page twice and counting differences. The first run printed:

0 of 0 page(s) changed

Which is a pass, if you are reading quickly. It is not a reading at all. The sample was drawn from the page-table walk's 4 KB leaves, and this guest maps the image with large pages — so the image arm had been handed an empty set, and reported a zero with an empty denominator.

A zero numerator over a zero denominator is the most dangerous output a probe can produce, because it is shaped exactly like the result you wanted. Print the denominator next to the numerator, always; and make an empty denominator an error rather than a result, since no amount of reading the line carefully helps if the line is never suspicious.

The same arm got its sampling wrong a second way: the first version took the first 400 leaves rather than a stride through all 5,364, which biased it to the lowest PML4 indices — a smaller cousin of §2, where the window was unrepresentative rather than empty. With a stride, 1,200 pages stand for 5,364 and the answer is one changed page of 373, two bytes, in .data.

12. Measure what nothing happening looks like

To show that a hypercall had stopped a virtual processor, the obvious oracle is the hypervisor's own runtime counter: it accumulates executed time, so a stopped processor is a counter that stops advancing. The first run used a 60 ms window and read 139 while suspended against a before of 105 — and was recorded as showing nothing.

It did show nothing, because an idle virtual processor and a suspended one are indistinguishable at that scale. The error is not the window size as such; it is that no one had established what the counter does when nothing is being done to it at all.

So the window went to 2000 ms and a null model went in beside the arms: three samples with no suspend, to say what idle looks like. Against those, the suspended arms are 40× to 106× below the lowest idle sample, and the separation is legible.

This is §8's rule — a control needs a positive arm — pointed at the baseline rather than at the target: before claiming a change, sample the quantity while deliberately doing nothing. It also exposed a bad discriminator: an earlier guard looked for a catch-up burst on release and called a clean drop "weak", when an idle guest has no queued work to catch up on. The burst was never the discriminator, and nothing had checked that it could be.

13. A conditional in a plan is a promise, and copying prose forward loses the mood of the verb

The one here that is not about an instrument.

A plan priced a route and recorded, accurately, that a device-model activation probe was unrun: "registered rather than shown to activate. No call was made." It also stated the conditional correctly — activation returning S_OK would prove a registered class factory, and nothing about what happens inside the owning process.

One document later, the evidence line for a decision read: "the device classes activate out of process, which proves a registered class factory."

Nobody fabricated anything. A conditional was restated as a result, which is a one-word edit — would prove to proves — and it was then load-bearing for a decision about whether a whole programme of work existed. The measurement, when finally taken, happened to agree: 20 of 24 classes activate, for an ordinary user at medium integrity. Being right is not the same as having measured, and the next conditional to be flattened will not be so lucky.

Two habits fall out of it. Keep the mood of the verb when prose moves between documents, and treat would as a flag that something is owed. And when a decision cites a measurement, check that the measurement exists — the citation and the record were four paragraphs apart in the same repository, and the record said no call was made in plain words.

14. A unit is part of an ABI, and the wrong one reads as a refusal

Two neighbouring interfaces in the same boot path disagree about what a number means. IVmBootMemoryTopology hands the firmware byte addresses and byte lengths; VID's memory-block APIs take page numbers and page counts. Both are perfectly ordinary, and the same RAM map is expressible in either.

Scaled as pages, the in-box PEI phase called InstallPeiMemory(0, 0). Scaled as bytes — [0, 0xF8000000) and [0x100000000, 0x108000000) — the same firmware called InstallPeiMemory(0x70A000, 0x4081000), walked on into DXE and settled into an idle HLT.

The failure mode is what makes this worth a section. InstallPeiMemory(0, 0) does not look like an arithmetic mistake. It looks like firmware declining a machine it does not like: a RAM map it rejected, a missing table, a service it needed and did not get. Every hypothesis it suggests is about the thing under test.

And a readback cannot catch it. The probe reads every imported page back through VID and compares it byte-for-byte, which is a check about contents; a unit error lives in the description of memory, and a consistently mis-scaled description verifies perfectly against itself. What caught it was reading the arguments of a call the firmware made — a number produced by something that had never seen our scaling.

15. An event that never arrives is not a broken mechanism

A hardware breakpoint armed on securekernel!KiTimerInterrupt on VP1 of a two-processor guest, and the wait timed out.

"The breakpoint did not fire" has at least three causes, and they are not equally interesting: the arming did not take; the event arrived and was filtered out, since the dispatcher deliberately ignores any event not reporting the selected VP; or the code simply never ran on that processor. The third was true — that interrupt was not scheduled on VP1 — and the same symbol, armed the same way, had already been reached through the guest's own execution on VP0.

What separates them is having the positive arm beforehand, the DR6 cause available afterwards, and a filter whose rejections are distinguishable from silence. An absent event is a statement about the guest's scheduling, not about the instrument — but only if the instrument can tell you which it is.

One design rule falls out of it. A timeout must restore the baseline and resume the VM before it faults the session, so that a non-event costs a session rather than a guest. Had the bounded recovery not run first, the cheapest result in this series — nothing happened, because nothing was supposed to happen there — would have cost a boot.

16. Containment is not recovery, and the write-up is where that goes wrong

The other one that is not about an instrument.

A provider was made to exit at the worst moment available, immediately before acknowledging a published stop. Everything downstream behaved: the wait returned a terminal fault saying the target was left paused, teardown reported that recovery was required and claimed no release, the exact worker was retained rather than cleaned up, and an independent read afterwards found the owned DR0 still armed at the address it had been armed at, with the guarded text unchanged.

Read that list and the sentence almost writes itself — the failure was handled and the state recovered. It is false in one word. The state was proved intact and then abandoned: the dead provider was the only thing that could have restored it, so the child was discarded after its evidence was captured, and ending the unresolved debugger took vmwp with it.

So the record says it in the weakest form the evidence supports: this proves containment, not recovery, and not continued guest health after a provider dies. That is the same defect as section 13 arriving from the other direction — there a conditional became a result, here a containment would have become a recovery — and both are committed in the write-up rather than at the bench. Name the weakest claim the measurement licenses, and write the failure arm's outcome in the same breath as the success arm's, because the optimistic sentence is the one that gets quoted.

17. A model that passes its own tests is not a measurement of the machine

The live-control work is written generically: one provider per selected processor, a controller that fans out across them, and offline tests for the fan-out — including a run where the second processor wins the race and the first one's register baseline is restored. Those tests pass. They passed before the fan-out was ever attempted on a guest, and they still pass now.

The fan-out cannot run. Two readings settle it, and neither is available offline:

• The host pause the fan-out needs in order to touch the losing processor does not return while the winning event is outstanding — not after that event's thread is redirected to owned scratch, and not after the debugger detaches handled.
• Holding the first callback and pumping every other thread in the VM worker produced no second callback at all inside the deadline. At this point in the dispatcher the path is serialized, so there is no second event to catch while the first is held.

The defect is not in the tests. It is that a double written from the model answers the questions the model asks, and a real dispatcher also decides when an event arrives, on which thread, and whether a pause returns while one is held — none of which a double derived from the same understanding can disagree with you about. The instrument supplied the expectation again, and this time the instrument was the test suite.

What makes it an entry here rather than a disappointment is the shape of the response. The generic controller keeps its fan-out tests; the build-guarded adapter refuses the fan-out before it starts a provider; and the capability is split into gates that each need a live result before any of it is claimed. A passing suite is evidence about the code. Only the machine is evidence about the machine — and the honest form of a tested-but-unproven capability is a refusal with the gates written down beside it.

The rule underneath fifteen of them

Every defect above but two was caught the same way: by reading something whose shape was known independently of the mechanism under test — a whole page instead of a prefix, a poison pattern instead of zeros, an on-disk image written by neither the hypercall nor the driver, a second partition, a bogus call code, a reachability scan instead of a string, a mutating write verified through the other route, a null model, a denominator, the arguments of a call the firmware made for itself, the same breakpoint already reached on another processor, a host pause that refused to return.

The two that are not — 13 and 16 — are caught by a different move, and it is the reason the posts in this series read the way they do: keep the wrong version. The record of how a negative was overturned is as much a result as the pass is, and a corrected sentence left in place with its correction beside it is the only thing that stops the same wrong step being taken again by someone who never saw the first attempt — including me, three days later. Every numbered section above exists because the wrong version was still there to be read.

Back to the results: the hypervisor grants the registers and refuses the memory, the checkpoint that needed no driver, a receiver with no sender, and stopping an initialized Secure Kernel.