Count each failed switch once. Replay the oldest unfinished job during scheduling rescue, and route captured SMMU faults through guilty-job recovery. Trace event 70 retains the fault, active table, restart prefix and recovery outcome. The cause of the observed null-address fault remains unresolved.
Validation: focused submit fixture passes with AddressSanitizer and UndefinedBehaviorSanitizer, including repeated timeout observation, oldest-job replay and fault classification.
A first-ring job that waits while the CP stays on the second ring after
a switch back timed out is stranded, as a second-ring job already was:
the restart replays every job and no client loses its device. The
switch timeout record also carries CP_CONTEXT_SWITCH_CNTL and the two
rings of the switch.
With concurrent binning BR waits for the visibility pass that BV runs,
so BR's ring and IB words stand still while the GPU works. The hang
check and the stall handler read the BV read pointer, IB bases and
dwords left as well, and treat a change in them as progress.
The submission's audit phase compared CPU cache lines with DRAM for the
write-combined IBs of the first 16 submissions and for the synchronized
driver buffers of the first 8 and every 1024th. They only detect a CPU
cache that holds a line the GPU should see; they found none in any run
(dirty=0), pool BOs have no cacheable alias for them to compare, and the
write-combine re-assertion, the correction, still runs every time. In
run68 the 8 buffer audits were 11.9 of the phase's 12.1 ms (1.5 ms each
under the driver's lock), and every 1024th submission paid one more.
gpu-audit=1 runs them as before.
Every retired job is now timed, with or without diag=1: by the CP's
stamps when gpu-timeline is on, else by the driver's clock (from the later
of its submission and the previous retirement on its ring). The 8 longest
since boot and the newest 16 over 8 ms are kept with pid, queue, ring,
priority, IB count and bytes (native_adreno_long_jobs, _slow_jobs,
_job_time).
A stall's first stage (write pointer again, translations dropped) now
records the CP's IB progress at the fence's first timed-out wait, at the
stage and 500 us later, the job, and the time from the stage to the job's
retirement (native_adreno_stall_kick). The CP snapshot is taken before the
stage acts, so its registers show the stall rather than the stage's
result.
Not run on hardware.
Every process table maps the GPU context's eleven kernel buffers (8548 KiB,
two preemption records of 4192 KiB among them), and each submission cleaned
and invalidated all of them: 11 passes and 8548 KiB a submission in macOS
runs 57 to 61, about 351 of a submission's 579 us in run61.
Their users keep the GPU's view current themselves (a non-cacheable view, or
a clean after a store through the kernel's mapping), so the buffers are
declared synchronized and a map, an unmap and a submission leave them alone.
The probe's store through the kernel's mapping is cleaned where it is made.
The first 8 submissions and every 1024th audit the head of each buffer
against DRAM; a dirty line restores the maintenance for good.
A submission, a completion and the write-combine re-assertion visit only the
mappings they have work for instead of every mapping of the client. The
submission log reads the IB words only with diag=1 and no longer through one
kernel mapping per word.
gpu-maint-all=1 restores the passes over everything for a comparison.
Counters: native_adreno_maint_kind, native_adreno_maint_walk,
native_adreno_sync_audit, native_adreno_sync_skips, native_adreno_submit_us.
Host tests cover it (test_maintenance, test_gpuctx, test_submit). Not run on
hardware.
A waiter slept on the board's pause (the next GPU interrupt of any kind, at
most 1 ms) and then polled: 34767 of 42325 pauses of one boot ended by the
timeout. Three defects came with it:
- The poll before the sleep used the display's completed prefix, which stays
below a finished fence of the high ring while an older fence of the low
ring is pending. Such a waiter slept until its deadline (100 ms slices)
with its own fence written.
- Nothing looked at the switch between the rings when a fence completed, so
a low-ring job queued behind the high ring got the CP only at the next
submission, zero-timeout poll or expired deadline.
- A finished switch was dropped when its handler found the lock busy, and
the incoming ring's write pointer waited for the next locked call.
The driver exports native_adreno_gpu_interrupt(status) for the board's GPU
interrupt handler to call with the status word it read and cleared. A fence
or a finished switch asks for the completion pass on a thread call:
it retires finished jobs (which runs the ring switch decision and completes a
finished switch), then wakes exactly the waiters whose fence the CP has
written on its own ring (NativeAdrenoSubmit::fencePending). The hang check
runs one pass per period for an interrupt that never came. A waiter arms
itself before it looks at the fence and sleeps on its own record until the
pass disarms it or the caller's deadline passes, so no completion is lost
and no timed poll is left. Until the board delivers interrupts, or with
gpu-fence-wake=0, the waiter keeps the board's pause, with the per-ring
check. Counters: native_adreno_fence_wake. Host tests cover fencePending on
one and two rings. Not run on hardware.
A wait with a timeout runs with the driver's lock held for all of it (a
connection's teardown and the table hand-back wait 2 s, a submission's
wait_fence, the board's calls). It decided the switch between the two rings
once, at its start, and then only read the fence word between pauses, while
the lock kept every other caller of the switch out. A low-ring job behind a
busy high ring therefore stayed without the CP after the high ring went idle,
for the whole timeout; a switch the CP finished meanwhile was not completed
either. The hang check then found a low-ring job older than 2 s with the
high ring idle, took the ring for unreachable, gave it up and restarted the
GPU (one boot log shows a 2 s wait followed by "hangs 1 restarts 1", "dead 3
rescues 1" and a 2.17 s wait of another caller).
wait() and observeFence() go through waitOnRing(), which with two rings
runs driveSwitch again after every pause. A host test plays the CP from the
pause: without the change the wait times out, with it the switch is
triggered, completed and the low-ring job finishes. Not run on hardware.
A thread checks the ring every 500 ms as msm's hangcheck_handler does:
no fence progress with work pending and no CP progress (a6xx_progress,
at most three retries) is a hang. Recovery follows recover_worker: the
hung job's VM takes the fault and, unless it is the kernel's, becomes
unusable; the fence moves past it; the board's restart hook resets the
GPU and runs hw_init and CP init on the same ring; every later job is
replayed with its own fence. Clients read the global fault count plus
their own VM's, so only the guilty client sees its device lost. The
ring gains restart, unfenced submission, a6xx_idle, progress state and
the GBIF bus halt; CP init can run unfenced.
ABI version 2 raises the IB list to 128 entries, so the ring prologue
(cache flush, idle wait, thread switch) runs once per frame instead of
once per 16 IBs. The submit lists live in the client and submit objects
instead of the kernel stack. The stall handler drops the translations and
gives the write pointer again after 10 waits; the IB counts per submission
are recorded.