Commit Graph

29 Commits

  • feat(vm): record the mapping a failed map met
    native_adreno_map_met keeps, for the last failed BoMap, the address and
    bytes asked, whether the handle was mapped already, and the mapping at
    or past that address: its address, bytes, handle, jobs and kind.
    
    Run on hardware: a refused map at the window's first address showed
    another mapping of the same size there.
  • fix(priority): give a GPU-denied task the background priority
    RunningBoard sets the Deny role on most macOS processes that render
    nothing visible (345 of 484 role changes in one session). Only the
    background and utility roles took a queue's background value, so a
    denied task's submissions ran at foreground priority beside the render
    server's.
    
    Not run on hardware.
  • refactor: drop the sheng- prefix from boot arguments
    Every boot argument loses its sheng- prefix (sheng-touch=4 is now
    touch=4). No XNU or Apple kext boot argument of iPadOS 27 or macOS 27.3
    uses any of the new names.
  • fix(diagnostics): keep only seven-issue GPU trace events
    The kernel ring keeps submission, retirement, display arrival, fence and
    latch, build identity and the render server's render, encode and completion
    thread identities. Pixel capture, dispatch hooks and the switch, recovery,
    fence-query and import events are removed; the ring holds 65536 records.
  • feat(diag): record who set which GPU priority
    The newest 32 setPriority calls keep the pid, both values, the task's GPU
    role, the time and how many times the connection had told the driver
    before, so a priority that the frontend forwards can be told apart from
    the runtime's own call at device open.
    
    Submissions carry the task's GPU role to the timeline.
  • feat(gpu): CP preemption between two rings (A1)
    Add hardware CP context switch via CP_CONTEXT_SWITCH_CNTL on a second
    ring. Use a conservative BO-disjoint rule: only allow a queued job to
    preempt the running ring when their buffer sets share no handle. Two
    4192 KiB preemption records and two a7xx_cp_smmu_info pages are mapped
    at fixed IOVAs. Ring priority comes from the kernel GPU role of the
    submitting task (BACKGROUND/UTILITY → ring 1, UI/focal/realtime → ring 0).
    Diagnostics gated on sheng-diag=1; feature gated on sheng-gpu-preempt=1.
  • feat(timeline): owner pid and retire time per GPU job
    Each timeline entry records the submitting process and when the driver saw the job retired, so a frame's wait for the ring and for completion can be split.
  • perf(bo): free system BO pages after the device lock
    BoFree unmaps the BO under the lock and hands its still-wired descriptor out; the pages are unwired and freed once the lock is let go. i e38a0b1eb: a BoFree held the lock up to 38 ms.
  • feat(adreno): job timeline, stall ring and CP counter stamps
    Framed submissions stamp the CP always-on counter before and after their buffers (sheng-gpu-timeline, default on); the retired jobs, the stall handler's actions and a ring status word are kept for the boot export. Drops the large-import record. Host tests cover the stamps and the shim symbols.
  • perf(memory): make and wire BoAlloc's pages outside the lock
    The descriptor of a user BO's system memory is made, wired and, for
    default-attribute pages, written back before the driver's lock is taken;
    allocateSystem adopts it under the lock and pages it does not take are
    given back after the call. i14aef633e held the lock 4.0 s in 5454
    allocations while every other process's submit and wait poll queued.
  • feat(telemetry): each client's own BO bytes live and at peak
    The kernel's total of system BOs could not say which process held them.
    Each user client now counts its own BOs (imports not counted) for the
    board log's [gpu objects] row.
  • perf(memory): user BOs from system memory, no fixed BO pool
    User BOs are pageable IOBufferMemoryDescriptors with no task and no
    kernel mapping, wired by prepare(), as Metal resources are system
    memory. The VM zero-fills them and sets the write-combined attribute in
    one batch while wiring, and restores it when the object is reaped.
    Client views map the descriptor itself, so no device-pager view resets
    the page attribute and submissions no longer re-assert it.
    
    The 1536 MiB boot-time pool is gone. In untagged-ed1611b1 it filled
    with the runtime's blocks, 404 BOs fell back to scattered kernel memory,
    96 allocations of 96 MiB failed and the pressure flag stayed set.
    Memory pressure is now reported when the system refuses a BO.
    Imports are proven exclusive through a page bitmap of the driver's own
    BOs instead of a pairwise span check.
  • perf(adreno): no XNU contiguous search for BOs when the pool exists
    BoAlloc tried one contiguous run from XNU for every BO of 64 KiB or
    more before the pool. Once the pool was full each such BO scanned
    physical memory (untagged-203fd45bf: 96 MiB searches over 1.2 million
    pages that found nothing) under the publication lock. With the board's
    pool the allocation already takes a contiguous run from it first and
    then scattered pages, as msm's GEM objects take pages.
  • feat(adreno): grow object, queue and job tables with demand
    No fixed object count: the handle index, the VM's mappings and ranges,
    a client's entries and its BO views grow with memory only. Queues grow
    as msm's submitqueues do; each client owns a 2^32 range of the shared
    queue space. Jobs are bounded by ring space: a full ring answers
    kIOReturnBusy, as msm waits for room. A BO is limited by the client's
    GPU address window, which Query reports as max_bo_bytes; the 256 MiB
    budget and the ABI's queue and size constants are gone.
  • feat(adreno): msm hangcheck and GPU restart with replay
    A thread checks the ring every 500 ms as msm's hangcheck_handler does:
    no fence progress with work pending and no CP progress (a6xx_progress,
    at most three retries) is a hang. Recovery follows recover_worker: the
    hung job's VM takes the fault and, unless it is the kernel's, becomes
    unusable; the fence moves past it; the board's restart hook resets the
    GPU and runs hw_init and CP init on the same ring; every later job is
    replayed with its own fence. Clients read the global fault count plus
    their own VM's, so only the guilty client sees its device lost. The
    ring gains restart, unfenced submission, a6xx_idle, progress state and
    the GBIF bus halt; CP init can run unfenced.
  • feat(adreno): report BO pool pressure in Query flags
    NativeAdrenoParamsPoolPressure tells clients to give cached BOs back.
  • feat(adreno): IO-coherent owned BOs behind sheng-gpu-owned-coherent
    With the arg set and the coherence proof passed, the GPU's own BOs are not write-combined: the ABI reports NativeAdrenoParamsOwnedCoherent so the client maps its CPU view write-back, and the GPU maps the pages write-back inner-shareable. The write-combined audit and per-submission cache maintenance are skipped for them. Default off.
  • perf(adreno): count cache maintenance per phase and skip uncached imports
    Phase counters for the pool, allocate, map, submit, unmap, complete and table passes. Imports the owner maps uncached need no maintenance. Process page tables come from cacheable blocks with the walk coherence proved at board start, so a table page is ordered instead of cleaned. A freed BO whose first pages a user task still maps is counted. Submit records, handle index and user client changes of the iPadOS 27 bring-up are included.
  • perf(client): find objects through a handle index and hold 16384 of them
    A client's objects are found through a handle-to-entry index instead of
    a walk over every entry, free entries start from a hint, and the live
    count is kept rather than recounted. A BO's view token names its own
    slot, so CPU map and unmap no longer walk every view.
    
    The object cap goes from 4096 to 16384 per client, as many as the shared
    VM table holds: at 4096 backboardd ran out of objects and crashed twice
    (i567). On sheng (i568) no respring happened with a peak of 5725.
  • feat(submit): take a whole frame of up to 128 IBs in one submission
    ABI version 2 raises the IB list to 128 entries, so the ring prologue
    (cache flush, idle wait, thread switch) runs once per frame instead of
    once per 16 IBs. The submit lists live in the client and submit objects
    instead of the kernel stack. The stall handler drops the translations and
    gives the write pointer again after 10 waits; the IB counts per submission
    are recorded.
  • fix(vm): map GPU-written imports non-coherent and skip their per-submission cache work
    Imports the GPU writes (render targets) are mapped non-cacheable instead of
    write-back inner-shareable: the GPU stopped inside its end-of-pass cache clean
    when system memory passes wrote them coherently. The owner's CPU does not touch
    their pixels while the GPU owns them, so they were cleaned and invalidated at
    map time and need no maintenance on every submission (about 60 ms per frame).
    Read-only imports stay coherent.
  • feat(userclient): record live imports and refused frees
    A render server thread waited inside vm_deallocate for minutes on
    int-233 and int-234. To tell whether an import of user pages is still
    held over that range, the client now keeps the count of live imports,
    the frees of an import it refused with the last status, and the eight
    newest imports (address, owner pid, bytes, handle, state).
  • fix(vm): raise the GPU object limits to 4096
    The shared VM mapping table held 1024 objects. int-237 filled it twice
    (BoMap refused with kIOReturnNoResources at objects=1024): Turnip turned
    the refusal into VK_ERROR_OUT_OF_DEVICE_MEMORY inside a command stream
    reservation it does not check, and backboardd crashed writing through a
    NULL stream pointer; the other run froze with the same refusals.
    
    Raise the VM table, the per-client entry table and the per-client view
    cache to 4096, and record per client slot the live objects, their peak,
    refused BoAllocs and the window's highest mapped end
    (native_adreno_client_stats).
    
    Claude-Session: https://claude.ai/code/session_01Ci5sg8z8jkQHR4QoXMBqfY
  • fix(userclient): serve metal clients beside the render server
    - Four client windows: the render server keeps 1 GiB, and the
      wallpaper, icon and widget processes share the rest of the aperture
      (170 MiB each).
    - A client that owns no queue range yet is no longer refused by
      setScope, held up in close or waited on in releaseAll because the
      render server has jobs in flight in the default range 0 (int-200).
      New regression block in the user-client host test.
    - BoCpuUnmap keeps the shared CPU view while another mapping lease of
      the same BO is held; dropping it unmapped the other holder's pages
      and crashed the render server in vkCmdCopyBuffer (int-177).
    - Diagnostics: the reason the latest mount failed, mappings that fall
      outside a client window, a text row per non-render-server process
      and a name for the mailbox poller thread.
  • feat(publish): pool only user BOs and serve one connection per process
    The SMMU tables and ring/fence/SQE buffers no longer come from the
    write-combined pool, so CPU table walks stay cached. The published
    service keeps one user-client core per connection: the render server
    gets the 1 GiB window at 0x20000000 and two other processes 256 MiB
    each, with their own queue ranges and teardown.
  • Import NativeAdreno
    Co-authored-by: rpone <rponeawa@qq.com>