xnu: integrate matching open source pthread kernel provider for sweet

.github/workflows/xnu.yml

@@ -47,7 +47,7 @@ jobs:
           git -C dxb fetch -q --depth 1 "$DXB_REPO" "$DXB_SHA"
           git -C dxb checkout -q FETCH_HEAD
           # 12.5-14.3 use the top-level patches dir; ours go next to blacktop's
-          cp xnu/patches/*.patch dxb/patches/ 2>/dev/null || true
+          cp xnu/patches/*.patch xnu/patches/*.sh dxb/patches/
           # libfirehose_kernel is built arm64e-only; the SWEET kernel is plain arm64
           sed -i '' 's/ARCHS="x86_64 arm64e" VALID_ARCHS="x86_64 arm64e"/ARCHS="x86_64 arm64e arm64" VALID_ARCHS="x86_64 arm64e arm64"/' dxb/build.sh
           grep -c 'VALID_ARCHS="x86_64 arm64e arm64"' dxb/build.sh

NOTES.md

@@ -3,6 +3,8 @@
 ## Agent handoff / active work (2026-10-15, latest)
 
 - Goal: iOS 16 generation XNU on sweet. Always commit completed work, use gh/Actions for kernel builds, keep this handoff current. Temporary fastboot boot only; preserve Android/backups.
+- **Hardware photo confirms 0018 progress** (`IMG_20261015_173951 (2).jpg`): real phone prints `SWEET CPU: boot CPU 0 registered, SGI1 verified, initialization complete`, enters BSD/MAC, reaches `mbinit: done`, then the same missing-pthread-function-table panic as QEMU at pthread_shims.c:318. CPU completion/interrupt delivery is now physically observed. The photo was received; no further phone boot performed yet.
+- **Patch0019 ready for CI:** genuine Apple OSS libpthread-514.100.2 provider (immutable commit `a6c70ca0e544347a0377babf748f831577ec24f9`), paired with xnu-8796.141.3 by Apple's `distribution-macOS/macos-135/release.json` (`research/macos-135-release.json`). `0019-stage-libpthread.sh` fetches/stages it before patches; workflow now copies shell scripts too. SWEET-only wrappers compile all three real kernel sources, rename provider-side symbols, use kext header view and vm_fault_external, register the real function table at EARLY_BOOT SECOND. Existing pthread_init then runs the actual provider initialization. No pthread wait/syscall is bypassed. Source staging includes bounded explicit address casts/zero user addresses and panic-format fixes. SWEET kernel opaque ksyn storage enlarged from48 to56 bytes to contain this provider's trailing kwe_uth, with a provider size assertion. Local compile of three provider TUs plus pthread_shims/kern_fork passed; imports resolve and provider exports do not collide with existing kernel symbols. All19 patches apply and changed sources match (`research/patch0019-check.log`). Source sandbox `research/bt22/xnu`, originals `research/pre0019`, objects `research/pthread-objs`. Local TargetConditionals fixture only has target macros for syntax; CI stages the real macOS SDK header and is authoritative. Next Actions artifacts: out-0019 / bt23 / run23.log; run QEMU >=45 seconds. No 0019 build/run yet.
 - **Latest kernel implementation `e7d9550`, patches0001-0018; CI `38760004398` PASSED (3m56s)**: https://github.com/dipa-studio/ios-sweet/actions/runs/38760004398. No build is pending. Kernel `xnu/out-0018/kernel.development.sweet`, SHA256 `b3c8d59acc88f93a2c836eb2527a05ed4fbaac7c580a2ea8c3b795137f4aadf1`; checksum verified. Source/log `research/bt22` / `research/run22.log` (baseline for patch0019; no nested .git). New driver/config sources match CI exactly.
 - **CPU wait FIXED in QEMU**: SweetCPU uses legacy IOCPU, validates single boot CPU, ml_processor_register -> processor_start -> initCPU real SGI1 -> scheduler/IOKit completion -> ml_set_max_cpus. It does NOT force a flag or skip a wait. CPU APIs passed: Running CPU0, processor_array[0] association, SGI signalling enabled, name, unsupported secondary-start error (`research/check-cpu18.gdb` / `.log`, `research/q0018-api`). Single-core only; SMP and suspend unsupported. Two-CPU topology fails explicitly (`research/q0018-smp`).
 - **Next blocker reproduced in 45s QEMU** (`research/q0018-long`, zero unknown glyphs): BSD initialization reaches MAC framework, network mcache and `mbinit: done`, then **`pthread kernel extension not loaded (function table is NULL). @pthread_shims.c:318`**. Do not bypass pthread_init: find/integrate the matching open-source libpthread kernel implementation and register its real function table. A 12s run `research/q0018` looked panic-free before the 30s IORTC wait elapsed; use >=45s or GDB checkpoints for the next iteration. No userland/root filesystem boot yet.

xnu/patches/0019-stage-libpthread.sh

@@ -0,0 +1,65 @@
+#!/bin/bash
+# dxb runs patches/*.sh from the XNU source root before applying *.patch.
+set -euo pipefail
+
+[[ "${MACHINE_CONFIG:-}" == SWEET ]] || exit 0
+[[ "${MACOS_VERSION:-}" == 13.5 ]] || {
+    echo 'SWEET libpthread integration requires xnu-8796.141.3 (macOS 13.5)' >&2
+    exit 1
+}
+[[ -f bsd/pthread/pthread_shims.c ]] || exit 1
+
+# libpthread-514.100.2 is paired with xnu-8796.141.3 in Apple's macos-135
+# release.json. Fetch the immutable commit, never a moving branch/tag.
+pthread_sha=a6c70ca0e544347a0377babf748f831577ec24f9
+pthread_tmp=$(mktemp -d)
+trap 'rm -rf "$pthread_tmp"' EXIT
+git init -q "$pthread_tmp"
+git -C "$pthread_tmp" fetch -q --depth 1 \
+    https://github.com/apple-oss-distributions/libpthread "$pthread_sha"
+git -C "$pthread_tmp" checkout -q FETCH_HEAD
+[[ "$(git -C "$pthread_tmp" rev-parse HEAD)" == "$pthread_sha" ]]
+
+python3 - "$pthread_tmp" <<'PY'
+from pathlib import Path
+import shutil
+import sys
+
+source = Path(sys.argv[1])
+target = Path('bsd/pthread/sweet_libpthread')
+if target.exists():
+    raise SystemExit(f'refusing to overwrite existing provider: {target}')
+# Preserve upstream copyright/license headers; no downloaded sources in git.
+for directory in ('kern', 'private', 'include'):
+    shutil.copytree(source / directory, target / directory)
+
+def replace(file, old, new, count):
+    path = target / 'kern' / file
+    text = path.read_text()
+    if text.count(old) != count:
+        raise SystemExit(f'libpthread source changed: {file}: {old!r}')
+    path.write_text(text.replace(old, new))
+
+# The public kernel source has stricter address types than the kext SDK.
+# Keep the address bits intact with explicit casts; user addresses use zero.
+replace('kern_trace.h', 'vm_kernel_unslide_or_perm_external(ptr,',
+        'vm_kernel_unslide_or_perm_external((vm_offset_t)ptr,', 1)
+for callback, count in (('psynch_wait_update_owner', 4),
+                        ('psynch_wait_complete', 1), ('psynch_wait_wakeup', 2)):
+    replace('kern_synch.c', f'pthread_kern->{callback}(kwq,',
+            f'pthread_kern->{callback}((uintptr_t)kwq,', count)
+replace('kern_support.c', '*kevent_list_out = NULL;', '*kevent_list_out = 0;', 1)
+replace('kern_support.c', 'user_addr_t kevent_list = NULL;',
+        'user_addr_t kevent_list = 0;', 2)
+replace('kern_support.c', 'workq_kevent(p, &th_addrs, NULL,',
+        'workq_kevent(p, &th_addrs, 0,', 1)
+replace('kern_support.c',
+        'panic(__func__ ": thread_set_wq_state failed: %d", error);',
+        'panic("%s: thread_set_wq_state failed: %d", __func__, error);', 4)
+PY
+
+# Kernel builds use -nostdinc. Carry the real SDK target definitions beside
+# the provider so local compiles from the uploaded build tree work as well.
+pthread_sdk=$(xcrun --sdk macosx --show-sdk-path)
+cp "$pthread_sdk/usr/include/TargetConditionals.h" bsd/pthread/sweet_libpthread/
+echo "Staged libpthread-514.100.2 ($pthread_sha) for SWEET"

xnu/patches/0019-sweet-builtin-pthread.patch

@@ -0,0 +1,129 @@
+diff --git a/bsd/conf/files b/bsd/conf/files
+--- a/bsd/conf/files
++++ b/bsd/conf/files
+@@ -361,6 +361,9 @@
+ bsd/security/audit/audit_worker.c		optional config_audit
+ 
+ bsd/pthread/pthread_shims.c		standard
++bsd/pthread/sweet_pthread_init.c	standard
++bsd/pthread/sweet_pthread_support.c	standard
++bsd/pthread/sweet_pthread_synch.c	standard
+ bsd/pthread/pthread_priority.c		standard
+ bsd/pthread/pthread_workqueue.c		standard
+ 
+diff --git a/bsd/conf/Makefile.template b/bsd/conf/Makefile.template
+--- a/bsd/conf/Makefile.template
++++ b/bsd/conf/Makefile.template
+@@ -43,6 +43,15 @@
+ 	-DBSD_KERNEL_PRIVATE -DLP64_DEBUG=0 \
+         -Warray-bounds-pointer-arithmetic
+ SFLAGS+= -include meta_features.h
++
++# This provider is built into SWEET; other boards use their normal kext.
++ifeq ($(CURRENT_MACHINE_CONFIG),SWEET)
++SWEET_PTHREAD_ROOT = $(SRCROOT)/bsd/pthread/sweet_libpthread
++SWEET_PTHREAD_INC = -I$(SWEET_PTHREAD_ROOT) -I$(SWEET_PTHREAD_ROOT)/private -I$(SWEET_PTHREAD_ROOT)/include
++sweet_pthread_init.o_INCFLAGS = $(SWEET_PTHREAD_INC)
++sweet_pthread_support.o_INCFLAGS = $(SWEET_PTHREAD_INC)
++sweet_pthread_synch.o_INCFLAGS = $(SWEET_PTHREAD_INC)
++endif
+ 
+ #
+ # Directories for mig generated files
+diff --git a/bsd/sys/pthread_internal.h b/bsd/sys/pthread_internal.h
+--- a/bsd/sys/pthread_internal.h
++++ b/bsd/sys/pthread_internal.h
+@@ -34,7 +34,12 @@
+ 
+ struct ksyn_waitq_element {
+ #if __LP64__
++	#if defined(ARM64_BOARD_CONFIG_SWEET)
++	/* libpthread-514.100.2 includes the trailing kwe_uth pointer. */
++	char opaque[56];
++#else
+ 	char opaque[48];
++#endif
+ #else
+ 	char opaque[32];
+ #endif
+diff --git a/bsd/pthread/sweet_pthread_compat.h b/bsd/pthread/sweet_pthread_compat.h
+new file mode 100644
+--- /dev/null
++++ b/bsd/pthread/sweet_pthread_compat.h
+@@ -0,0 +1,25 @@
++/*
++ * Compile the matching libpthread provider with its kext view of BSD headers.
++ * It owns its full ksyn_waitq_element layout; the kernel's opaque counterpart
++ * reserves this layout. All communication with private BSD state uses callbacks.
++ */
++#ifndef SWEET_PTHREAD_COMPAT_H
++#define SWEET_PTHREAD_COMPAT_H
++#undef BSD_KERNEL_PRIVATE
++#undef XNU_KERNEL_PRIVATE
++#define __PTHREAD_EXPOSE_INTERNALS__ 1
++#define NEEDS_SCHED_CALL_T 1
++
++/* The seven-argument kext API forwards to the eight-argument kernel vm_fault. */
++#define vm_fault vm_fault_external
++
++/* Avoid collisions with the kernel side of the callback interface. */
++#define pthread_kern sweet_pthread_kern
++#define current_uthread sweet_pthread_current_uthread
++#define pthread_start sweet_pthread_start
++#define pthread_stop sweet_pthread_stop
++
++/* os/log passes &__dso_handle as the containing Mach-O image address. */
++extern void *sweet_pthread_log_image __asm("__mh_execute_header");
++#define __dso_handle sweet_pthread_log_image
++#endif
+diff --git a/bsd/pthread/sweet_pthread_init.c b/bsd/pthread/sweet_pthread_init.c
+new file mode 100644
+--- /dev/null
++++ b/bsd/pthread/sweet_pthread_init.c
+@@ -0,0 +1,26 @@
++/* Built-in provider: Apple libpthread-514.100.2, paired with xnu-8796.141.3. */
++#if defined(ARM64_BOARD_CONFIG_SWEET)
++#include "sweet_pthread_compat.h"
++#include "sweet_libpthread/kern/kern_init.c"
++
++/* Keep the provider's full layout within the kernel's reserved storage. */
++_Static_assert(sizeof(struct ksyn_waitq_element) == 56,
++    "update SWEET ksyn storage in sys/pthread_internal.h");
++
++/* Restore the startup interface after compiling the provider's kext code. */
++#define XNU_KERNEL_PRIVATE 1
++#include <kern/startup.h>
++#include <pexpert/pexpert.h>
++
++__startup_func
++static void
++sweet_pthread_register(void)
++{
++    kern_return_t kr = sweet_pthread_start(NULL, NULL);
++    if (kr != KERN_SUCCESS || sweet_pthread_kern == NULL) {
++        panic("SWEET: pthread provider registration failed: %d", kr);
++    }
++    kprintf("SWEET pthread: libpthread-514.100.2 registered\n");
++}
++STARTUP(EARLY_BOOT, STARTUP_RANK_SECOND, sweet_pthread_register);
++#endif /* ARM64_BOARD_CONFIG_SWEET */
+diff --git a/bsd/pthread/sweet_pthread_support.c b/bsd/pthread/sweet_pthread_support.c
+new file mode 100644
+--- /dev/null
++++ b/bsd/pthread/sweet_pthread_support.c
+@@ -0,0 +1,5 @@
++/* Built-in provider: Apple libpthread-514.100.2, paired with xnu-8796.141.3. */
++#if defined(ARM64_BOARD_CONFIG_SWEET)
++#include "sweet_pthread_compat.h"
++#include "sweet_libpthread/kern/kern_support.c"
++#endif /* ARM64_BOARD_CONFIG_SWEET */
+diff --git a/bsd/pthread/sweet_pthread_synch.c b/bsd/pthread/sweet_pthread_synch.c
+new file mode 100644
+--- /dev/null
++++ b/bsd/pthread/sweet_pthread_synch.c
+@@ -0,0 +1,5 @@
++/* Built-in provider: Apple libpthread-514.100.2, paired with xnu-8796.141.3. */
++#if defined(ARM64_BOARD_CONFIG_SWEET)
++#include "sweet_pthread_compat.h"
++#include "sweet_libpthread/kern/kern_synch.c"
++#endif /* ARM64_BOARD_CONFIG_SWEET */