Commit Graph

30 Commits

  • Merge pull request #263 from CrossDarwin/boot/qualified-firmware-dtb
    fix(bootimg): Preserve qualified firmware transport and resolve its DTB
  • feat(scripts): rebuild a vendor_boot v4 image around a new vendor ramdisk
    GKI devices carry no ramdisk in boot and a vendor ABL rejects one there, so
    anything large that has to reach the bootloader belongs in vendor_boot's vendor
    ramdisk. Nothing in the tree could rewrite that image while leaving the stock one
    intact.
    
    Copy the device tree, the ramdisk table, the bootconfig and the vbmeta blob
    verbatim; change only the ramdisk bytes, the header's vendor_ramdisk_size, the
    ramdisk address and each table entry's size and offset; retarget the AVB footer
    because the vbmeta blob moves. --ramdisk-addr relocates the ramdisk, which stock
    images need when the default window is smaller than the payload.
    
    --verify-against repacks the stock ramdisk and requires a byte-identical result
    before the real ramdisk is written, so a device whose layout differs fails loudly
    instead of producing an unbootable partition.
    
    The offline tests build a synthetic v4 image and cover the repack, the
    relocation, the partition limit and the refusals; the command and its flags are
    documented in docs/reference/cli.md and the Chinese page. Checked on the NX769J
    against its stock vendor_boot: the stock image is reproduced byte for byte, and
    the 64 MiB payload image the tool produces is the one that boots on hardware.
  • fix(release): pass the board's firmware device tree to the pack step and the firmware path to preflight scripts
    - cmd_release: [bootimg] dtb, relative to the build tree, is the device tree of the boot image.
    - firmware.build: preflight scripts get UNIXNU_UEFI, the firmware checkout of the run.
  • feat(release): one-command device packaging with per-OS overrides
    unixnu release runs deps, preflight, patches, build, pack and verify and writes
    package.manifest.json (sizes and SHA-256 of boot.img, payload.img and
    kernel.macho, module revisions, patch order, command line length). verify
    refuses a missing or changed output, a command line of 1000 bytes or more and a
    boot image above [package] boot_partition_bytes.
    
    board.toml [package.defconfig] maps an OS profile to its defconfig, and
    [os_overrides."<profile>"] merges the values that differ for one profile over
    the board; its remove list drops keys the profile does not have. --os picks the
    profile, so one branch serves 17, 26 and 27.
    
    templates/package.sh is the device repositories' fixed script. It reads only
    [package] unixnu and runs unixnu release.
  • feat(gpu): GMEM swap patch, unixnu --os, module pointers for the diagnostics batch
    turnip-gmem-mutable-swap.patch moves GMEM loads and stores with the identity swap on the memory side and adds the render pass and submission rings; unixnu --os 17|26|27 selects the OS profile and its output directory.
  • feat(scripts): iPadOS 27 kernel, driver, GPU addon and seed build steps
    Source-driver and source-storage builds, the arm64e PAC rewrite for the Rust bridge, the IOAccelerator base ABI of 24A446, the bridge with mimalloc and link-time optimisation, the plugin with the prime list, addon options for a prime list, a pipeline pack and cache export, wallpaper overlay and seed changes, and payload, kernel and userdata steps for the new profile.
  • feat(userdata): build the userdata image on Linux
    The userdata step read each IPSW disk image through macOS hdiutil, so it
    stopped on Linux with "hdiutil not found on PATH". It only needs the raw
    disk bytes, so on hosts other than macOS it now decodes them with
    scripts/unixnu/udif.py.
    
    udif.py reads the koly trailer and the blkx (mish) tables of a UDIF
    image and yields the disk in sector order: raw chunks are copied,
    zero-fill and ignored chunks read as zeros, and zlib, bzip2 and LZFSE
    chunks are decompressed (LZFSE with the lzfse tool the Linux toolchain
    installs). It refuses gaps, unsupported chunk types and chunks that do
    not decode to their size. A file without a koly trailer is a raw image
    and is copied: the SystemOS cryptex is one, while the OS image is UDIF
    with LZFSE chunks. test_udif.py checks this with synthetic images.
    
    The window layout, hashes, SHNGROOT manifest and report are unchanged,
    and macOS keeps hdiutil. For iPadOS 17.7.11 (21H461) the image built on
    Linux and the one built on macOS from the same disk images are byte
    for byte identical (9382658048 bytes, sha256 fd43ee7035ce185c...), as
    are the window hashes (OS e5634d04..., SystemOS b4908cde...).
    
    The lzfse lookup that ipsw.py had for the kernelcache moves to util.py,
    which both use. The Linux notice now names --skip-setup and
    seed-data-grow as the steps that need macOS.
    
    Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
  • build(kernel): run the kernel step on Linux
    On a Linux host the driver sets the environment of the
    scripts/linux/setup-toolchain.py toolchain: the xcrun shim answers SDK
    queries with the build tree's fakeroot, clang targets plain arm64 for
    iOS 17, and CCC_OVERRIDE_OPTIONS turns off two warnings upstream clang
    has and Apple clang does not (the archive builders use -Werror and
    replay xnu's compile records, so only the clang driver sees them).
    
    - build-xnu-4k-baseline.sh runs scripts/linux/prepare-xnu.py: source
      rewrites the xnu sources that assume Apple's tools (SHA/AES assembler
      macros, makesyscalls.sh under mawk, kextsymboltool and /dev/null) and
      installs TargetConditionals.h into the fakeroot; libs builds
      libcc_kext.a from compiler-rt's cc_kext profile sources. xnu's make
      also gets the host compiler, mig and the machine sizes a Mac supplies.
      ditto becomes cp.
    - build-firehose.sh compiles libfirehose_kernel from its xcconfig.
    - build-corecrypto-14.py reads project.pbxproj with scripts/pbxproj.py
      (same JSON as plutil for all 13,112 objects).
    - The kernelcache is decoded with lzfse.
    
    The macOS path is unchanged apart from ditto. Built from the same
    sources and configuration on Linux and with Xcode 27.1, the kernels have
    the same global symbols, linked objects, archive symbols, 1,265
    kalloc_type signatures, and prelink bindings and kexts.
    
    Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
  • feat(payload): write a gpu addon into the data volume at boot
    `unixnu build payload --gpu-addon FILE` carries a file made by
    scripts/make-gpu-addon.py next to the kernel. The loader hands it over as
    /chosen/memory-map/GpuAddon, and before launchd starts the kernel writes
    every file whose content or mode differs into /private/var/ngpu of the
    mounted Data volume, then mounts the addon views. The Adreno Metal plugin
    then no longer has to be part of the Data seed, so the seed's persistent
    store survives a plugin rebuild. Every file is zlib compressed to keep
    the payload FAT at 192 MiB. Updates DarwinBoot and NativeRamBlock for the
    addon handoff.
  • feat(unixnu): build the board's archives before the kernel link
    board.toml [kernel] archives lists the archive builders in order. The kernel step runs a bootstrap pass for the compile records when they are missing (it may stop at the link), runs the builders, then links, and reports archives that no builder makes. Prelink replaces an earlier candidate instead of refusing. libpthread and corecrypto-2023 join feeds.toml and the kernel references.
  • feat(unixnu): replace board kernel arguments by name, accept an owner trust cache
    An extra --kernel-arg NAME=VALUE now replaces the board's argument of the same name instead of being appended, so a test can change a default within the 1000 byte command line. --owner-trust-cache takes an owner-signed bundle that carries the original caches plus extra modules, such as the GPU plugin, and skips the board allow-list. The length error names the real limit.
  • build: split drivers and kits into module submodules
    Add the driver and kit repositories as public UniXNU submodules. Update
    submodules during unixnu fetch and before tree assembly.
  • feat(userdata): build the userdata partition image
    build userdata writes the OS and SystemOS disk images of the IPSW behind
    the SHNGROOT manifest, checked against the partition size of the board.
    It prints the fastboot flash command and flashes nothing. The image
    matches a previously built one byte for byte.
  • feat(payload): build the boot image ramdisk payload
    The payload step writes a FAT volume with the CrabApple config, the
    prelinked kernel (optionally with the panic screen patched into a copy),
    the generated device tree and the optional trust cache and Data seed.
    The volume is written with mtools and is reproducible. bootimg uses it
    as the ramdisk when present.
    
    --skip-setup marks Setup as done in a copy of the Data seed so the Hello
    pages are skipped. A non-macOS host now only prints a note, and the
    kernel step checks for xcodebuild.
  • feat(kernel): take the Darwin source versions from feeds.toml
    The xnu tag, commit, Darwin and source versions, the libplatform,
    libdispatch and AvailabilityVersions tags and the tags of the sources
    that still carry the DriverKit and TrustCache headers are set once in
    feeds.toml. The driver passes them to build-xnu-4k-baseline.sh and
    prepare-xnu-source.py, which keep the qualified versions as defaults.
  • feat(scripts): build the firmware from a fresh checkout
    Make the firmware step work on clean clones of the remote repositories:
    initialize the Mu-Silicium submodules the board lists, fetch the two
    extra edk2 submodules (brotli decompressor, libspdm), generate the
    CLANG_BIN directory the CLANGPDB tool chain needs from the Homebrew LLVM
    and lld, pin SOURCE_DATE_EPOCH to the commit time, and add
    --skip-preflight for the board preflight scripts.
  • feat(examples): mark the example device and refuse to build it
    board.toml gains example = true. status shows [EXAMPLE], and build and
    bootimg stop with a message, so a copied sample is never built by
    accident.
  • feat(scripts): add the unixnu build driver
    A Python 3 driver with defconfig, fetch, tree, build, bootimg, status
    and clean. Device repositories supply board.toml and configs/*_defconfig;
    external sources come from feeds.toml and are fetched with shallow,
    blobless, sparse git checkouts pinned in unixnu.lock. Modules can come
    from local directories through --local, --local-<kind> and
    --local-module. The boot image step packs BootShim, the firmware volume
    and the device tree for a Mu-Silicium checkout given with --uefi.