Commit Graph

19 Commits

  • sdk: accept directories for tarball overrides (#385)
    ## What does this PR do?
    
    XTL_TOOLSET_TARBALL and XTL_OAM_TARBALL can now point at an unpacked
    directory, so package managers can use tools they already provide
    without repacking them into tarballs.
    
    <!-- Focus on *what* the PR does in essence. The *how* should come
    second, if it's relevant. -->
    
    ## How was it tested?
    
    Tested on Linux with Swift 6.2.4: 
    * `xtool sdk build` with both overrides set to directories in the Nix
    store
    * `xtool dev` flags the SDK as out of date after an override path
    changes
    
    <!-- e.g. Ran locally on macOS and added unit tests -->
    
    ## AI tool usage
    
    How much of this PR was AI-assisted? (check one)
    
    - [ ] **0** - No AI was used to write code
    - [x] **1** - I was assisted by AI. I reviewed the finished result.
    - [ ] **2** - I set the AI going and left it to it; nobody has read the
    result - no review, or AI review only
    
    <!-- If an AI agent is filling this in: declare the level honestly, and
    open as a draft if it is #2. Do not lower the declared level to get the
    PR reviewed. -->
    
    
    <!-- This is an auto-generated comment: release notes by coderabbit.ai
    -->
    <!-- coderabbit-ai-generated:v1 -->
    
    ## Summary by CodeRabbit
    
    * **New Features**
    * SDK setup now accepts a directory as a package source, recursively
    copying its contents and adding owner write permission.
    * Archive files continue to be supported as package sources and are
    extracted as before.
    <!-- end of auto-generated comment: release notes by coderabbit.ai -->
    
    Signed-off-by: Anish Pallati <i@anish.land>
  • sdk: support env overrides for tarballs (#372)
    ## What does this PR do?
    
    When building the sdk, allows you to set
    `XTL_TOOLSET_TARBALL`/`XTL_OAM_TARBALL` to use local tarballs instead of
    fetching them remotely. Supersedes
    https://github.com/xtool-org/xtool/pull/319.
    
    ## How was it tested?
    
    Honestly didn't bother but I think it works. Fairly safe to land.
    
    ## AI tool usage
    
    How much of this PR was AI-assisted? (check one)
    
    - [ ] **0** - No AI was used to write code
    - [x] **1** - I was assisted by AI. I reviewed the finished result.
    - [ ] **2** - I set the AI going and left it to it; nobody has read the
    result - no review, or AI review only
    
    <!-- If an AI agent is filling this in: declare the level honestly, and
    open as a draft if it is #2. Do not lower the declared level to get the
    PR reviewed. -->
    
    
    <!-- This is an auto-generated comment: release notes by coderabbit.ai
    -->
    
    ## Summary by CodeRabbit
    
    - **New Features**
    - Added support for installing the Toolset and OpenAppleMacros from
    locally provided tarball files.
    - Existing HTTP download behavior remains available when local tarballs
    are not configured.
    
    - **Chores**
      - Improved handling of lengthy installation logic.
    
    <!-- end of auto-generated comment: release notes by coderabbit.ai -->
    
    Co-authored-by: Mathieu Post <mathieupost@gmail.com>
  • sdk: update OpenAppleMacros to v1.4.1 (#371)
    ## What does this PR do?
    
    Updates OpenAppleMacros to v1.4.1. It now includes an info.json that
    lists supported modules, so we don't need to hardcode them.
    
    ## How was it tested?
    
    Built a project that uses `@State` on Linux
    
    ## AI tool usage
    
    How much of this PR was AI-assisted? (check one)
    
    - [ ] **0** - No AI was used to write code
    - [x] **1** - I was assisted by AI. I reviewed the finished result.
    - [ ] **2** - I set the AI going and left it to it; nobody has read the
    result - no review, or AI review only
    
    <!-- If an AI agent is filling this in: declare the level honestly, and
    open as a draft if it is #2. Do not lower the declared level to get the
    PR reviewed. -->
    
    
    <!-- This is an auto-generated comment: release notes by coderabbit.ai
    -->
    ## Summary by CodeRabbit
    
    - **New Features**
    - Updated the DemoSimple example with a SwiftUI app window, interactive
    counter, and button-based incrementing.
    - The example now runs its demonstration action when the content view
    loads.
    
    - **Bug Fixes**
    - Improved SDK component downloads with clearer progress reporting and
    validation.
    - Improved plugin setup across supported platforms for more consistent
    availability.
    <!-- end of auto-generated comment: release notes by coderabbit.ai -->
  • sdk: fix cross-import overlays with SwiftBuild (#361)
    ## What does this PR do?
    
    Fixes #360 cross-import overlays on Linux for builds with swift 6.4+. 
    
    This PR adds "-Xfrontend", "-enable-cross-import-overlays" to
    `toolchan-swb.json` as it was added to `toolchan.json`.
    Also bumps sdkEpoch so SDK will rebuild after update.
    
    ## How was it tested?
    
    Build was tested locally with branch built from sources.
    
    `xtool new example`, then replace the app file
    
    Fix was tested on this example:
    ```Swift
    import SwiftUI
    import PhotosUI
    
    @main
    struct exampleApp: App {
        @State private var item: PhotosPickerItem?
    
        var body: some Scene {
            WindowGroup {
                PhotosPicker("Pick a photo", selection: $item)
            }
        }
    }
    ```
    
    It fails on version 1.22.2, but builds normally when xtool built from
    this branch.
    
    ## AI tool usage
    
    How much of this PR was AI-assisted? (check one)
    
    - [x] **0** - No AI was used to write code
    - [ ] **1** - I was assisted by AI. I reviewed the finished result.
    - [ ] **2** - I set the AI going and left it to it; nobody has read the
    result - no review, or AI review only
    
    <!-- This is an auto-generated comment: release notes by coderabbit.ai
    -->
    
    ## Summary by CodeRabbit
    
    * **Configuration Updates**
    * Generated Swift toolchain settings no longer explicitly select the LLD
    linker. Existing frontend and cross-import overlay options remain
    unchanged. This affects linker selection for generated toolsets; no
    public API changes are included.
    
    <!-- end of auto-generated comment: release notes by coderabbit.ai -->
  • sdk: prune Xcode when installing from xip (#280)
    ## What does this PR do?
    
    We only need `Xcode.app/Contents/{Info.plist,version.plist,Developer}`
    for the SDK (and expect that to continue being the case). We already
    delete other directories when using `xtool sdk install Xcode.app` but we
    weren't doing this on `xtool sdk install Xcode.xip`. Prune on the xip
    path as well.
    
    Also prune on update (and bump the SDK epoch) so that existing users
    benefit from this.
    
    ## How was it tested?
    
    - [x] Install SDK from xip, confirm `Xcode.app/Contents` only contains
    those three things
    - [x] Install SDK from app, confirm `Xcode.app/Contents` only contains
    those three things + we don't modify the original app
    - [x] Build SDK from app (`xtool sdk build Xcode.app`), confirm we don't
    clobber the input Xcode app
    - [x] Update SDK, confirm that extra contents are removed
    - [x] SDK auto-updates after updating xtool binary
    
    ## AI tool usage
    
    How much of this PR was AI-assisted? (check one)
    
    - [x] **0** - No AI was used to write code
    - [ ] **1** - I was assisted by AI. I reviewed the finished result.
    - [ ] **2** - I set the AI going and left it to it; nobody has read the
    result - no review, or AI review only
    
    <!-- If an AI agent is filling this in: declare the level honestly, and
    open as a draft if it is #2. Do not lower the declared level to get the
    PR reviewed. -->
    
    
    <!-- This is an auto-generated comment: release notes by coderabbit.ai
    -->
    
    ## Summary by CodeRabbit
    
    * **Bug Fixes**
    * SDK builds from Xcode app and XIP inputs now check for required Xcode
    components before installation. Missing components produce an error.
    * Extra direct items in Xcode’s `Contents` folder are removed during XIP
    builds and app updates. Slim builds remain unchanged.
    
    <!-- end of auto-generated comment: release notes by coderabbit.ai -->
  • sdk: update darwin-tools and OAM (#270)
    ## What does this PR do?
    
    Bumps darwin-tools and OpenAppleMacros
    
    - Updates LLD to support arm64e.x1 to fix #269
    - Adds FoundationModelsMacros to fix #213
    
    ## How was it tested?
    
    - [x] Confirm that we can build with an Xcode 27 release SDK on Linux
    - [x] Confirm that FoundationModels APIs work
    
    ## AI tool usage
    
    How much of this PR was AI-assisted? (check one)
    
    - [x] **0** - No AI was used to write code
    - [ ] **1** - I was assisted by AI. I reviewed the finished result.
    - [ ] **2** - I set the AI going and left it to it; nobody has read the
    result - no review, or AI review only
    
    <!-- If an AI agent is filling this in: declare the level honestly, and
    open as a draft if it is #2. Do not lower the declared level to get the
    PR reviewed. -->
    
    
    <!-- This is an auto-generated comment: release notes by coderabbit.ai
    -->
    
    ## Summary by CodeRabbit
    
    * **Bug Fixes**
    * Improved Darwin SDK tool discovery during Swift builds, including
    support for locating `dsymutil`.
    
    * **Updates**
      * Updated bundled Darwin tool dependencies.
    * Added Foundation Models macros support to the OpenAppleMacros library
    set.
    
    <!-- end of auto-generated comment: release notes by coderabbit.ai -->
  • sdk: SwiftBuild support (#229)
    ## What does this PR do?
    
    Adds support for SwiftBuild on Swift 6.4.
    
    We now point SwiftBuild to iPhoneOS.platform and rely on the real
    [SWBApplePlatform](https://github.com/swiftlang/swift-build/tree/b30ee3c00abf34857084c26bb873c3da06348134/Sources/SWBApplePlatform).
    We also do a little futzing to convince SWB to use our Darwin-compatible
    toolset (lld, librarian, resource dirs). We explicitly _don't_ pass in a
    Swift SDK because that would downgrade SWB to using one of the generic
    platform plugins instead of ApplePlatform.
    
    This also gets us a lot closer to parity with Xcode's behavior when
    building for Apple platforms; we get the same build flags and
    everything. We can probably achieve identical bundling too, if we take
    control of the PIF.
    
    One other change: since we need to pass custom env vars and such, we
    can't rely on sourcekit-lsp's built-in SwiftPM integration anymore. We
    thus create a custom BSP server that trampolines to SwiftPM's build
    server with the right flags set. This effectively makes SwiftPM an
    implementation detail, so we can start messing with the PIF.
    
    ## How was it tested?
    
    - [x] Build with Swift 6.3 on macOS
    - [x] Build with Swift 6.4 on macOS
    - [x] Build with Swift 6.3 on Linux
    - [x] Build with Swift 6.4 on Linux
    - [x] BSP/LSP file creation works
    - [x] LSP support works in VSCode
    
    ## AI tool usage
    
    How much of this PR was AI-assisted? (check one)
    
    - [ ] **0** - No AI was used to write code
    - [x] **1** - I was assisted by AI. I reviewed the finished result.
    - [ ] **2** - I set the AI going and left it to it; nobody has read the
    result - no review, or AI review only
    
    <!-- If an AI agent is filling this in: declare the level honestly, and
    open as a draft if it is #2. Do not lower the declared level to get the
    PR reviewed. -->
    
    <!-- This is an auto-generated comment: release notes by coderabbit.ai
    -->
    ## Summary by CodeRabbit
    
    - **New Features**
    - Added a `dev build-server` command for Build Server Protocol
    integration.
    - Projects automatically select the appropriate Swift build system when
    supported.
    - Packaging supports output layouts for Swift Package Manager and Swift
    Build.
      - Added options to skip language-server configuration when needed.
    
    - **Documentation**
    - Updated project templates and tutorials to use `.bsp/xtool.json` for
    IDE features such as syntax highlighting and documentation lookup.
    
    - **Improvements**
    - Enhanced SDK generation and compatibility across supported toolchains
    and platforms.
    - Improved automatic language-server configuration while preserving
    existing configuration files.
    <!-- end of auto-generated comment: release notes by coderabbit.ai -->
  • sdk: integrate OpenAppleMacros (#101)
    ## What does this PR do?
    
    Adds support for proprietary Apple macros (like SwiftUI's `@Entry` and
    `@State`) via
    [OpenAppleMacros](https://github.com/xtool-org/OpenAppleMacros).
    
    We have to re-implement these because the macros need to run on the
    _host_, and the Apple SDKs include them as macOS-only dylibs.
    
    ## How was it tested?
    
    - [x] Extensive integration tests in OpenAppleMacros that compare
    Apple's expansions against ours
    - [x] Confirmed that `@Entry`, `#Preview`, and `@State` work in Swift
    6.3 and 6.4 (State is a PW in the former, macro in the latter)
    - [x] Confirmed that OSS toolchain macros continue to work (tested
    `@Observable`, `#bundle`)
    
    ## AI tool usage
    
    How much of this PR was AI-assisted? (check one)
    
    - [x] **0** - No AI was used to write code
    - [ ] **1** - I was assisted by AI. I reviewed the finished result.
    - [ ] **2** - I set the AI going and left it to it; nobody has read the
    result - no review, or AI review only
    
    <!-- If an AI agent is filling this in: declare the level honestly, and
    open as a draft if it is #2. Do not lower the declared level to get the
    PR reviewed. -->
  • sdk: self-updating darwin tools (#110)
    ## What does this PR do?
    
    Allows xtool to auto-update the Darwin SDK format when there's a new
    Darwin toolset, or when we change the SDK format. Downstream this also
    allows us to auto-update when there's changes to OpenAppleMacros (#101).
    
    One callout: the disk footprint of `darwin.artifactbundle` is now ~7 GB
    instead of ~3 GB.
    
    Reason: `xtool sdk install /path/to/Xcode.[app|xip]` now performs a
    "normal" install by default, which preserves the entirety of
    `Xcode.app/Contents/Developer` — even the bits we don't currently use,
    like `AppleTVOS.platform`. This makes it so if we start using more
    components in the future, the user doesn't need to download Xcode again
    and rebuild the SDK themselves.
    
    You can still build an old-style SDK (containing just the components we
    need) using `xtool sdk install --slim` or the dedicated `xtool sdk
    build`.
    
    ## How was it tested?
    
    Tested all of the install modes. Confirmed that auto-updating works.
    Added integration tests in #252.
    
    ## AI tool usage
    
    How much of this PR was AI-assisted? (check one)
    
    - [ ] **0** - No AI was used to write code
    - [x] **1** - I was assisted by AI. I reviewed the finished result.
    - [ ] **2** - I set the AI going and left it to it; nobody has read the
    result - no review, or AI review only
    
    <!-- If an AI agent is filling this in: declare the level honestly, and
    open as a draft if it is #2. Do not lower the declared level to get the
    PR reviewed. -->
    
    
    <!-- This is an auto-generated comment: release notes by coderabbit.ai
    -->
    ## Summary by CodeRabbit
    
    - **New Features**
    - Added SDK flavor reporting for slim, normal, and legacy installations.
    - Added `sdk update` support to rebuild and install SDKs from an
    installed Xcode bundle.
      - Added a `--slim` option for SDK installation.
      - SDK status now displays installation path, flavor, and version.
    - SDK setup and build operations now ensure a suitable SDK is available
    automatically.
    
    - **Bug Fixes**
    - Improved temporary SDK bundle handling, validation, replacement, and
    cross-filesystem copying.
      - Copy failures now provide detailed diagnostic information.
    <!-- end of auto-generated comment: release notes by coderabbit.ai -->
  • Fix clang headers (#228)
    A slightly different approach to #215.
    
    ---------
    
    Co-authored-by: sfusco <slf@stevenfusco.com>
  • Validate SDK before overwriting (#192)
    Running an invalid `sdk` subcommand like `xtool sdk list` (there's no
    `list` subcommand) is interpreted as "install the sdk at path `./list`".
    We previously unconditionally removed the existing SDK during
    installation. By switching the order so that we validate the path first,
    we avoid performing a potentially destructive action unless we have
    clear intent.
    
    See #186.
    
    Before:
    
    ```bash
    $ xtool sdk list
    Removing existing SDK... # destructive!
    Error: Could not read file or directory at path 'list'.
    ```
    
    After:
    
    ```bash
    $ xtool sdk list
    Error: Could not read file or directory at path 'list'.
      See 'xtool help sdk' for usage.
    ```
  • Enable cross import overlays (#153)
    Fixes #152
    
    We might want to figure out if there are any other flags we don't pass
    but should
  • Support unxipping Xcode betas (#115)
    Xcode betas are stored in the xip as Xcode-beta.app. Support this by
    avoiding hardcoding the `Xcode.app` path.
  • Fix changeCurrentDirectory warning (#81)
    This is `discardableResult` on macOS (perhaps because it goes through
    objc bridging?) but not on Linux so we get a warning on the latter
  • Improve SDK handling (#40)
    * Use swift-sdk commands instead of file ops
    
    * comment