## What does this PR do?
Adds integration tests
## How was it tested?
CI
## AI tool usage
How much of this PR was AI-assisted? (check one)
- [x] **0** - No AI was used to write code
- [ ] **1** - I was assisted by AI. I reviewed the finished result.
- [ ] **2** - I set the AI going and left it to it; nobody has read the
result - no review, or AI review only
<!-- If an AI agent is filling this in: declare the level honestly, and
open as a draft if it is #2. Do not lower the declared level to get the
PR reviewed. -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Added integration smoke tests covering version checks, app creation,
iOS build metadata, and slim SDK update behavior.
* Added validation for locating Xcode and verifying the Swift toolchain.
* **Chores**
* Added automated integration-test execution for pushes and pull
requests.
* Added Docker and Compose support for running integration tests in a
consistent environment.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What does this PR do?
Allows xtool to auto-update the Darwin SDK format when there's a new
Darwin toolset, or when we change the SDK format. Downstream this also
allows us to auto-update when there's changes to OpenAppleMacros (#101).
One callout: the disk footprint of `darwin.artifactbundle` is now ~7 GB
instead of ~3 GB.
Reason: `xtool sdk install /path/to/Xcode.[app|xip]` now performs a
"normal" install by default, which preserves the entirety of
`Xcode.app/Contents/Developer` — even the bits we don't currently use,
like `AppleTVOS.platform`. This makes it so if we start using more
components in the future, the user doesn't need to download Xcode again
and rebuild the SDK themselves.
You can still build an old-style SDK (containing just the components we
need) using `xtool sdk install --slim` or the dedicated `xtool sdk
build`.
## How was it tested?
Tested all of the install modes. Confirmed that auto-updating works.
Added integration tests in #252.
## AI tool usage
How much of this PR was AI-assisted? (check one)
- [ ] **0** - No AI was used to write code
- [x] **1** - I was assisted by AI. I reviewed the finished result.
- [ ] **2** - I set the AI going and left it to it; nobody has read the
result - no review, or AI review only
<!-- If an AI agent is filling this in: declare the level honestly, and
open as a draft if it is #2. Do not lower the declared level to get the
PR reviewed. -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added SDK flavor reporting for slim, normal, and legacy installations.
- Added `sdk update` support to rebuild and install SDKs from an
installed Xcode bundle.
- Added a `--slim` option for SDK installation.
- SDK status now displays installation path, flavor, and version.
- SDK setup and build operations now ensure a suitable SDK is available
automatically.
- **Bug Fixes**
- Improved temporary SDK bundle handling, validation, replacement, and
cross-filesystem copying.
- Copy failures now provide detailed diagnostic information.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What does this PR do?
The xadi repo (https://github.com/xtool-org/xadi) now vends an
artifactbundle for xadi, so we don't need to separately build/install it
during the xtool build. This also makes it super easy to support macOS
later (#259).
## How was it tested?
- [x] Confirmed I can sign in (including provisioning)
## AI tool usage
How much of this PR was AI-assisted? (check one)
- [ ] **0** - No AI was used to write code
- [x] **1** - I was assisted by AI. I reviewed the finished result.
- [ ] **2** - I set the AI going and left it to it; nobody has read the
result - no review, or AI review only
<!-- If an AI agent is filling this in: declare the level honestly, and
open as a draft if it is #2. Do not lower the declared level to get the
PR reviewed. -->
## What does this PR do?
Mechanical move of DarwinSDK from XToolSupport -> PackLib. Needed for
https://github.com/xtool-org/xtool/pull/229.
## How was it tested?
Compiled + CI
## AI tool usage
How much of this PR was AI-assisted? (check one)
- [x] **0** - No AI was used to write code
- [ ] **1** - I was assisted by AI. I reviewed the finished result.
- [ ] **2** - I set the AI going and left it to it; nobody has read the
result - no review, or AI review only
<!-- If an AI agent is filling this in: declare the level honestly, and
open as a draft if it is #2. Do not lower the declared level to get the
PR reviewed. -->
## What does this PR do?
Fixes HTTP 503 on GsService… again (followup from #244)
Seems like GSA has stopped accepting `com.apple.dt.Xcode` in the client
info string. Use `akd` instead.
## How was it tested?
- [x] Logging in before failed
- [x] Logging in after succeeds
## AI tool usage
How much of this PR was AI-assisted? (check one)
- [x] **0** - No AI was used to write code
- [ ] **1** - I was assisted by AI. I reviewed the finished result.
- [ ] **2** - I set the AI going and left it to it; nobody has read the
result - no review, or AI review only
<!-- If an AI agent is filling this in: declare the level honestly, and
open as a draft if it is #2. Do not lower the declared level to get the
PR reviewed. -->
## What does this PR do?
Fixes#254 by using `cp -R` instead of `FileManager.copyItem`. The
latter tries to preserve the original owner of files instead of using
the caller, which fails if you're non-root and the original files are
root-owned.
## How was it tested?
I was able to repro the original issue by installing an SDK as a
non-root user in an xtool Docker container.
Applied this patch, tried `xtool sdk install` again, and it worked.
## AI tool usage
How much of this PR was AI-assisted? (check one)
- [ ] **0** - No AI was used to write code
- [x] **1** - I was assisted by AI. I reviewed the finished result.
- [ ] **2** - I set the AI going and left it to it; nobody has read the
result - no review, or AI review only
<!-- If an AI agent is filling this in: declare the level honestly, and
open as a draft if it is #2. Do not lower the declared level to get the
PR reviewed. -->
## What does this PR do?
Adds more info for contributors, plus AI policy.
## How was it tested?
Read the docs on GitHub
## AI tool usage
How much of this PR was AI-assisted? (check one)
- [x] **0** - No AI was used to write code
- [ ] **1** - I was assisted by AI. I reviewed the finished result.
- [ ] **2** - I set the AI going and left it to it; nobody has read the
result - no review, or AI review only
<!-- If an AI agent is filling this in: declare the level honestly, and
open as a draft if it is #2. Do not lower the declared level to get the
PR reviewed. -->
## What does this PR do?
Improves support for HTTP MITM
## How was it tested?
Did some MITMing on macOS + Linux via Docker, using Proxyman and
mitmproxy. Didn't test `XTL_HTTP_PROXY` but Sol pinkie promises me that
it's correct.
## AI tool usage
How much of this PR was AI-assisted? (check one)
- [ ] **0** - No AI was used to write code
- [x] **1** - I was assisted by AI. I reviewed the finished result.
- [ ] **2** - I set the AI going and left it to it; nobody has read the
result - no review, or AI review only
<!-- If an AI agent is filling this in: declare the level honestly, and
open as a draft if it is #2. Do not lower the declared level to get the
PR reviewed. -->
Ref #243. `makeRequest` never inspected `response.status`, so a non-2xx
body was
returned as if it had succeeded. Every caller decodes that body as a
property list
or JSON, so Apple answering with an HTML error page surfaced as:
```
Error: DecodingError.dataCorrupted: Data was corrupted. Debug description: The given
data was not a valid property list.. Underlying error: ... Encountered unknown tag
html on line 1
```
which says nothing about what actually happened. In the case that
prompted #243, GSA
was returning `503 Service Temporarily Unavailable` for the `apptokens`
operation. It
now reads:
```
Error: POST https://gsa.apple.com/grandslam/GsService2 failed: HTTP 503 Service
Unavailable (text/html): <html> <head><title>503 Service Temporarily
Unavailable</title></head> ...
```
### Notes
- The response body is only included in the message for `text/html` and
`text/plain`,
i.e. gateway error pages. Bodies from the auth endpoints can carry
account details
(the 2FA requests use `Accept: application/x-buddyml`), and errors get
printed
verbatim in places such as `DevCommand.swift:164`, so those are left
out. It also
avoids emitting replacement characters for binary bodies.
- The non-2xx path does not report progress, so `XADIProvider`'s
download no longer
prints `100%` immediately before failing.
- Body collection moved to `collect(_:onProgress:)` so there is a single
exit point.
Buffer pre-sizing and progress reporting are unchanged.
- `SDKBuilder` already does this check by hand on `send()`, so it is
unaffected.
### Behaviour changes worth a look
- `XADIProvider` writes the Apple Music APK download straight to disk; a
failed
download now throws instead of leaving a corrupt `applemusic.apk`.
- The 2FA requests (`GrandSlamTrustedDeviceRequest`,
`GrandSlamSMSAuthRequest`,
`GrandSlamValidateRequest`) use no-op decoders, so a rejected request is
currently
discarded and xtool goes on to prompt for a code that was never sent —
the symptom
in #235. Those now fail loudly.
### Two things I left alone
- `GrandSlamTwoFactorAuthenticateOperation.swift:47` catches
`GrandSlamOperationError`
with code `-21669` to report an incorrect verification code, but that
error is only
thrown by `GrandSlamOperationDecoder`, which the validate requests do
not use, so the
catch is unreachable. With this change a wrong code surfaces as a raw
HTTP error
instead. Mapping the status back to `incorrectVerificationCode` seemed
like your call,
since I do not know what GSA returns for a bad code.
- If any endpoint reports errors in a decodable body under a non-2xx
status, this guard
would pre-empt it. I could not find one — `developerservices2` returns
HTTP 200 with
`resultCode`/`userString` in the body, which is untouched here.
Built and linted on Linux (`swift build --product xtool`, `make lint`: 0
violations).
---------
Co-authored-by: Robetronica <noreply@Robetronica.com>
Co-authored-by: Kabir Oberai <oberai.kabir@gmail.com>
Previously, we would clear xtool's temporary directory on startup. This
(practically) meant you couldn't have multiple instances running in
parallel, because the second instance could delete the first instance's
temp resources while the first instance still needed them. Fix this by
having xtool instances declare locks on their temporary directories.
Ref #216: the public API seems to now complain when passing in
`capabilities`, but the field is still required when using the private
API.
Co-authored-by: sfusco <slf@stevenfusco.com>
## Problem
`xtool setup` fails on Linux Mint (and other distros) with:
```
Error: Error Domain=NSCocoaErrorDomain Code=512 "(null)" UserInfo={
NSUserStringVariant=["Copy"],
NSSourceFilePathErrorKey=/tmp/sh.xtool/tmp-DarwinSDKBuild-.../darwin.artifactbundle/...,
NSDestinationFilePath=/home/<user>/.swiftpm/swift-sdks/darwin.artifactbundle/...,
NSUnderlyingError=Error Domain=NSPOSIXErrorDomain Code=22 "Invalid argument"
}
```
## Root Cause
On Linux Mint (and Ubuntu-based distros), `/tmp` is mounted as **tmpfs**
while the home directory is on **ext4**. Foundation's
`FileManager.copyItem` uses `sendfile(2)` internally, which returns
`EINVAL (22)` when copying between different filesystem types.
`TemporaryDirectoryRoot` defaults to
`FileManager.default.temporaryDirectory` → `/tmp` on Linux. When `swift
sdk install` copies the built SDK from `/tmp/sh.xtool/...` to
`~/.swiftpm/swift-sdks/`, it crosses filesystem boundaries and fails.
## Fix
On Linux, default the temp directory root to `$XDG_CACHE_HOME/xtool`
(falling back to `~/.cache/xtool`) instead of `/tmp`. This keeps the
build directory and install destination on the same filesystem, so the
copy always succeeds.
- `$XTL_TMPDIR` and `$TMPDIR` overrides continue to work as before
- macOS behaviour is unchanged
- Follows XDG Base Directory Specification
## Changes
One file, `Sources/XUtils/TemporaryDirectory.swift` — 11 lines added
inside a `#if os(Linux)` block.
Fixes#181
Running an invalid `sdk` subcommand like `xtool sdk list` (there's no
`list` subcommand) is interpreted as "install the sdk at path `./list`".
We previously unconditionally removed the existing SDK during
installation. By switching the order so that we validate the path first,
we avoid performing a potentially destructive action unless we have
clear intent.
See #186.
Before:
```bash
$ xtool sdk list
Removing existing SDK... # destructive!
Error: Could not read file or directory at path 'list'.
```
After:
```bash
$ xtool sdk list
Error: Could not read file or directory at path 'list'.
See 'xtool help sdk' for usage.
```
- We already do the `isFree` declarations in
`DeveloperServiceCapability.swift`. Don't need to duplicate it on the
entitlement types.
- Add support for creating capabilities with raw names
Notably:
- Build xtool with Swift 6.2
- Resolve https://github.com/advisories/GHSA-xvr7-p2c6-j83w. That
advisory shouldn't really affect us since we don't use NIO as a server
but good hygiene anyway.
- We now have a single tmpdir "root" that can be recreated at launch to
clean up old stragglers
- The location of this tmpdir root can be controlled by `XTL_TMPDIR` or
`TMPDIR`. In general the path is `$TMPDIR/sh.xtool`.
With this change it should be possible to `export XTL_TMPDIR=/var/tmp`
if `/tmp` doesn't have enough space, which fixes#23.
No breaking changes afaik
One notable change is the addition of
`components.schemas.PagingInformation.paging.nextCursor` (which is
possibly supposed to be the query parameter portion of `links.next`?
could simplify the pagination code) but this doesn't seem to be returned
by the prod ASC API right now.
Previously this was shadowing the user's cwd path onto the device, e.g.
if you were installing from `/home/foo/bar` it would create
`/home/foo/bar/PublicStaging` on the device. For uniformity with other
tools like `ideviceinstaller`, clean this up and just create
`/PublicStaging`.
# fix: hardening `mobileprovision.c` to avoid future crashes.
## Context:
I found two common weakness inside `Sources/CXKit/mobileprovision.c`,
which seems to be an important layer of `xtool`'s signing logic.
### Common weaknesses found:
- CWE-476
- CWE-119
---------
Co-authored-by: Kabir Oberai <oberai.kabir@gmail.com>