iOS Device Benchmarking with devicectl

Collective Library edition. This is the complete technical report. Private filesystem paths, internal run identifiers, campaign-control notes, and repository navigation were removed. Technical claims, code, measurements, evidence labels, citations, corrections, and falsification criteria are preserved.

September 6, 2026

This field guide covers the shell-operated tooling layer for running a Core ML measurement app on a physical iPhone: build and install, stage model data, launch and monitor, collect durable results, inspect compute plans, and recover crash evidence. It is reusable operating guidance, not a verdict on any the reference implementation model or certification campaign.

Executive Summary

• devicectl device copy to is not cp. Firsthand implementation evidence: copying a file to a missing parent inside appDataContainer can exit without creating anything; copying the containing directory works. Verify every transfer.
• Never use --remove-existing-content true casually. Current command help describes destination-directory cleanup, but firsthand implementation evidence shows it emptied the entire app data container.
• In device process launch, every token after the positional bundle identifier is application argv. Put every devicectl option before that positional argument.
• --console connects standard streams and waits for exit, but firsthand implementation evidence shows ordinary Swift print output can remain invisible because stdout is block-buffered. Emit critical progress to stderr and write final receipts to Documents.
• Crash evidence is separate from app data. systemCrashLogs is a supported file-service domain and can contain both app and Core ML or ANE service failures.
• An on-device MLComputePlan is target-specific anticipated placement, not a trace of a completed prediction. Read deviceUsage(for:) and estimatedCost(of:), then use runtime telemetry for an execution claim.
• A shell with no Xcode account signed in needs a manually managed development profile and matching development identity. This is firsthand the reference implementation behavior, not a claim that every CI signing setup must be manual.
• Community reports about fixed iPhone memory ceilings, JIT compilation spikes, AOT paging, background suspension, and re-signing are useful hypotheses. They remain labeled until reproduced on the target tuple.

Evidence Labels

• Current command help: reproduced with Xcode 26.6 and devicectl 518.33. Re-run xcrun devicectl help ... after changing Xcode.
• Documented API: current Apple Core ML documentation retrieved through Context7 library /websites/developer_apple_coreml.
• Firsthand implementation evidence: reproduced by the local iOS harness workflow and recorded in the linked harness README or research brief.
• Community / unverified: synthesized from the raw report's public sources. Keep the recipe, but test it on the exact hardware, OS, Xcode, app, and model before relying on it.

1. Operating Model

A robust run has five explicit boundaries:

signed .app
    -> install
    -> stage one immutable input directory
    -> launch one configuration
    -> pull one result receipt and all relevant crash evidence

Treat the host, the app container, and system diagnostics as separate stores. A successful command invocation is not proof that a file arrived, an app completed, a model used the requested device, or no system daemon crashed.

For automation, prefer --json-output <path> where devicectl supports it. Current command help states that a user-provided JSON file is the only supported interface for programs consuming command output. Human-readable stdout is not a stable parser contract.

Use privacy-safe run identifiers in paths and receipts. Do not record device identifiers, device names, signing credentials, private audio paths, or transcript content.

2. Build and Install a Signed App

devicectl device install app installs an already signed .app; it does not solve signing or provisioning:

xcrun devicectl device install app \
  --device "<device-id>" \
  --json-output install.json \
  "/path/to/Benchmark.app"

The placeholder device ID is operational input only. Do not persist it in a public receipt.

2.1. Manual signing when no Xcode account is signed in

Firsthand implementation evidence: automatic signing with -allowProvisioningUpdates failed with a "No Accounts" error when the build host had no account signed into Xcode. An Xcode-managed development profile was then rejected under manual signing. A non-Xcode-managed development profile that included the registered target device worked with manual signing:

xcodebuild \
  -project Benchmark.xcodeproj \
  -scheme Benchmark \
  -configuration Release \
  -destination "generic/platform=iOS" \
  -derivedDataPath .build/xcode \
  CODE_SIGN_STYLE=Manual \
  CODE_SIGN_IDENTITY="Apple Development" \
  PROVISIONING_PROFILE_SPECIFIER="<manual-profile-name>" \
  build

The signing identity's private key and the provisioning profile must be available to the shell. The profile must authorize the app identifier, entitlements, team, and target device. Unlocking a keychain can fix private-key access errors, but it does not invent a missing Xcode account or repair a mismatched profile.

security find-identity -v -p codesigning
security cms -D -i "/path/to/profile.mobileprovision"
codesign -d --entitlements :- "/path/to/Benchmark.app"
codesign --verify --deep --strict --verbose=2 "/path/to/Benchmark.app"

Do not print keychain passwords, private keys, or full provisioning payloads into CI logs.

2.2. Archive and export

The raw report's archive/export route is worth keeping for CI systems that produce an archive. Xcode 26.6 marks export method development deprecated; use debugging.

xcodebuild archive \
  -workspace Benchmark.xcworkspace \
  -scheme Benchmark \
  -destination "generic/platform=iOS" \
  -archivePath .build/Benchmark.xcarchive \
  CODE_SIGN_STYLE=Manual \
  CODE_SIGN_IDENTITY="Apple Development" \
  PROVISIONING_PROFILE_SPECIFIER="<manual-profile-name>"

xcodebuild -exportArchive \
  -archivePath .build/Benchmark.xcarchive \
  -exportPath .build/export \
  -exportOptionsPlist ExportOptions.plist

A minimal current export configuration can name the profile rather than embedding private identifiers:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>method</key>
  <string>debugging</string>
  <key>signingStyle</key>
  <string>manual</string>
  <key>signingCertificate</key>
  <string>Apple Development</string>
  <key>provisioningProfiles</key>
  <dict>
    <key>com.example.benchmark</key>
    <string>Benchmark Development</string>
  </dict>
</dict>
</plist>

Community / unverified: some headless setups require provisioning profiles to be installed under ~/Library/MobileDevice/Provisioning Profiles/ using the profile's internal identifier as the filename. Other tooling accepts a profile name or identifier in provisioningProfiles. Inspect the active Xcode version's xcodebuild -help; do not assume one naming convention is universal.

2.3. Re-signing and signing automation

Community / worth trying: re-signing a prebuilt app or IPA can work when the executable, embedded frameworks, entitlements, app identifier, and profile all agree. Sign nested code before the containing app:

for framework in Payload/Benchmark.app/Frameworks/*.framework; do
  codesign --force --sign "<development-identity>" "$framework"
done

codesign --force \
  --sign "<development-identity>" \
  --entitlements Entitlements.plist \
  Payload/Benchmark.app

This recipe is incomplete for bundles containing extensions, app clips, nested apps, or other signed code. Enumerate and verify the actual bundle graph; do not apply --deep as a substitute for understanding it.

Community: fastlane match, sigh, and cert can synchronize the same profile-and-identity inputs. They reduce distribution friction but do not change Apple's signing contract. altool and notarytool are not the mechanism for installing a development build on a registered iOS device.

Signing: do this / avoid this

3. Stage and Verify App-Container Data

3.1. Copy a directory, not a file into a missing parent

The basic app-container copy syntax is:

xcrun devicectl device copy to \
  --device "<device-id>" \
  --domain-type appDataContainer \
  --domain-identifier com.example.benchmark \
  --source ./run-input \
  --destination Documents/run-input \
  --json-output copy.json

Firsthand implementation evidence: a file-to-file copy whose destination parent did not already exist completed without producing the target file. Copying the whole local directory created the destination directory and transferred its contents. Therefore:

1. Build one local staging directory for each logical payload.
2. Copy that directory as a unit.
3. List the remote destination after every copy.
4. Verify expected names, sizes, and, when practical, content hashes in the app before loading a model.

Current command help says copy to skips files that have not been modified and supports repeated --source flags. It documents an overall --timeout, but it does not promise resumable transfer. Unverified hypothesis: interrupted multi-gigabyte transfers must be restarted. Design staging to be idempotent and independently verifiable.

3.2. List the domain directly

The raw report correctly identified a direct file-listing command:

xcrun devicectl device info files \
  --device "<device-id>" \
  --domain-type appDataContainer \
  --domain-identifier com.example.benchmark \
  --subdirectory Documents/run-input \
  --recurse \
  --json-output files.json

Current help confirms --subdirectory, recursive listing, filtering, sorting, and JSON output. Use this rather than treating a zero exit status from copy to as proof.

3.3. --remove-existing-content can wipe the container

Current copy to help describes --remove-existing-content as removing files from the destination directory and says it applies only to directory transfers.

Firsthand the reference implementation contradiction: --remove-existing-content true emptied the entire app data container, not only the named subtree. The observed behavior is more destructive than the help text implies. Do not use this flag in a qualifying workflow.

The safer pattern is in-app, allow-listed cleanup:

let fileManager = FileManager.default
let staleDirectory = documentsURL.appendingPathComponent("bundles/stale", isDirectory: true)
if fileManager.fileExists(atPath: staleDirectory.path) {
    try fileManager.removeItem(at: staleDirectory)
}

Constrain deletion to a known child beneath a known app-owned root. Refuse absolute paths, .., symlinks that escape the root, or an empty run identifier.

3.4. Other domains

Current command help lists temporary, appDataContainer, appGroupDataContainer, and systemCrashLogs.

For an App Group, use appGroupDataContainer and the App Group identifier, not the application bundle identifier:

xcrun devicectl device info files \
  --device "<device-id>" \
  --domain-type appGroupDataContainer \
  --domain-identifier group.com.example.benchmark \
  --json-output group-files.json

Documented platform constraint, externally synthesized: the app and extensions must carry matching App Group entitlements. Treat App Group staging as a separate security boundary, not an alias for the app container.

File transfer: do this / avoid this

4. Launch and Monitor One Configuration

4.1. Argument ordering is a hard boundary

Current command help defines the grammar as <bundle-identifier-or-path> [<command-line-arguments> ...]. Firsthand implementation evidence: every token after the positional bundle identifier was forwarded to the launched app.

xcrun devicectl device process launch \
  --device "<device-id>" \
  --terminate-existing \
  --console \
  --environment-variables '{"BENCHMARK_MODE":"latency"}' \
  com.example.benchmark \
  --model-path Documents/run-input/encoder.mlmodelc \
  --iterations 100

Here --model-path and --iterations belong to the app. Every devicectl option, including --device, --console, --environment-variables, and --terminate-existing, appears before the bundle identifier.

No -- separator is required by the documented grammar. Do not introduce one without checking the installed tool's help and the app's observed argv.

4.2. Environment variables have two supported routes

The raw report said environment variables must be passed with --environment-variables. Current command help documents two routes:

1. Pass a JSON dictionary before the bundle identifier.
2. Export host variables with a DEVICECTL_CHILD_ prefix.

export DEVICECTL_CHILD_BENCHMARK_MODE=latency
xcrun devicectl device process launch \
  --device "<device-id>" \
  com.example.benchmark

An explicit --environment-variables dictionary overrides prefixed caller variables. Do not put secrets into either route; process environments and logs are not a credential vault.

4.3. Lifecycle options

• --activate is the default where supported and requests foreground activation.
• --terminate-existing kills an existing app instance before launch where supported. Use it when a clean process is part of the measurement contract, not as an unconditional default.
• --start-stopped launches suspended for debugger attachment.
• --console connects standard streams, waits for termination, and forwards catchable signals.
• --timeout limits the devicectl command. It is not proof that the app persisted or completed after the host command exits.

Inspect running processes separately:

xcrun devicectl device info processes \
  --device "<device-id>" \
  --filter "name CONTAINS[c] 'Benchmark'" \
  --json-output processes.json

Community / unverified: launching through devicectl may avoid some launch-time behavior seen with icon launches. Do not convert that anecdote into a watchdog guarantee. Measure startup, foreground state, suspension, and termination on the target OS.

Process launch: do this / avoid this

5. Capture Progress and Durable Results

5.1. --console does not make Swift print durable

Firsthand implementation evidence: plain Swift print(...) produced no visible output under --console; stdout was block-buffered because it was not attached to an interactive terminal. Stderr worked, and a file written under Documents could be pulled reliably.

Use stderr for progress:

import Foundation

func emitProgress(_ message: String) {
    FileHandle.standardError.write(Data((message + "\n").utf8))
}

Use a file plus an atomic done marker for the final machine-readable result:

let payload = try JSONEncoder().encode(result)
let temporaryURL = resultsURL.appendingPathComponent("\(runID).json.tmp")
let finalURL = resultsURL.appendingPathComponent("\(runID).json")
try payload.write(to: temporaryURL, options: .atomic)
try FileManager.default.moveItem(at: temporaryURL, to: finalURL)
try Data().write(to: resultsURL.appendingPathComponent("\(runID).done"))

Pull it after the done marker appears:

xcrun devicectl device copy from \
  --device "<device-id>" \
  --domain-type appDataContainer \
  --domain-identifier com.example.benchmark \
  --source "Documents/results/<run-id>.json" \
  --destination "./results/<run-id>.json" \
  --json-output pull-result.json

Validate the receipt schema, run ID, configuration identity, model signature, and completion status after transfer.

5.2. Unified logging

The raw report's OSLog advice remains useful: unified logging survives beyond one stdout pipe, but dynamic fields may be private unless marked public. Never mark private audio, transcripts, credentials, or device identifiers public merely to simplify debugging.

Historical device collection is supported by the host /usr/bin/log collect:

sudo /usr/bin/log collect \
  --device-udid "<device-id>" \
  --last 15m \
  --output benchmark.logarchive

/usr/bin/log show \
  --archive benchmark.logarchive \
  --predicate 'process == "Benchmark"' \
  --style ndjson

Correction to the raw report: on the verified host, /usr/bin/log stream --help has no --device, --device-name, or --device-udid option. The raw log stream --device-udid ... recipe is not valid there. For live output, use devicectl ... --console with stderr, an app-owned progress file, or a separately verified device-log tool. Re-check the installed OS before using a remote log stream recipe.

Console and log capture: do this / avoid this

6. Recover Crash Evidence

systemCrashLogs is a current devicectl file-service domain:

xcrun devicectl device copy from \
  --device "<device-id>" \
  --domain-type systemCrashLogs \
  --source . \
  --destination ./crashes \
  --json-output pull-crashes.json

List before pulling when a focused selection is useful:

xcrun devicectl device info files \
  --device "<device-id>" \
  --domain-type systemCrashLogs \
  --json-output crash-files.json

Apple crash reports commonly use .ips. Correlate by process, timestamp, run window, and launch metadata rather than filename alone:

jq -r '(.procName // .process // "unknown") + " - " + (.timestamp // "unknown")' \
  crash.ips

Some .ips files use a JSON metadata line followed by another JSON object; others vary by OS. Inspect the format before hard-coding head -n 1.

Community / worth checking: Core ML workloads can fail in out-of-process services such as E5 runtime or ANE compiler processes, so an app-name-only scan may miss the root cause. Pull the domain around a bounded run window and inspect all new reports. The exact service names and report schemas are OS-version dependent.

Jetsam termination is not necessarily a catchable Swift error. Treat a missing done marker, vanished process, and fresh resource-limit report as a correlated failure requiring the crash artifact, not as a timeout to retry blindly.

7. Keep Long Measurements Alive

For a foreground UIKit harness, disable automatic screen idle while a run is active:

await MainActor.run {
    UIApplication.shared.isIdleTimerDisabled = true
}

Restore the prior setting after completion.

Community / platform hypothesis: a standard app can still be suspended after it is backgrounded or the device locks. A devicectl launch does not grant indefinite background execution. Keep the measurement app foreground, keep the device in a controlled powered state, and make suspension observable through timestamps and done markers.

Do not claim an audio, location, or other background mode solely to keep a benchmark alive. Use only a mode that the app genuinely performs and Apple's policy permits. If foreground operation is impossible, design the run into resumable bounded units rather than hiding it behind an unrelated entitlement.

Thermal state, battery state, charging, screen state, and other active Core ML clients can change latency. Bind those conditions into the experiment protocol when they matter, but keep device identifiers out of public receipts.

8. Memory, JIT, AOT, and Jetsam

The raw report preserves a useful failure model but overstates it as universal:

• Community reports: some iPhones exhibit a per-process memory ceiling near 6 GB even when total physical memory is larger.
• Community reports: on-device compilation of large Core ML or MPSGraph packages can create transient resident-memory spikes and trigger jetsam or compiler-service failure.
• Community hypothesis: ahead-of-time compiled .mlmodelc assets can reduce compilation work and permit more file-backed paging than loading an uncompiled package.
• Community reports: the increased-memory-limit entitlement may provide limited or device-dependent relief, with different behavior on iPad.

None of those numbers is portable across devices, OS builds, entitlements, model formats, or Core ML revisions. Measure peak physical footprint in the app, record cold-load and warm-run separately, and pair every unexplained termination with systemCrashLogs.

Prefer a compiled model for device deployment:

xcrun coremlcompiler compile \
  "/path/to/Model.mlpackage" \
  .build/compiled

Unverified mechanism: memory mapping can move some model storage from dirty resident memory to file-backed pages, but .mlmodelc is not a promise of zero runtime compilation or safe peak memory. The proof is a target-device cold-load receipt plus crash scan.

9. Inspect MLComputePlan Correctly

9.1. Current documented Swift surface

Context7 verification against Apple Core ML documentation gives these signatures:

static func load(
    contentsOf url: URL,
    configuration: MLModelConfiguration
) async throws -> MLComputePlan

func deviceUsage(
    for operation: MLModelStructure.Program.Operation
) -> MLComputePlan.DeviceUsage?

func estimatedCost(
    of operation: MLModelStructure.Program.Operation
) -> MLComputePlan.Cost?

deviceUsage(for:) returns nil when usage cannot be determined. Swift DeviceUsage exposes preferred and supported compute devices. estimatedCost(of:) exists for ML Program operations and returns an optional cost. Cost.weight is documented as an estimated workload fraction from 0.0 to 1.0 over the total model evaluation.

9.2. Minimal on-device walk

Pass the compiled .mlmodelc URL and the same configuration used for the model run:

import CoreML
import Foundation

@available(iOS 17.4, *)
func writeComputePlan(
    compiledModelURL: URL,
    configuration: MLModelConfiguration
) async throws {
    let plan = try await MLComputePlan.load(
        contentsOf: compiledModelURL,
        configuration: configuration
    )

    guard case let .program(program) = plan.modelStructure,
          let main = program.functions["main"] else {
        throw BenchmarkError.expectedMLProgram
    }

    for operation in main.block.operations {
        let usage = plan.deviceUsage(for: operation)
        let cost = plan.estimatedCost(of: operation)
        let line = [
            operation.operatorName,
            String(describing: usage?.preferred),
            String(describing: cost?.weight),
        ].joined(separator: "\t")
        FileHandle.standardError.write(Data((line + "\n").utf8))
    }
}

The BenchmarkError case is app-owned and must be defined by the harness.

9.3. What the values prove

• usage?.preferred is anticipated preferred placement for that operation under the supplied configuration. It is not a trace proving where a completed prediction executed.
• usage?.supported can describe available devices where exposed, but support is not selection.
• cost?.weight is an estimate, not milliseconds. Aggregate it within one plan; do not treat it as absolute latency.
• Firsthand implementation evidence: const operations returned nil for both usage and cost in the tested compiled program. Apple documents the optional return and undetermined-usage meaning, not a universal "all const operations are nil" rule. Keep the unwrap for every operation.
• The plan is model-, shape-, configuration-, device-, and OS-specific. A simulator or Mac plan does not certify a physical iPhone.

The raw report called on-device Swift output "irrefutable ground truth." That is too strong. It is more target-specific than an off-device estimate, but it remains a compute plan. Pair it with latency, runtime telemetry, or a trace before claiming actual ANE, GPU, or CPU execution. See the Neural Engine residency guide for that distinction.

10. What Not to Do

11. Debugging Playbook

Copy reports success but the file is absent

1. List the intended parent with device info files.
2. If the parent is absent, copy the containing local directory instead of the file.
3. Pull or app-hash the staged payload before loading it.
4. Preserve the copy JSON and file-list JSON as operational evidence.

Previous app state disappeared

1. Search the driver command for --remove-existing-content.
2. Treat the entire container as potentially wiped.
3. Restore only known public inputs.
4. Remove the flag permanently and move cleanup into bounded app logic.

The app receives --device or other unexpected arguments

1. Capture the app's CommandLine.arguments to stderr or a result file.
2. Move every devicectl option before the bundle identifier.
3. Keep only app arguments after the bundle identifier.

--console is blank

1. Confirm the process exists with device info processes.
2. Emit one flushed stderr line.
3. Poll for an app-owned progress or done file.
4. Pull a historical log archive if OSLog was enabled.

Signing reports "No Accounts"

1. Decide whether the runner is supposed to use an interactive Xcode account. Do not assume.
2. If no account is available, select a manually managed development profile that authorizes the target.
3. Verify that the development identity and private key are visible to the shell.
4. Build with manual signing and inspect the final entitlements before install.

Signing says the profile is Xcode-managed but manual signing is required

1. Stop mixing an Xcode-managed profile with manual signing.
2. Create or obtain a non-Xcode-managed development profile for the app and registered target.
3. Install it without logging its private payload.
4. Rebuild with PROVISIONING_PROFILE_SPECIFIER naming that profile.

The process vanishes during model load

1. Do not retry immediately.
2. Pull systemCrashLogs for the bounded launch window.
3. Check app resource-limit reports and Core ML, E5, or ANE service reports.
4. Separate cold compilation from warm inference.
5. Try an AOT-compiled .mlmodelc as a diagnostic, then remeasure peak footprint.

Compute plan reports no device or no cost

1. Confirm the model is an ML Program and the walked operation belongs to the loaded plan's structure.
2. Keep optional handling; nil means the value could not be determined.
3. Record operation name, model signature, compute units, OS, and device class without a unique identifier.
4. Do not replace missing plan data with guessed placement.

12. Ingest Decision Log

Corrected against current Core ML documentation

Corrected against current command help

Firsthand the reference implementation annotations retained

1. Missing destination parents can make file-to-file copy to silently produce nothing; directory copy works.
2. --remove-existing-content true emptied the entire app data container, contradicting the narrower current help wording.
3. Tokens after the process-launch bundle identifier are app arguments.
4. Swift print was silent under --console; stderr and a Documents result file worked.
5. With no Xcode account signed in, a manually managed development profile authorizing the target worked and an Xcode-managed profile did not.
6. systemCrashLogs is available for crash retrieval.

Preserved as community or unverified

• Multi-gigabyte transfers lack resume and must restart after interruption.
• devicectl launch changes launch-watchdog behavior.
• Foreground activation plus idle-timer suppression is sufficient for a long unattended run.
• A roughly 6 GB per-process ceiling applies to modern iPhones.
• JIT compilation drives transient jetsam failures, while AOT compilation and file-backed mapping reduce the peak.
• Increased-memory-limit entitlements add little headroom on iPhone and more on iPad.
• Core ML, E5 runtime, and ANE compiler services can crash independently of the app.
• Profile filename conventions under ~/Library/MobileDevice/Provisioning Profiles/.
• Inside-out codesign re-signing and fastlane-based credential synchronization.

Each remains concrete enough to test. None is promoted to a target-device fact without a bounded receipt.

Removed with evidence

• The executable log stream --device-udid ... recipe was removed because the verified host rejects that option. The underlying need for live device logging is retained with supported and explicitly unverified alternatives.
• No other substantive mechanism, workaround, deployment route, or failure mode was removed.

Claim-Coverage Audit

Works Cited

1. Apple Core ML: MLComputePlan.load(contentsOf:configuration:) — exact asynchronous Swift load signature.
2. Apple Core ML: deviceUsage(for:) — optional anticipated device usage for an ML Program operation.
3. Apple Core ML: estimatedCost(of:) — optional operation cost.
4. Apple Core ML: Cost.weight — estimated workload fraction from zero to one.
5. Current Xcode 26.6 command help: xcrun devicectl help device copy to, copy from, info files, info processes, process launch, and install app.
6. Current macOS command help: /usr/bin/log stream --help and /usr/bin/log collect --help.
7. Current Xcode export help: xcodebuild -help.