MuggleAI
Remote Testing with MCP / iOS App Testing

iOS App Testing

Upload iOS simulator builds, run acceptance tests on cloud simulators, and gate CI on the verdict.

iOS App Testing

Muggle Test runs acceptance tests against iOS apps on cloud simulators. You upload a simulator build of your app, describe a goal in plain language, and an agent installs the build on a simulator and works through the app toward that goal, recording each step with a screenshot.

On This Page

SectionDescription
RequirementsWhat a build must be
Run LifecycleUpload, queue, run, verdict
Status and VerdictReading a run's result
From Your AI AssistantTesting through the Muggle MCP
From CIGating a pipeline on the verdict
LimitationsWhat is not supported

Requirements

RequirementDetail
Build typeA .app bundle built for the iOS Simulator (-sdk iphonesimulator), zipped
Archive sizeAt most 512 MB
Build machinemacOS with Xcode, to produce the simulator build
ProjectA Muggle Test project you can edit

Build and zip the app:

xcodebuild -scheme MyApp -sdk iphonesimulator -configuration Debug \
  -derivedDataPath build CODE_SIGNING_ALLOWED=NO
ditto -c -k --keepParent build/Build/Products/Debug-iphonesimulator/MyApp.app MyApp.app.zip

Run Lifecycle

Cloud simulatorMuggle TestYouCloud simulatorMuggle TestYouUpload MyApp.app.zipBuild digestQueue a run (digest, task, device)Install the build, drive the appSteps, screenshots, verdictRead the run
  • Builds are content-addressed. Each upload is identified by the sha256 digest of the zip. Uploading an unchanged build again returns the stored one.
  • A run names one goal. Give each run one observable behavior, such as "Sign in and add the first product to the cart". Queue several runs to cover several behaviors; they run in parallel.
  • Credentials stay in the secret store. A run that signs in references project secrets by name, and the simulator host resolves them when it starts the run. Secret values never appear in the task.

Status and Verdict

A run reports two separate fields. A pipeline gate reads both.

FieldValuesMeaning
statusqueued, claimed, running, completed, infra_error, timed_out, cancelledWhether the run finished executing
verdictpassed, failed, goal_unachievableWhat the test found; present only once completed

queued

claimed

running

completed

infra_error

timed_out

cancelled

OutcomeStatusVerdictTreat as
App did what the task askedcompletedpassedPass
App did something wrongcompletedfailedFail
The goal cannot be reached in this appcompletedgoal_unachievableFail
The run itself brokeinfra_error, timed_out, cancellednoneInconclusive; retry the run

A run waits in queued until a simulator is free; it never fails for lack of capacity.

From Your AI Assistant

With the Muggle MCP installed, ask your assistant to test the app:

Test my iOS app: upload build/MyApp.app.zip and check that adding a note works.

The /muggle:muggle-test skill recognizes an iOS build and uses the iOS tools. The upload tool reads the zip from disk itself, so the assistant never handles the archive. See iOS App Tools for each tool's input and output.

From CI

Upload each build from your pipeline and fail the job unless the verdict is passed. See iOS App Builds for a GitHub Actions workflow.

Limitations

  • iOS Simulator builds only; physical devices and App Store or TestFlight builds are not supported.
  • One app per run; flows that leave the app under test are out of scope.
  • iOS runs do not yet appear in the PR visual walkthrough; read results with the run tools or the API.

Was this article helpful?