Skip to content

Conversation

@kotlarmilos
Copy link
Member

@kotlarmilos kotlarmilos commented May 19, 2025

Description

XHarness currently retrieves test results over a TCP tunnel between the device and host. On iOS/tvOS 18 and newer, tunnels no longer work. This PR updates the device test runner to write results to file and implements a new ResultFileHandler to copy them to the host.

Changes

  • TestRunner is updated to write results to a file on devices and simulators with version >=18
  • AppTester is updated to invoke ResultFileHandler after test execution
  • ResultFileHandler is implemented to copy results file from device to host

Backwards and forward compatibility

Host can work with old app bundles (without the new test runner) on OS versions < 18. For versions >= 18, apps must be recompiled with the updated test runner to support file‑based result retrieval.

Validation

  • Unit tests for ResultFileHandler are implemented.
  • This PR was tested on both default and scouting queues: 9a2345a and 9a2345a
Testing device Default version Scouting version
iOS 16.5 18.1.1
tvOS 16.5 18.5
iOSSimulator 17.0 N/A
tvOSSimulator 17.0 18.0
MacCatalyst N/A N/A

Next steps

  • Create tracking issue for mlaunch failures: [apple] mlaunch errors with ios/tvos devices macios#23038
  • Re-build test bundles and run tests
  • Confirm device, simulators, and maccatalyst tests are running successfully
  • Release the new XHarness version
  • Pull the new XHarness version in dotnet/runtime and run tests

Fixes #1367 dotnet/macios#22632

@kotlarmilos kotlarmilos self-assigned this May 19, 2025
@kotlarmilos kotlarmilos added the apple iOS/tvOS/WatchOS/Mac Catalyst area label May 19, 2025
@kotlarmilos kotlarmilos marked this pull request as draft May 19, 2025 11:42
@kotlarmilos kotlarmilos changed the title [Apple] Test file listener for Apple mobile devices [Apple] Update XHarness Apple test execution for iOS 18 Jun 5, 2025
@kotlarmilos kotlarmilos marked this pull request as ready for review June 5, 2025 11:45
@kotlarmilos kotlarmilos changed the title [Apple] Update XHarness Apple test execution for iOS 18 [Apple] Handle iOS 18+ test results via file copy instead of TCP tunnel Jun 5, 2025
@kotlarmilos
Copy link
Member Author

kotlarmilos commented Jun 12, 2025

This PR is ready for review. It was tested on both default and scouting queues.

@kotlarmilos kotlarmilos merged commit a859d8d into dotnet:main Jun 17, 2025
17 checks passed
@kotlarmilos kotlarmilos deleted the improvement/remove-tcp-tunnel branch June 17, 2025 09:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

apple iOS/tvOS/WatchOS/Mac Catalyst area

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update iOS device queue to 18.x iOS

7 participants