[NativeAOT] App Store Connect warning ITMS-90999: Invalid executable

[rolfbjarne]

From @tipa on Thu, 02 Nov 2023 18:59:22 GMT

I read about the NativeAOT experimental feature here and wanted to give it a try right away.
The app can be downloaded via TestFlight and is significantly smaller than before (with the Mono AOT) - awesome! I also don't see any functional issues

Steps to Reproduce

1. Add <PublishAot>true</PublishAot> to .csproj
2. dotnet publish
3. Upload resulting .ipa to App Store Connect

Expected Behavior

No warning/error

Actual Behavior

Email from Apple:

We identified one or more issues with a recent delivery for your app, "MyApp" 1.0.0 (1.0.0). Your delivery was successful, but you may wish to correct the following issues in your next delivery: 
ITMS-90999: Invalid executable - The executable in “MyApp.iOS.app” contains multiple segments with the executable permission bit set when it should only contain one. 
After you’ve corrected the issues, you can upload a new binary to App Store Connect. 

Environment

.NET8 RC2.1

Copied from original issue dotnet/macios#19387

[rolfbjarne]

From @rolfbjarne on Mon, 06 Nov 2023 09:46:15 GMT

I've never seen that ITMS message before, and according to Google not many other people has either (https://developer.apple.com/forums/thread/740085 is the only other hit).

Would you be able to attach/provide the .ipa you uploaded?

[rolfbjarne]

From @tipa on Mon, 06 Nov 2023 10:41:02 GMT

I sent you the .ipa file in a private message. Has Store submission of a NativeAot .ipa been tested by Microsoft?

[rolfbjarne]

From @rolfbjarne on Mon, 06 Nov 2023 10:50:23 GMT

I sent you the .ipa file in a private message.

Thanks!

Has Store submission of a NativeAot .ipa been tested by Microsoft?

Yes, but we might have missed warnings.

[rolfbjarne]

From @rolfbjarne on Mon, 06 Nov 2023 10:55:12 GMT

I wonder if this is the reason:

Load command 4
      cmd LC_SEGMENT_64
  cmdsize 152
  segname __THUNKS
   vmaddr 0x0000000100b84000
   vmsize 0x0000000000008000
  fileoff 10977280
 filesize 32768
  maxprot 0x00000005
 initprot 0x00000005
   nsects 1
    flags 0x0
Section
  sectname __thunks
   segname __THUNKS
      addr 0x0000000100b84000
      size 0x0000000000008000
    offset 10977280
     align 2^14 (16384)
    reloff 0
    nreloc 0
     flags 0x80000400
 reserved1 0
 reserved2 0

This is an executable segment, but the __TEXT segment is also executable.

@filipnavara any reason we can't put the __thunks section in the __TEXT segment?

[rolfbjarne]

From @filipnavara on Mon, 06 Nov 2023 12:20:13 GMT

@filipnavara any reason we can't put the __thunks section in the __TEXT segment?

The reason is that the __THUNKS (TEXT) and __THUNKS_DATA (DATA) have to be next to each other. The sections are mapped multiple times to support marshalled delegates. The executable code references the data by relative offset and the data part is modified at runtime.

Fixed number of the marshalled delegates fits into the pre-generated code/data table, and then it gets mapped again at a different memory location. The second mapping needs to a) use the same relative offsets between the code and the data, b) still needs the code part to have valid code signature.

If Apple started blocking that we have a problem. I don't know on any method to ensure an executable layout that would allow the remapping.

[rolfbjarne]

From @rolfbjarne on Mon, 06 Nov 2023 12:25:15 GMT

I don't know on any method to ensure an executable layout that would allow the remapping.

Isn't this what Mono has been doing for years (with no app store warnings)?

[rolfbjarne]

From @filipnavara on Mon, 06 Nov 2023 12:35:05 GMT

Isn't this what Mono has been doing for years (with no app store warnings)?

I frankly have no idea. I think Mono may have some fixed limit... @lambdageek, any pointers how marshalled delegates are implemented on Mono/iOS?

[rolfbjarne]

From @rolfbjarne on Mon, 06 Nov 2023 14:49:14 GMT

This looks related:

runtime/src/mono/mono/mini/aot-compiler.c

Lines 2065 to 2081 in 6066de1

	/*
	* arch_emit_specific_trampoline_pages:
	*
	* Emits a page full of trampolines: each trampoline uses its own address to
	* lookup both the generic trampoline code and the data argument.
	* This page can be remapped in process multiple times so we can get an
	* unlimited number of trampolines.
	* Specifically this implementation uses the following trick: two memory pages
	* are allocated, with the first containing the data and the second containing the trampolines.
	* To reduce trampoline size, each trampoline jumps at the start of the page where a common
	* implementation does all the lifting.
	* Note that the ARM single trampoline size is 8 bytes, exactly like the data that needs to be stored
	* on the arm 32 bit system.
	*/
	static void
	arch_emit_specific_trampoline_pages (MonoAotCompile *acfg)
	{

runtime/src/mono/mono/mini/aot-runtime.c

Lines 5743 to 5764 in 6066de1

	addr = 0;
	/* allocate two contiguous pages of memory: the first page will contain the data (like a local constant pool)
	* while the second will contain the trampolines.
	*/
	do {
	ret = vm_allocate (mach_task_self (), &addr, psize * 2, VM_FLAGS_ANYWHERE);
	} while (ret == KERN_ABORTED);
	if (ret != KERN_SUCCESS) {
	g_error ("Cannot allocate memory for trampolines: %d", ret);
	break;
	}
	/*g_warning ("allocated trampoline double page at %x", addr);*/
	/* replace the second page with a remapped trampoline page */
	taddr = addr + psize;
	vm_deallocate (mach_task_self (), taddr, psize);
	ret = vm_remap (mach_task_self (), &taddr, psize, 0, FALSE, mach_task_self(), (vm_address_t)tpage, FALSE, &prot, &max_prot, VM_INHERIT_SHARE);
	if (ret != KERN_SUCCESS) {
	/* someone else got the page, try again */
	vm_deallocate (mach_task_self (), addr, psize);
	continue;
	}
	/*g_warning ("remapped trampoline page at %x", taddr);*/

[rolfbjarne]

From @charlesroddie on Thu, 09 Nov 2023 18:41:11 GMT

Also got this on uploading to testflight. Something new between dotnet8-rc1 and latest dotnet8.

[rolfbjarne]

From @filipnavara on Sun, 12 Nov 2023 16:25:01 GMT

The vm_remap trick in Mono is actually quite cool. I think it can be improved with the VM_FLAGS_OVERWRITE flag to avoid the retry loop and it may be quite reliable way to map the trampoline pages without having them as consecutive blocks in the executable file.

/cc @ivanpovazan

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

From @tipa on Thu, 02 Nov 2023 18:59:22 GMT

I read about the NativeAOT experimental feature here and wanted to give it a try right away.
The app can be downloaded via TestFlight and is significantly smaller than before (with the Mono AOT) - awesome! I also don't see any functional issues

Steps to Reproduce

1. Add <PublishAot>true</PublishAot> to .csproj
2. dotnet publish
3. Upload resulting .ipa to App Store Connect

Expected Behavior

No warning/error

Actual Behavior

Email from Apple:

We identified one or more issues with a recent delivery for your app, "MyApp" 1.0.0 (1.0.0). Your delivery was successful, but you may wish to correct the following issues in your next delivery: 
ITMS-90999: Invalid executable - The executable in “MyApp.iOS.app” contains multiple segments with the executable permission bit set when it should only contain one. 
After you’ve corrected the issues, you can upload a new binary to App Store Connect. 

Environment

.NET8 RC2.1

Copied from original issue dotnet/macios#19387

Author:	rolfbjarne
Assignees:	-
Labels:	untriaged, area-NativeAOT-coreclr
Milestone:	-