[Xamarin.Android.Build.Tasks] Microsoft.Resource.Designer Assembly

[dellis1972]

Context #6310

The goal of this commit is to replace the existing Resource.designer.cs
generation code with a newer Reference Assembly based system. This new
system will result in both smaller and faster starting applications.
It also allows F# users to get the same experience as C# users, this makes
the https://github.com/xamarin/Xamarin.Android.FSharp.ResourceProvider
obsolete. F# projects will no longer need to use this work around.

The Old System

The old system generated a Resource.designer.cs file for each library
and application project. This means that each assembly had a copy of
the same code (almost). This code declared a field for each android resource
in the project AND any resource declared in a referenced assembly or
.aar file. This can result in 1000's of fields ending up in each Resource
class.

Because we only know the final Id values are app packaging time library
projects could not know those values at build time. This meant that we
needed to update those library values at startup with the ones that were
compiled into the final application project. This is handled by the
UpdateIdValues method. This method is called by reflection on app
startup and contains code such as

global::SkiaSharp.Views.Windows.Resource.Styleable.View_paddingEnd = global::testapp.Mobile.Resource.Styleable.View_paddingEnd;
global::SkiaSharp.Views.Windows.Resource.Styleable.View_paddingStart = global::testapp.Mobile.Resource.Styleable.View_paddingStart;
global::SkiaSharp.Views.Windows.Resource.Styleable.View_theme = global::testapp.Mobile.Resource.Styleable.View_theme;

And it does this for EVERY field that is referenced by the library projects.
This can result in 1000s of fields being updated on app startup. This impacts
performance quite allot.

One of the major problems with this system in the inability to properly link
out unused fields. This is because the UpdateIdValues method ends up referencing
EVERY field. So the linker has nothing to work with. Custom linker steps were
introduced via the AndroidLinkResources MSBuild feature, but this is not a
great solution and can be quite unstable since in relies on searching for specific
IL patterns. We need a better solution.

The New System

The new system takes advantage of newer technology which has been available
for a few years now, the Reference Assembly. These types of assembly are
designed to be replaced at runtime, they are generally used to provide
placeholder API's which can be swapped out later. So the idea of the new system
is fairly simple. Library projects will at build time generate a Reference Assembly
which will contain similar classes to those in a Resource.designer.cs file.
The main difference is this will be properties rather than fields. This
Reference Assembly does not need to shipped with the library or copied anywhere it
is purely there to provide an API to which the library can compile against.

The main application will then at build time generate the final full assembly. This
is the one which will be packaged with the application. Because it will have
the same name and API as the Reference Assembly the libraries generated the runtime
will redirect all the calls to this new assembly. This has a number of benefits

1. All the property declarations are in one place and are not
   duplicated (ish.. more on that later). As a result the size of the app will be
   reduced.
2. Because we no longer need the UpdateIdValues method, start up time will be
   reduced.
3. The linker can now do its job and property link out unused properties. This further
   reduces application size.
4. F# is now fully supported.

The Details

The new system will make use of Mono.Cecil to generate the IL for the assembly. While
this new system could be written to generate source code and then call the csc compiler
manually, it was deemed more efficient to generate the IL directly. For library projects
the new assembly will contain the same classes as the old Resource.designer.cs file.
The only difference is these will use properties rather than fields. The Resource
properties will be generated directly from the R.txt which aapt2 emits as part of
the build process.

For applications however we need to generate two Resource classes. This is for historical
and backward compatibility reasons. The old system generated const values for the application
Resource.designer.cs file rather than the static values for libraries. This was so that
the const values could be used directly by the compiler in the main app. As a result users
were able to write switch statements such as

switch (id) {
    case Resource.Id.foo:
        // do something
        break;
    case Resource.Id.bar:
        // do something
        break;
}

which is only possible if you use constant values. We need to maintain this feature since we
do not want to break code when people upgrade. So we need a way to use properties as well as
const fields in the Resource nested classes so the entire system works. We do this by
generating two classes. The one called Resource contains all the nested types which contains
the property based API. This will be used by ALL the library projects in the final app.
Then we have one called ResourceConstant which contains const fields and is ONLY used by
the final application. This way ALL the ResourceConstant values will be directly used in the
app and we will maintain backward compatibility.

One thing we haven't mentioned so far is namespaces. In the old system the Resource class
was generated in the RootNamespace of the library project. So someone could do the following

var foo = MyLibrary.Resource.Id.foo;

Because ALL the resources are now in a single assembly they need to be in one
common namespace. In this case its Microsoft.Android.Resource.Designer.
However we still need to maintain that backward compatibility. So during the
build process we generate a file __Microsoft.Android.Resource.Designer.cs.
This file contains code like this

namespace MyLibrary {
    public class Resource : Microsoft.Android.Resource.Designer.Resource {

    }
}

This file is compiled into the library or application and exposes the Resource
class in the correct namespace. This maintains backward compatibility.

The new build system introduces a number of new Tasks and Targets to bring this
all together. It also unify's some code between the old and new system which
would otherwise be duplicated. The old system will be maintained for now for
backward compatibility, however the new system will be on by default for .net 8.

The new system is mostly contained in the Xamarin.Android.Resource.Designer.targets
file. The entry point for this set of targets is _BuildResourceDesigner. This
will only be run if the global MSbuild property AndroidUseDesignerAssembly is
set to true, as it will be for .NET 8+.

New tasks are as follows.

• GenerateRtxt is responsible for scanning the resource directory and generating
  an aapt2 compatible R.txt file. This will be used by the GenerateResourceDesignerAssembly.

• GenerateResourceCaseMap is responsible for generating a casemap.txt file
  which will map the all lower case android resources to the casing required
  for the C# code. Android requires ALL resources be lower case, but our system
  allows the user to define the case using any system then want. This task handles
  generating this mapping between what the android system needs and what the user
  is expecting. Its output is used by the GenerateResourceDesignerAssembly task
  when generating the IL in _Microsoft.Android.Resource.Designer.dll. It is
  also used by the old system to generate the same file.

• GenerateResourceDesignerIntermediateClass is responsible for generating
  the __Microsoft.Android.Resource.Designer.csfile in the $(IntermediateOutputPath).

• GenerateResourceDesignerAssembly is the key to the whole system. This task will
  read the R.xt file and generate a _Microsoft.Android.Resource.Designer.dll assembly
  in the $(IntermediateOutputPath).
  This task is called in two places. The first is in _GenerateResourceDesignerAssembly,
  this is called as part of the build which happens just before CoreCompile and only
  for design time builds.
  It is also called in _UpdateAndroidResgen which happens as part of the build and runs
  just after aapt2 is called. This makes sure we always use the most up to date
  version of R.txt to generate the new assembly.

Because we are using the R.txt file to drive the generation of the new assembly,
we needed some way for that to work when aapt2 was not being run. This usually
happens on a first time design time build. The current system has a class
GenerateResourceDesigner which is responsible for both scanning the resources
and generating a design time Resource.designer.cs file. While we could have
duplicated the code it made more sense to split our the resource scanner into
its own class. So we have a new task called GenerateRtxt which is responsible
for scanning the resources and generating an R.txt file. This is only used
when we are not doing a full build with aapt2. This new task lets us generate
the needed R.txt which can then be used by both the old and new system to generate
their respective outputs.

As part of this we have two other classes RtxtReader and RtxtWriter. The
RtxtReader unify's the code which was used to read the values of the R.txt
into one class which can be used by both systems. The RtxtWriter is responsible
for writing the R.txt file for design time builds. Again it will be used by
both the old and new system.

The _AddResourceDesignerFiles is a new target which is responsible for making
sure that the new assembly and __Microsoft.Android.Resource.Designer.cs get
added to the correct item groups. These are @(ReferencePath) for the assembly
and @(Compile) for the source file. In the case of F# the __Microsoft.Android.Resource.Designer.fs
file which gets generated has to be added to the @(CompileBefore) ItemGroup,
this is so that the types are resolved in the correct order.

To ensure that the new assembly is added to the final application we have to
introduce the _AddResourceDesignerToPublishFiles target. This target makes sure
that the new assembly is added to the @(ResolvedFileToPublish) ItemGroup. It also
adds the require MetaData items such as IsTrimmable and PostprocessAssembly which
are required to get the assembly linked correctly.

Results

Here is a summary of the kinds of results one might see in an application.
Results will vary as it will depend on how many resources are used in both libraries
and applications. However this should give a good baseline.

Basic Android Application

Apk Size changes

Size difference in bytes ([*1] apk1 only, [*2] apk2 only):
  +       3,800 lib/x86_64/libaot-_Microsoft.Android.Resource.Designer.dll.so *2
  +       3,792 lib/arm64-v8a/libaot-_Microsoft.Android.Resource.Designer.dll.so *2
  +       2,644 lib/armeabi-v7a/libaot-_Microsoft.Android.Resource.Designer.dll.so *2
  +       2,600 lib/x86/libaot-_Microsoft.Android.Resource.Designer.dll.so *2
  +       1,139 assemblies/_Microsoft.Android.Resource.Designer.dll *2
  +         663 META-INF/BNDLTOOL.SF
  +         663 META-INF/MANIFEST.MF
  +         544 lib/x86_64/libxamarin-app.so
  +         512 lib/arm64-v8a/libxamarin-app.so
  +         308 lib/armeabi-v7a/libxamarin-app.so
  +         284 lib/x86/libxamarin-app.so
  +          48 classes.dex
    +           1 strings count
    +          44 data section size
  +          32 lib/x86/libaot-System.Private.CoreLib.dll.so
  -          32 lib/armeabi-v7a/libaot-System.Private.CoreLib.dll.so
  -          56 lib/arm64-v8a/libaot-System.Private.CoreLib.dll.so
  -          61 assemblies/armeabi-v7a/System.Runtime.dll
  -          61 assemblies/x86/System.Runtime.dll
  -          63 assemblies/arm64-v8a/System.Runtime.dll
  -          63 assemblies/x86_64/System.Runtime.dll
  -          64 lib/x86_64/libaot-System.Private.CoreLib.dll.so
  -          96 lib/x86_64/libaot-Mono.Android.dll.so
  -          96 lib/x86/libaot-Mono.Android.dll.so
  -         112 lib/arm64-v8a/libaot-Mono.Android.dll.so
  -         160 lib/armeabi-v7a/libaot-Mono.Android.dll.so
  -         178 assemblies/armeabi-v7a/System.Private.CoreLib.dll
    Type System.AppDomain
      -             Method public System.Reflection.Assembly[] GetAssemblies ()
    -             Type System.CodeDom.Compiler.GeneratedCodeAttribute
    Type System.Runtime.Loader.AssemblyLoadContext
      -             Method static System.Reflection.Assembly[] GetLoadedAssemblies ()
      -             Method static System.Reflection.Assembly[] InternalGetLoadedAssemblies ()
    Type System.Reflection.Assembly
      -             Method public static System.Reflection.Assembly GetExecutingAssembly ()
      -             Method static System.Reflection.RuntimeAssembly GetExecutingAssembly (System.Threading.StackCrawlMark&)
  -         210 assemblies/x86/System.Private.CoreLib.dll
    Type System.AppDomain
      -             Method public System.Reflection.Assembly[] GetAssemblies ()
    -             Type System.CodeDom.Compiler.GeneratedCodeAttribute
    Type System.Runtime.Loader.AssemblyLoadContext
      -             Method static System.Reflection.Assembly[] GetLoadedAssemblies ()
      -             Method static System.Reflection.Assembly[] InternalGetLoadedAssemblies ()
    Type System.Reflection.Assembly
      -             Method public static System.Reflection.Assembly GetExecutingAssembly ()
      -             Method static System.Reflection.RuntimeAssembly GetExecutingAssembly (System.Threading.StackCrawlMark&)
  -         223 assemblies/x86_64/System.Private.CoreLib.dll
    Type System.AppDomain
      -             Method public System.Reflection.Assembly[] GetAssemblies ()
    -             Type System.CodeDom.Compiler.GeneratedCodeAttribute
    Type System.Runtime.Loader.AssemblyLoadContext
      -             Method static System.Reflection.Assembly[] GetLoadedAssemblies ()
      -             Method static System.Reflection.Assembly[] InternalGetLoadedAssemblies ()
    Type System.Reflection.Assembly
      -             Method public static System.Reflection.Assembly GetExecutingAssembly ()
      -             Method static System.Reflection.RuntimeAssembly GetExecutingAssembly (System.Threading.StackCrawlMark&)
  -         230 assemblies/arm64-v8a/System.Private.CoreLib.dll
    Type System.AppDomain
      -             Method public System.Reflection.Assembly[] GetAssemblies ()
    -             Type System.CodeDom.Compiler.GeneratedCodeAttribute
    Type System.Runtime.Loader.AssemblyLoadContext
      -             Method static System.Reflection.Assembly[] GetLoadedAssemblies ()
      -             Method static System.Reflection.Assembly[] InternalGetLoadedAssemblies ()
    Type System.Reflection.Assembly
      -             Method public static System.Reflection.Assembly GetExecutingAssembly ()
      -             Method static System.Reflection.RuntimeAssembly GetExecutingAssembly (System.Threading.StackCrawlMark&)
  -         614 assemblies/Mono.Android.dll
    -             Type Android.Runtime.ResourceDesignerAttribute
    -             Type Android.Runtime.ResourceIdManager
  -       1,296 lib/arm64-v8a/libaot-DotNet6AndroidTest.Library.dll.so
  -       1,320 lib/x86_64/libaot-DotNet6AndroidTest.Library.dll.so
  -       1,320 lib/x86/libaot-DotNet6AndroidTest.Library.dll.so
  -       1,632 lib/armeabi-v7a/libaot-DotNet6AndroidTest.Library.dll.so
  -      34,893 assemblies/DotNet6AndroidTest.Library.dll
    Type DotNet6AndroidTest.Library.Resource
      -             CustomAttribute System.CodeDom.Compiler.GeneratedCodeAttribute
      -             Method static void .cctor ()
      -             Type Animation
      -             Type Animator
      -             Type Attribute
      -             Type Boolean
      -             Type Color
      -             Type Dimension
      -             Type Drawable
      -             Type Id
      -             Type Integer
      -             Type Interpolator
      -             Type Layout
      -             Type String
      -             Type Style
      -             Type Styleable
    -             Type <PrivateImplementationDetails>
  -      57,449 assemblies/DotNet6AndroidTest.App.dll
    Type DotNet6AndroidTest.App.Resource
      -             CustomAttribute System.CodeDom.Compiler.GeneratedCodeAttribute
      -             Method static void .cctor ()
      -             Method public static void UpdateIdValues ()
      -             Type Animation
      -             Type Animator
      -             Type Attribute
      -             Type Boolean
      -             Type Color
      -             Type Dimension
      -             Type Drawable
      -             Type Id
      -             Type Integer
      -             Type Interpolator
      -             Type Layout
      -             Type Mipmap
      -             Type String
      -             Type Style
      -             Type Styleable
    -             Type <PrivateImplementationDetails>
  -      60,032 lib/x86/libaot-DotNet6AndroidTest.App.dll.so
  -      60,640 lib/x86_64/libaot-DotNet6AndroidTest.App.dll.so
  -      79,680 lib/arm64-v8a/libaot-DotNet6AndroidTest.App.dll.so
  -      87,472 lib/armeabi-v7a/libaot-DotNet6AndroidTest.App.dll.so
Summary:
  +       1,326 Other entries 0.20% (of 666,811)
  -      92,906 Assemblies -3.71% (of 2,504,510)
  +          48 Dalvik executables 0.00% (of 1,209,784)
  -     279,492 Shared libraries -1.17% (of 23,968,748)
  -     174,592 Uncompressed assemblies -3.41% (of 5,123,584)
  -     167,406 Package size difference -1.25% (of 13,418,690)

From this example we reduce the package size by 167kb. Allot of code is removed.

Here is another example of a release aab file.

Size difference in bytes ([*1] apk1 only, [*2] apk2 only):
  +       3,744 base/lib/x86_64/libaot-_Microsoft.Android.Resource.Designer.dll.so *2
  +       3,736 base/lib/arm64-v8a/libaot-_Microsoft.Android.Resource.Designer.dll.so *2
  +       2,564 base/lib/armeabi-v7a/libaot-_Microsoft.Android.Resource.Designer.dll.so *2
  +       2,544 base/lib/x86/libaot-_Microsoft.Android.Resource.Designer.dll.so *2
  +         563 META-INF/ANDROIDD.SF
  +         563 META-INF/MANIFEST.MF
  +         464 base/lib/x86_64/libxamarin-app.so
  +         456 base/lib/arm64-v8a/libxamarin-app.so
  +         284 base/lib/armeabi-v7a/libxamarin-app.so
  +         260 base/lib/x86/libxamarin-app.so
  +          88 base/root/assemblies/assemblies.manifest
  +          52 base/dex/classes.dex
    +           1 strings count
    +          48 data section size
  -       7,384 base/lib/x86_64/libaot-DotNet6AndroidBasic.dll.so
  -       8,152 base/lib/x86/libaot-DotNet6AndroidBasic.dll.so
  -       8,808 base/lib/arm64-v8a/libaot-DotNet6AndroidBasic.dll.so
  -       9,264 base/lib/armeabi-v7a/libaot-DotNet6AndroidBasic.dll.so
  -      37,046 base/root/assemblies/assemblies.blob
Summary:
  -      35,832 Other entries -1.06% (of 3,385,849)
  +          52 Dalvik executables 0.00% (of 2,074,712)
  -      19,556 Shared libraries -0.04% (of 55,776,348)
  -      30,141 Package size difference -0.11% (of 27,615,908)

The following show that we can get a 2% reduction in startup time on a basic
Android application.

Total init

All runs

Before	After	Δ	Notes
55.486	54.094	-2.51% ✓	defaults; 64-bit build

Without slowest and fastest runs

Before	After	Δ	Notes
55.723	54.306	-2.54% ✓	defaults; 64-bit build

Without the slowest runs

Before	After	Δ	Notes
55.318	53.900	-2.56% ✓	defaults; 64-bit build

A more complete example of a basic android app which one SkiaSharp view

Displayed

All runs

Before	After	Δ	Notes
292.100	297.400	+1.78% ✗	defaults; 32-bit build
295.750	296.000	+0.08% ✗	defaults; profiled AOT; 32-bit build
297.050	295.800	-0.42% ✓	defaults; full AOT; 32-bit build
387.300	386.850	-0.12% ✓	defaults; full AOT+LLVM; 32-bit build
292.750	291.750	-0.34% ✓	defaults; 64-bit build
299.350	299.950	+0.20% ✗	defaults; profiled AOT; 64-bit build
293.350	296.500	+1.06% ✗	defaults; full AOT; 64-bit build
392.900	391.650	-0.32% ✓	defaults; full AOT+LLVM; 64-bit build

Without slowest and fastest runs

Before	After	Δ	Notes
290.556	297.000	+2.17% ✗	defaults; 32-bit build
296.222	295.667	-0.19% ✓	defaults; profiled AOT; 32-bit build
297.111	295.944	-0.39% ✓	defaults; full AOT; 32-bit build
387.222	387.444	+0.06% ✗	defaults; full AOT+LLVM; 32-bit build
292.333	292.056	-0.10% ✓	defaults; 64-bit build
299.333	299.056	-0.09% ✓	defaults; profiled AOT; 64-bit build
293.944	296.444	+0.84% ✗	defaults; full AOT; 64-bit build
393.333	391.889	-0.37% ✓	defaults; full AOT+LLVM; 64-bit build

Without the slowest runs

Before	After	Δ	Notes
290.105	296.000	+1.99% ✗	defaults; 32-bit build
295.053	294.947	-0.04% ✓	defaults; profiled AOT; 32-bit build
296.158	294.895	-0.43% ✓	defaults; full AOT; 32-bit build
385.684	385.842	+0.04% ✗	defaults; full AOT+LLVM; 32-bit build
291.526	291.053	-0.16% ✓	defaults; 64-bit build
297.947	298.316	+0.12% ✗	defaults; profiled AOT; 64-bit build
292.789	295.632	+0.96% ✗	defaults; full AOT; 64-bit build
391.526	390.474	-0.27% ✓	defaults; full AOT+LLVM; 64-bit build

Native to managed

All runs

Before	After	Δ	Notes
28.151	29.251	+3.76% ✗	defaults; 32-bit build
28.873	28.959	+0.30% ✗	defaults; profiled AOT; 32-bit build
28.726	28.604	-0.42% ✓	defaults; full AOT; 32-bit build
70.550	73.070	+3.45% ✗	defaults; full AOT+LLVM; 32-bit build
29.180	28.520	-2.26% ✓	defaults; 64-bit build
29.024	28.815	-0.72% ✓	defaults; profiled AOT; 64-bit build
28.633	28.709	+0.26% ✗	defaults; full AOT; 64-bit build
74.205	72.610	-2.15% ✓	defaults; full AOT+LLVM; 64-bit build

Without slowest and fastest runs

Before	After	Δ	Notes
28.320	29.434	+3.78% ✗	defaults; 32-bit build
29.067	29.130	+0.21% ✗	defaults; profiled AOT; 32-bit build
28.870	28.748	-0.42% ✓	defaults; full AOT; 32-bit build
70.733	73.500	+3.77% ✗	defaults; full AOT+LLVM; 32-bit build
29.151	28.710	-1.51% ✓	defaults; 64-bit build
29.192	28.816	-1.29% ✓	defaults; profiled AOT; 64-bit build
28.803	28.910	+0.37% ✗	defaults; full AOT; 64-bit build
74.730	73.077	-2.21% ✓	defaults; full AOT+LLVM; 64-bit build

Without the slowest runs

Before	After	Δ	Notes
28.074	29.132	+3.63% ✗	defaults; 32-bit build
28.763	28.870	+0.37% ✗	defaults; profiled AOT; 32-bit build
28.640	28.466	-0.61% ✓	defaults; full AOT; 32-bit build
70.103	72.736	+3.62% ✗	defaults; full AOT+LLVM; 32-bit build
29.108	28.441	-2.29% ✓	defaults; 64-bit build
28.949	28.710	-0.83% ✓	defaults; profiled AOT; 64-bit build
28.552	28.617	+0.23% ✗	defaults; full AOT; 64-bit build
73.989	72.342	-2.23% ✓	defaults; full AOT+LLVM; 64-bit build

Total init

All runs

Before	After	Δ	Notes
53.909	55.524	+2.91% ✗	defaults; 32-bit build
54.837	54.636	-0.37% ✓	defaults; profiled AOT; 32-bit build
54.645	54.621	-0.04% ✓	defaults; full AOT; 32-bit build
103.731	107.067	+3.12% ✗	defaults; full AOT+LLVM; 32-bit build
55.216	54.779	-0.79% ✓	defaults; 64-bit build
55.099	55.016	-0.15% ✓	defaults; profiled AOT; 64-bit build
54.455	54.859	+0.74% ✗	defaults; full AOT; 64-bit build
108.140	106.496	-1.52% ✓	defaults; full AOT+LLVM; 64-bit build

Without slowest and fastest runs

Before	After	Δ	Notes
54.185	55.864	+3.01% ✗	defaults; 32-bit build
55.067	54.934	-0.24% ✓	defaults; profiled AOT; 32-bit build
54.941	54.854	-0.16% ✓	defaults; full AOT; 32-bit build
104.081	107.616	+3.28% ✗	defaults; full AOT+LLVM; 32-bit build
55.133	55.041	-0.17% ✓	defaults; 64-bit build
55.367	55.058	-0.56% ✓	defaults; profiled AOT; 64-bit build
54.722	55.192	+0.85% ✗	defaults; full AOT; 64-bit build
108.873	106.938	-1.78% ✓	defaults; full AOT+LLVM; 64-bit build

Without the slowest runs

Before	After	Δ	Notes
53.773	55.348	+2.85% ✗	defaults; 32-bit build
54.591	54.467	-0.23% ✓	defaults; profiled AOT; 32-bit build
54.484	54.392	-0.17% ✓	defaults; full AOT; 32-bit build
103.158	106.542	+3.18% ✗	defaults; full AOT+LLVM; 32-bit build
55.061	54.586	-0.86% ✓	defaults; 64-bit build
54.928	54.865	-0.11% ✓	defaults; profiled AOT; 64-bit build
54.297	54.713	+0.76% ✗	defaults; full AOT; 64-bit build
107.845	105.985	-1.72% ✓	defaults; full AOT+LLVM; 64-bit build

[dellis1972]

• Figure out if its best to use native assemblies (monodroid/.net6) or netstandard2.0 for the designer dll.
• Write a code generator to handle the namespaces for library projects so these can redirect the old Resource designer class to the new one.
• Figure out DTB
• Make sure the designer gets updated
• Check Strong Named assemblies see docs

[jonathanpeppers]

Looks like there is a bit still WIP, but I left a couple comments.

[dellis1972]

For reference some performance differences https://gist.github.com/dellis1972/f5a9c9475dadea1331c6c62b1a478cd8 for .net 6

[jonathanpeppers]

The last concern I have is this comment. Do we need to ask other teams about this? .NET MAUI team at least?

Any existing libraries from Telerik/DevExpress, will need to be updated to work in .NET 8? (If those are mostly UI, they would likely use a value from Resource.designer.cs)

[dellis1972]

old style Assemblies will be auto upgraded by the linker during a build. So existing NuGet packages will work.

[jonathanpeppers]

Ok, so you only hit an issue if you turn the setting off, got it.

[jonpryor]

Updated draft commit message:

[Xamarin.Android.Build.Tasks] _Microsoft.Android.Resource.Designer (#6427)

Fixes: https://github.com/xamarin/xamarin-android/issues/6310

Context: https://github.com/dotnet/runtime/commit/60d9b98938a9003d937efdaa53dfe6f0033de9bb
Context: https://github.com/dotnet/fsharp/issues/12640
Context: 103b5a755c048c6eaedb43c139cac32b920372cd   Optimize ResourceIdManager.UpdateIdValues() invocations
Context: 9e6ce03ca2d72f5415a2a1650185ba46113cc3dd   Adds $(AndroidLinkResource)
Context: 522d7fb61f3669d85d077ba4f2889dbe8c9c8ac3
Context: 9c0437866c7308794283b76125483adecdea9067   (AndroidEnablePreloadAssemblies crash)
Context: d521ac0280c0ad165570077a860cb1846025010b   (Styleables array values)

Replace the existing `Resource.designer.cs` generation code with a
new system that relies on Reference Assemblies.  This results in
smaller apps and faster startup.


~~ Bind `@(AndroidRoesource)` values as fields ~~

The original approach to binding `@(AndroidResource)` values was to
Do What Java Does™: there are two "styles" of `Resource.designer.cs`
files, one for Library projects, and one for App projects.

`Resource.designer.cs` for Library projects involves mutable read/write
fields:

	[assembly: Android.Runtime.ResourceDesignerAttribute ("ExampleLib.Resource", IsApplication=false)]
	namespace ExampleLib;
	partial class Resource {
	  partial class String {
	    public static int app_name = 2130771968;
	    static String() {
	      global::Android.Runtime.ResourceIdManager.UpdateIdValues();
	    }
	  }
	  partial class Styleable {
	    public static int[] MyLibraryWidget = new int[]{…};
	    static Styleable() {
	      global::Android.Runtime.ResourceIdManager.UpdateIdValues();
	    }
	  }
	}

`Resource.designer.cs` for App projects involves *`const`* fields:

	[assembly: Android.Runtime.ResourceDesignerAttribute ("App.Resource", IsApplication=true)]
	namespace App;
	partial class Resource {
	  partial class String {
	    public const int app_name = 2130968576;
	    static String() {
	      global::Android.Runtime.ResourceIdManager.UpdateIdValues();
	    }
	  }
	  partial class Styleable {
	    public static int[] MyLibraryWidget = new int[]{…}; // still read+write, not const
	    static Styleable() {
	      global::Android.Runtime.ResourceIdManager.UpdateIdValues();
	    }
	  }
	}

There is a field each Android `resource` in the project *and* any
`resource`s declared in a referenced assembly or `.aar` files.
This can result in 1000's of fields ending up in each `Resource` class.

Because we only know the final `Id` values at app packaging time,
library projects could not know those values at build time.  This meant
that we needed to update those library values at startup with the ones
that were compiled into the final application project.  This is handled
by the `Resource.UpdateIdValues()` method.  This method is called by
reflection on app startup and contains code to set the read/write
fields for *all* `Resource` types from *all referenced assemblies*:

	partial class Resource {
	  public static void UpdateIdValues() {
	    global::ExampleLib.Resource.String.app_name = String.app_name;
	    // plus all other resources
	  }
	}

**Pros**:

  * It's a "known good" construct, as it's what Java does!
    (Or *did*, circa 12 years ago…)

**Cons**:

  * There is a semantic difference between the use of the `Resource`
    types between Library and App projects: in an App project, you
    can use Resource IDs in switch `case`s, e.g.
    `case Resource.String.app_name: …`.
    This is not possible in Library projects.

  * As the App `Resource.UpdateIdValues()` method references *all*
    fields from all referenced libraries, the linker is not able to
    remove any of the fields.  This pattern is linker hostile.
    This results in larger `.apk` sizes, though this can be optimized
    via [`$(AndroidLinkResources)`][0] (9e6ce03c, d521ac02).

  * As the App `Resource.UpdateIdValues()` method references *all*
    fields from all referenced libraries, the method can be *huge*;
    it depends on how many resources the App and all dependencies
    pull in.  We have seen cases where the size of
    `Resource.UpdateIdValues()` would cause the interpreter to crash,
    breaking certain Hot Reload scenarios.
    (Fixed in dotnet/runtime@60d9b989).

  * The `Resource.UpdateIdValues()` method needs to be invoked during
    process startup, *before* any assemblies try to use their
    `Resource.…` values, and the method is looked up via *Reflection*.
    This means System.Reflection is part of the app startup path,
    which has overheads.
    (This overhead is also removed via `$(AndroidLinkResources)`.)


~~ Bind `@(AndroidRoesource)` values as properties  ~~

Replace the "bind resources as fields" approach with a new system
with significant differences:

 1. Android resource ids are bound as read-only *properties*, and

 2. The `Resource` class is placed into a *separate assembly*,
    `_Microsoft.Android.Resource.Designer.dll`.

The new `$(AndroidUseDesignerAssembly)` MSBuild property controls
which Android resource approach is used; if True -- the default for
.NET 8 -- then `_Microsoft.Android.Resource.Designer.dll` will be
used.  If False, then the previous "bind resource ids as fields"
approach will be used.  This property is only valid for Library
projects; App projects must use the property-oriented approach.

This new approach takes advantage of [Reference Assemblies][1].
Reference Assemblies are designed to be replaced at runtime, and are
generally used to provide placeholder API's which can be swapped out
later.

Library projects will generate a Reference Assembly for
`_Microsoft.Android.Resource.Designer.dll` which contains read-only
properties for each `@(AndroidResource)` within the project and all
dependencies.  This is otherwise identical to the "fields" approach,
*except* that the namespace is predefined, its a new assembly, and
properties are used instead of fields, *as if* it contained:

	// _Microsoft.Android.Resource.Designer.dll for Library project
	[assembly: System.Runtime.CompilerServices.ReferenceAssemblyAttribute]
	namespace Microsoft.Android.Resource.Designer;
	public partial class Resource {
	  public partial class String {
	    public static int app_name => 0;
	  }
	  public partial class Styleable {
	    public static int[] MyLibraryWidget => nullptr;
	  }
	}

Also note that `_Microsoft.Android.Resource.Designer.dll` is produced
*with Mono.Cecil* as a pre-build action; no C# source is generated.
The Library assembly references the generated
`_Microsoft.Android.Resource.Designer.dll`.

The generated `_Microsoft.Android.Resource.Designer.dll` should
***NOT*** be shipped with NuGet packages.

App projects will generate the "real"
`_Microsoft.Android.Resource.Designer.dll`, also as a pre-build step,
and the "real" assembly will contain actual values for resource ids.
The App-built `_Microsoft.Android.Resource.Designer.dll` will also
have `[assembly:InternalsVisibleToAttribute]` to the App assembly:

	// _Microsoft.Android.Resource.Designer.dll for App project
	[assembly: System.Runtime.CompilerServices.InternalsVisibleToAttribute ("App…")]
	namespace Microsoft.Android.Resource.Designer;
	public partial class Resource {
	  public partial class String {
	    public static int app_name => 2130968576;
	  }
	  public partial class Styleable {
	    static int[] MyLibraryWidget = new[]{…};
	    public static int[] MyLibraryWidget => MyLibraryWidget;
	  }
	}

This approach has a number of benefits

 1. All the property declarations are in one place and are not
    duplicated (-ish… more on that later).
    As a result the size of the app will be reduced.

 2. Because we no longer need the `Resource.UpdateIdValues()` method,
    start up time will be reduced.

 3. The linker can now do its job and properly link out unused
    properties.  This further reduces application size.

 4. F# is now fully supported.  See also: dotnet/fsharp#12640.


~~ Styleable Arrays ~~

Styleable resources may be arrays; see e.g. d521ac02.  Via the power
of Cecil (and not using C# as an intermediate codegen), the binding
of styleable arrays in the "Bind `@(AndroidRoesource)` values as
properties" world order involves a static field containing the array
data, and a public property which returns the private field, which
have the same name:

	public partial class Resource {
	  public partial class Styleable {
	    static int[] MyLibraryWidget = new[]{…};
	    public static int[] MyLibraryWidget => MyLibraryWidget;
	  }
	}

CIL-wise, *yes*, the field and the property have the same name (?!),
but because properties actually have `get_` method prefix, there will
actually be a `MyLibraryWidget` field and a `get_MyLibraryWidget()`
method, so there are no name collisions.

*Note*: ***The styleable array is not copied***.  This means it is
global mutable data, i.e. one can do this:

	Microsoft.Android.Resource.Designer.Resource.Styleable.MyLibraryWidget[0] = 42;

***DO NOT DO THIS***.	It will introduce runtime errors.

The e.g. `Resource.Styleable.MyLibraryWidget` property must be an
`int[]` in order to maintain compatibility, e.g. these are often
passed to methods which take `int[]` as the parameter type.  We thus
cannot instead use e.g. `IEnumeragble<int>` as the property type.
Additionally, the array isn't copied for performance reasons.

We do not think that this will be a problem in practice, as the
previous "Bind `@(AndroidRoesource)` values as fields" strategy
*also* had mutable `int[]` fields, and suffers from the same
safety concerns, and the world hasn't ended…


~~ Source Compatibility ~~

In the "bind resource ids as fields" approach, the `Resource` class
was in the default namespace for the Library project, set via the
[`$(RootNamespace)`][2] MSBuild property.  In order to maintain
source compatibility, Library projects will have a generated
`__Microsoft.Android.Resource.Designer.cs` file which contains a new
`Resource` declaration which *inherits* from the `Resource` type in
`_Microsoft.Android.Resource.Designer.dll`:

	// Generated __Microsoft.Android.Resource.Designer.cs in Library projects
	namespace ExampleLib;
	public class Resource : Microsoft.Android.Resource.Designer.Resource {
	}

This allows existing code such as `ExampleLib.Resource.String.app_name`
to continue to compile.

App projects also expect a `Resource` class in `$(RootNamespace)`,
*and* expect the values to be `const`.  To support this, the generated
`_Microsoft.Android.Resource.Designer.dll` *actually* has two sets
of `Resource` types, one with properties, and an *`internal`*
`ResourceConstant` type:

	// _Microsoft.Android.Resource.Designer.dll for Library project
	[assembly: System.Runtime.CompilerServices.InternalsVisibleToAttribute ("App…")]
	namespace Microsoft.Android.Resource.Designer;
	internal partial class ResourceConstant {
	  public partial class String {
	    public const int app_name = 2130968576;
	  }
	}

	public partial class Resource {
	  public partial class String {
	    public static int app_name => ResourceConstant.String.app_name;
	  }
	}

App projects *also* have a generated
`__Microsoft.Android.Resource.Designer.cs`, which has a `Resource` type
which inherits from `ResourceConstant`.  This is why the App-built
`_Microsoft.Android.Resource.Designer.dll` needs
`[assembly: InternalsVisibleToAttribute]`:

	// Generated __Microsoft.Android.Resource.Designer.cs in App projects
	namespace App;
	public class Resource : Microsoft.Android.Resource.Designer.ResourceConstant {
	}

This allows existing App code to use `App.Resource.String.app_name`
in `case` statements.


~~ Binary Compatibility ~~

Binary compatibility is maintained via a new
`MonoDroid.Tuner.FixLegacyResourceDesignerStep` linker step.
`FixLegacyResourceDesignerStep` rewrites Library assemblies to replace
`Resource.…` field access with property access to
`Microsoft.Android.Resource.Designer.Resource.…` in
`_Microsoft.Android.Resource.Designer.dll`.  Much of this code
overlaps with the existing logic of `$(AndroidLinkResources)`, and
allows existing Library assemblies to participate in the property-
oriented system.


~~ Internals ~~

The new build system introduces a number of new Tasks and Targets to
bring this all together.  It also unify's some code between the field-
oriented and property-oriented approaches which would otherwise be
duplicated.  The field-oriented system will be maintained for now for
backward compatibility, however the property-oriented system will be
enabled by default for .net 8.

The property-oriented system is mostly contained in
`Xamarin.Android.Resource.Designer.targets`.  The entry point for this
set of targets is `_BuildResourceDesigner`, which will only be run if
the `$(AndroidUseDesignerAssembly)` MSBuild property is `True`, as it
will be for .NET 8+.

New tasks are as follows.

  - `<GenerateRtxt/>` is responsible for scanning the resource
    directory and generating an `aapt2`-compatible `R.txt` file.
    This will be used by `<GenerateResourceDesignerAssembly/>`.

  - `<GenerateResourceCaseMap/>` is responsible for generating a
    `casemap.txt` file which will map the all lower case android
    resources to the casing required for the C# code.  Android requires
    ALL resources be lower case, but our system allows the user to
    define the case using any system then want.  This task handles
    generating this mapping between what the android system needs and
    what the user is expecting.  Its output is used by the
    `<GenerateResourceDesignerAssembly/>` task when generating the IL
    in `_Microsoft.Android.Resource.Designer.dll`.
    It is also used by the old system to generate the same file.

  - `<GenerateResourceDesignerIntermediateClass/>` is responsible for
    generating the `__Microsoft.Android.Resource.Designer.cs` file in
    `$(IntermediateOutputPath)`.


  - `<GenerateResourceDesignerAssembly/>` is the key to the whole
    property-oriented approach.  This task will read the `R.xt` file
    and generate a `_Microsoft.Android.Resource.Designer.dll` assembly
    in `$(IntermediateOutputPath)`.  This task is called in two places.
    The first is in `_GenerateResourceDesignerAssembly`, this is called
    as part of the build which happens just before `CoreCompile` and
    only for design time builds.
    It is also called in `_UpdateAndroidResgen` which happens as part
    of the build and runs just after `aapt2` is called.  This ensures
    we always use the most up to date version of `R.txt` to generate
    the new assembly.

Because we are using the `R.txt` file to drive the generation of the
new assembly, we needed some way for that to work when `aapt2` was
not being run.  This usually happens on a first time design time build.
The field-oriented approach has a `<GenerateResourceDesigner/>` task
which is responsible for both scanning the resources and generating a
design time `Resource.designer.cs` file.  While we could have
duplicated the code it made more sense to split out the resource
scanner into its own class.  We now have a new `<GenerateRtxt/>` task
which is responsible for scanning the resources and generating an
`R.txt` file.  This is only used when we are not doing a full build
with `aapt2`.  This new task lets us generate the needed `R.txt` which
can then be used by both the old and new system to generate their
respective outputs.

As part of this we have two other classes: `RtxtReader` and
`RtxtWriter`.  The `RtxtReader` unify's the code which was used to read
the values of the `R.txt` into one class which can be used by both
approaches.  The `RtxtWriter` is responsible for writing the `R.txt`
file for design time builds.  Again it will be used by both the old
and new system.

The `_AddResourceDesignerFiles` target is responsible for ensuring that
the new assembly and `__Microsoft.Android.Resource.Designer.cs` get
added to the correct item groups.  These are `@(ReferencePath)` for the
assembly and `@(Compile)` for the source file.  In the case of F# the
`__Microsoft.Android.Resource.Designer.fs` file which gets generated
has to be added to the `@(CompileBefore)` ItemGroup, this is so that
the types are resolved in the correct order.

To ensure that the new assembly is added to the final application we
have to introduce the `_AddResourceDesignerToPublishFiles` target.
This target makes sure that the new assembly is added to the
`@(ResolvedFileToPublish)` ItemGroup.  It also adds the require
MetaData items such as `%(IsTrimmable)` and `%(PostprocessAssembly)`
which are required to get the assembly linked correctly.


~~ Results ~~

Results are most visible when lots of Android Resources are used.
For a [Sample app][3] app which uses lots of resources, we see the
following improvements to the **ActivityTaskManager: Displayed** time:

| Before (ms) |  After (ms) |     Δ (%) | Notes                                |
| ----------: | ----------: | --------: | ------------------------------------ |
|     340.500 |     313.250 |  -8.00% ✓ | defaults; 64-bit build               |
|     341.950 |     316.200 |  -7.53% ✓ | defaults; profiled AOT; 64-bit build |
|     345.950 |     324.600 |  -6.17% ✓ | defaults; 32-bit build               |
|     341.000 |     323.050 |  -5.26% ✓ | defaults; profiled AOT; 32-bit build |

[0]: https://learn.microsoft.com/en-us/xamarin/android/deploy-test/building-apps/build-properties#androidlinkresources
[1]: https://learn.microsoft.com/en-us/dotnet/standard/assembly/reference-assemblies
[2]: https://learn.microsoft.com/en-us/visualstudio/msbuild/common-msbuild-project-properties?view=vs-2022
[3]: https://github.com/dellis1972/DotNetAndroidTest

[dellis1972]

@jonpryor

Commit message was ok, just a typo in

~~ Bind @(AndroidRoesource) values as fields ~~

but otherwise ok 👍

[TimLariviere]

Awesome job @dellis1972 !
Really looking forward to it and getting rid of FSharp.Android.Resource.

Is it planned to be released with net8, or will it be released in a service release of net7 as well?

[dellis1972]

This will be a net8 feature.