Future of this project and discussion about `objc2`

[madsmtm]

Hey @waywardmonkeys, @jrmuizel, @mrobinson and @wusyong (and CC @tmandry and @simlay).

Over the past few years, I've been working on replacements for the crates in this repo in the objc2 project, see #513 and #628 for some previous discussion. To rehash the benefits of the objc2-* crates:

• Code generation from headers, which means that basically every upstream API is available (only a few explicitly skipped APIs aren't), and this will remain true even as Apple releases new SDK versions over time (since we only need to rerun the generator).
• Automatic memory management, all objects are wrapped by the code generator in either objc2::rc::Retained or objc2_core_foundation::CFRetained, which will release the object on Drop. The generator knows about the "create rule" and about opt-outs like CF_RETURNS_NOT_RETAINED etc. (so it will do this correctly with a much higher probability than if a human did it).
• Type-safety, especially compared to cocoa and cocoa-foundation where everything is just an extension trait on the id pointer.

I'm motivated in bringing these benefits to the entirety of the Apple+Rust ecosystem, as I think the soundness, correctness and ergonomics improvements here are quite significant. To that end, I've been slowly migrating major Rust projects, see madsmtm/objc2#174.

A recent milestone is that I've released objc2-core-foundation v0.3.1, which I believe now covers everything that core-foundation does! I've also assessed every open issue and pull-request in this repo, and made sure that they're either resolved or tracked in the objc2 repo, see madsmtm/objc2#719. So at this point, I'd like to start discussing the future of this project, and how we can migrate Servo and other users to the objc2-* crates (assuming you agree that's a goal, if not then I'd also like to discuss that!)

My ideal scenario would be that the crates in this repo were marked #![deprecated] with re-exports of the objc2-* variants, both to give users an easier migration path, and because in a way it'd "bless" the objc2 crates, but I can see other solutions, including any of the following variations:

• Only do this for cocoa and cocoa-foundation (these are where objc2-* would be the most valuable).
• Soft-deprecate the crates in the README only, but without actually marking it #![deprecated].
• Publish an advisory to the RUSTSEC advisory database (my own preference would be not to, at least not yet).

A few drawbacks to consider:

• It is a fair amount of churn. I don't believe it to be unnecessary churn though, there really are a lot of bugs out there in Apple+Rust-related code, and a lot of them could be alleviated with the objc2-* crates.
• objc2-* crates are a bit less stable than core-foundation-rs crates. I do tend to be conservative of breaking changes, and to plan ahead to allow spacing them out as much as possible.
• It is my understanding that Firefox/Mozilla vets/audits every dependency, and that the objc2-* crates haven't been audited, so this might hurt them until that happens? CC @ErichDonGubler and @jimblandy maybe?
• This could be seen as a supply-chain attack. Don't know how to alleviate that, other than having people review my code ;)
• Bus factor, objc2 has fewer maintainers. I'd love to get others on board though, just say so if you want to! Or join the Matrix workspace to start with.

Crate status / comparison

Note that I'm not entirely done transitioning things here, see the below table for the current status (which is probably slightly incorrect, there's bound to be something I've missed):

Servo crate	objc2 crate	Comparison
cocoa	objc2-app-kit	Fully superseded ✅
cocoa-foundation	objc2-foundation	Fully superseded ✅
core-foundation-sys	objc2-core-foundation	Fully superseded ✅
core-foundation	objc2-core-foundation	Fully superseded ✅
core-graphics-types	objc2-core-foundation	Fully superseded ✅
core-graphics	objc2-core-graphics	Missing CGDirectDisplayID wrapper and most functions are still unsafe ⚠️
core-text	objc2-core-text	Still a fair bit of work to do to make it as nice ❗️
io-surface	objc2-io-surface	Fully superseded ✅

But I do intend to resolve this, so I still feel it's valuable to start discussing migration now.

How objc2-core-foundation differs from core-foundation

Just to get everyone up to speed, here's a quick overview of the differences.

The memory management strategy is as follows:

core-foundation	objc2-core-foundation
CFString	CFRetained<CFString>
&CFString	&CFRetained<CFString>
CFStringRef	*mut CFString
__CFString	CFString
Not really possible	&CFString

That is, objc2-core-foundation only has a single type that represents CFString, and memory management (CFRetain/CFRelease) is instead handled by a single generic wrapper type. This also meant that while I did initially consider building upon core-foundation instead of creating a new crate, it wasn't really an option in the end, as the internals are just too different.

objc2-core-foundation also avoids the *-sys split, instead preferring to expose everything in the same crate (again, to avoid a proliferation of types). A few of my guiding principles API-wise:

• Expose all internals such that users never need to write extern "C" { ... } themselves.
• Make everything as nice to use as possible when coming from Rust. Functions are exposed as associated functions/methods, conversions to/from common Rust types are provided.
• Be performance-minded when creating wrappers. For example, I wouldn't add a CFURL -> String conversion, since that requires at least two CoreFoundation functions to be called. Instead, CFURL -> CFString is provided, and so is CFString -> String.

[waywardmonkeys]

I'll keep this relatively short.

I've been very happy with switching other crates over to using the objc2 family of crates and look forward to continuing to do more of that.

They're far more complete and have been free of errors. There are still a lot of API gaps in this repo and binding them by hand is tedious and error prone. There was at least one thing where something in core-graphics maps to the wrong enumeration values.

Another thing is that since the coverage of the Apple APIs are more comprehensive. I am porting some code that used CGWindowListCreateImage. In the objc2 crates, it is marked as deprecated (as it is by Apple), and the replacement, ScreenCaptureKit, already has bindings in place: objc2-screen-capture-kit.

Another advantage is that these bindings are closer to the APIs from Apple and more consistently handled. An example of this might be cocoa::quartz_core::transaction vs objc2_quartz_core::CATransaction.

[waywardmonkeys]

One other comment ...

I know that this sort of transition will cause some pain. But I do think that:

• It will be relatively short-lived.
• It will result in some changes to the objc2 family of crates, probably causing breaking changes, but doing that to crates with this amount of auto-generation will be far easier.
• Even with fewer maintainers, objc2 crates are a significant step forward and provide better, more accurate, more comprehensive coverage.
• If you're looking at "Where do we want to be in 2 years?", I think the answer is pretty clear about what path forward will get us there.

[jdm]

I'm also in favour of transitioning Servo onto the objc2. Maintaining all of these hand-written bindings is a pain.

[mrobinson]

I also agree that migrating all of Servo to objc2 seems like the best path forward.

While the helpers provided by core-text and similar crates can simplify Servo code, they also make it harder to understand what exactly a particular bit of code does. For instance take the call to core_text::font_collection::get_family_names() in components/fonts/platform/macos/font_list.rs. It would be hard to understand that this function is simply calling CTFontManagerCopyAvailableFontFamilyNames() unless you went and looked at the code. The fact that this static method on CTFontManager is exposed in the font_collection module and doesn't have any documentation means that it is difficult to know where to find the Apple documentation which describes what this function actually does, which is found at https://developer.apple.com/documentation/coretext/ctfontmanagercopyavailablefontfamilynames().

[madsmtm]

While the helpers provided by core-text and similar crates can simplify Servo code, they also make it harder to understand what exactly a particular bit of code does.

FYI, I recently decided to add a similar function -> method renaming scheme, see madsmtm/objc2#736 (primary motivation was to follow suit with Swift). The renaming isn't done for CTFontManager in particular, because there's no class to attach it to. But e.g. CTFontDescriptorCreateCopyWithFeature is mapped as CTFontDescriptor::copy_with_feature.

The issue is somewhat alleviated in objc2-* by the fact that we generally don't do anything else than call the underlying method. I am also planning to have links to Apple's documentation in madsmtm/objc2#309, but it's a bit gnarly around the edge-cases.

We also already emit a #[doc(alias = ...)], which allows searching for the item, and allows rust-analyzer to suggest the renamed method if you wrote e.g. CTFontDescriptor::CreateCopyWithFeature.

But I'd be interested to know if you think it'd be valuable to show the actual name of the method in the documentation? (Alternatively, we could work together with rustdoc to perhaps expose aliases in the documentation itself?)

[mrobinson]

@madsmtm These all seem like really reasonable design decisions and seem clearer and more consistent that the previous status quo with crates like core-text.

[pronebird]

I have been transitioning a lot of code to objc2 as well and I think it's the way to go. Due to volume of changes in Apple frameworks, manually maintaining separate Rust bindings for each framework is just impossible. Automatic generation is the only way to stay up to date and I believe Apple do the same for Swift. I'd be really happy to see Mozilla join the effort to ensure the longevity of objc2.

[jdm]

@pronebird Mozilla is not involved with this project in any official capacity, as far as I'm aware.

[ErichDonGubler]

Mozilla's WebGPU team plans on using objc2-metal eventually (see gfx-rs/wgpu#5641), but not anytime soon. We're swamped with trying to ship during this year, and likely much of next year. I can't speak for other teams at Mozilla.

The only real obstacle is vetting dependencies according to cargo-vet's safe-to-deploy criteria. It appears to us a hefty ask to audit such a large codebase. We suspect that objc2* will be significantly easier to audit than an equivalent volume of handwritten code, but we're still uncertain what a correct estimate of effort would be.