humancode.us

All about mergeable libraries

January 2, 2024

Mergeable Libraries is a technology introduced in WWDC23 that allows you to “absorb” and reexport dynamic libraries into a higher-level dynamic library, or directly into your app’s binary.

To see the performance implications of using mergeable libraries in your app, see this post.

How mergeable libraries work

The mergeable libraries feature allows you to copy the contents of dependent1 dynamic libraries one level up while maintaining the illusion in code that the original dependent libraries still existed where they were, when using API such as dlopen and Bundle.

Because the contents of dependent libraries are copied into the binary that links them, unresolved symbols in that binary can be statically resolved against the library’s contents which are now stored within the same binary. This is how mergeable libraries improve load times: it allows the linker to turn load-time resolution of symbols into link-time resolution, giving you dynamic-library semantics with near-static-library performance.

Mergeable libraries only affect Release builds by default

By default, having mergeable libraries does not change the behavior of Debug builds. You still have to embed all the individual frameworks that make up the mergeable libraries into your app bundle, because you still need them at Debug time and for resource lookups. During Debug builds, any intermediate libraries that merge other dependent libraries will reexport2 those dependent libraries instead.

Think of mergeable libraries as a release-time optimization that changes the way library code is stored, but not their semantics.

Libraries have to be built as mergeable

A dynamic library or framework can make itself mergeable by setting MERGEABLE_LIBRARY = YES at build time, or setting the -make_mergeable ld option. Making a library mergeable doesn’t change the way it behaves; it merely generates metadata that allow future targets that link it to merge it if needed.

A binary target can merge its dependent dynamic libraries by setting MERGED_BINARY_TYPE to either manual or automatic. Setting this to manual will merge any dependent libraries that have been built with MERGEABLE_LIBRARY = YES. Setting it to automatic will force-merge any dependent libraries whose targets are found inside the same Xcode project, whether or not they have MERGEABLE_LIBRARY set to YES. You can manually manage this behavior by using the -merge-l or -merge-framework ld options.

Merging only occurs for mergeable libraries that are explicitly linked by a target. For instance, an app that links a dependent framework that itself links another framework will only merge the first one. To merge all dependent frameworks, your app must explicitly link them all.

Merged libraries are stubbed out in an app bundle

When you build an app target with mergeable libraries (say, an iOS app with embedded frameworks), Xcode replaces the merged libraries with small, stub binaries3. Those binaries are unused, as the entire contents of the originals would have been copied to and used directly within the app binary.

This means once again that you should continue to embed dependent frameworks in your app bundles. That way, your resources will remain where they are, resource lookups with Bundle will continue to work, but your library sizes will be reduced to near-zero as your framework contents are copied into your app binary instead.

Use intermediate libraries to control app bundle size

One of the main reasons to use embedded dynamic libraries is the fact that code is often shared between two or more binaries within your app bundle: the app itself, and one or more extensions. If you build all your shared frameworks as mergeable, each linking binary will have its own copy of the shared frameworks, increasing the size of your app.

This may be OK if your frameworks are small, but sometimes it pays to create intermediate frameworks that are not mergeable, but which themselves merge further dependent libraries. Your app will still have to load the intermediate framework at load time, but the load time can be far less than loading individual dependent frameworks. Intermediate frameworks give you most of the benefits of mergeable libraries without the size penalty.

To create an intermediate framework, create a framework target and set its MERGED_BINARY_TYPE to manual or automatic. Then, have that target link all of the dependent libraries you want to merge together. Then, have your app and extensions link that intermediate target instead of the dependent libraries. The intermediate target will not be merged into the app.

Note that even though your app links against the intermediate framework, you still have to embed all of the dependent libraries into your app bundle (and also the intermediate framework) so that the app can run in the Debug configuration where your intermediate framework will act as an umbrella framework, and so that your resources remain discoverable at runtime.

Recommendations

Here are my recommendations for using mergeable libraries.

Static libraries are still the fastest choice. If you want the best performance, your best bet is still to use static libraries. No dynamic library is going to beat the load-time performance of a static library.

Mark dynamic libraries as mergeable by default. Unless you’re creating intermediate libraries that intentionally aren’t mergeable, make your dynamic libraries mergeable by setting MERGEABLE_LIBRARY = YES. This is especially important if you distribute prebuilt binaries as xcframeworks, or have a build system that caches prebuilt binaries. Marking your libraries as mergeable adds metadata needed if the library’s user wants to merge it, but adds no penalty to those who don’t.

Enable merging strategically. Only specific targets should set MERGED_BINARY_TYPE, namely executables and intermediate frameworks.

Use manual merging only. Automatic merging is convenient for smaller projects, but doesn’t scale. I recommend using manual merging up front, so you can control which libraries get merged by building them as mergeable as needed.

Continue to embed all frameworks. Merging frameworks does not change their semantics. You must continue to embed all frameworks in your app bundle to build for Debug, and for resource lookup to work. If you create intermediate frameworks, you must embed them too.

Use intermediate frameworks to control code size. As described above, strategically create intermediate merged frameworks to prevent library code from being copied into multiple executables.

Create new Xcode-based frameworks as dynamic by default. Xcode-based frameworks tend to work better when targets are built as dynamic libraries: their semantics are more intuitive, link order matters less, and resource lookup is well-supported. With mergeable libraries, you can enjoy these conveniences while also reaping the benefits of near-static-linking performance.

Do not lazy-load frameworks or perform manual symbol resolution. Link all required frameworks up front. Lazy-loading frameworks and manually resolving symbols excludes you from the benefits of mergeable libraries.

  1. I use the term “dependent libraries” to refer to a library that is linked by another binary, i.e. the child nodes in a link dependency tree. I know this term can be confusing, but this is the terminology Apple uses in their docs. 

  2. Reexporting dependent libraries is an old technique used to create Umbrella frameworks. Cocoa.framework on macOS, for instance, reexports Foundation.framework, AppKit.framework, and CoreData.framework. Your app only needs to link Cocoa to automatically link all three dependent frameworks. 

  3. The documentation mentions that such binaries are removed, but in my experiments, a very small, mostly empty stub binary is left in its place. They could well be removed in the future, so don’t rely on their presence in your build products.