humancode.us

All about Item Providers

July 8, 2023

The NSItemProvider class in Foundation is a powerful abstraction for making data available across processes that are otherwise isolated from one another. I hope this post can be a one-stop reference for developers who want a solid understanding how item providers work, and how to use the API in a modern way.

This section provides a history of NSItemProvider and its API surfaces. Feel free to skip it if you’re not interested.

NSItemProvider was introduced in iOS 8.0 to support passing data between an application and extensions. The interface introduced at that time relied heavily on runtime Objective-C introspection, and every extension point specified its own peculiar way in which apps had to provide and receive data.

When Drag and Drop for iOS was introduced in iOS 11.0, a new set of API was introduced on the class to meet new requirements: Swift made it impractical to use Objective-C introspection, and Drag and Drop needed a more consistent and generalized abstraction, since the source and destination processes could be any app or extension.

The introduction of UniformTypeIdentifiers in iOS 16.0 added a further extension to the class which uses UTType instead of String-typed identifiers for data type specifications. Methods to bridge between the Swift Transferable protocol and NSItemProvider were also introduced.

The existence of many historical layers in the NSItemProvider API makes it confusing for the contemporary developer to understand exactly what they need to use in their code. It is my hope that this post can help steer developers toward the best set of API to use in their apps.

NSItemProvider is a data promise

An NSItemProvider is a promise to provide data. A provider constructs an NSItemProvider object and registers available data. It then passes the item provider to some system API which will make it available to some consumer which will load the data. The provider needs not provide any data up front—data is requested only when a consumer attempts to load it.

A provider may promise multiple representations for an item

Each NSItemProvider holds a promise for a single conceptual item, such as an image, document, or string. However, the item may be available in more than one representation. An image, for instance, may be available as a png, tiff, jpg, or bmp; but every representation encodes the same image.

Representations are identified by content types, represented by UTType objects (e.g. UTType.jpeg), or equivalent strings known as Uniform Type Identifiers (e.g "public.jpeg"). Content types may have conformance relationships. For example, the UTType.jpeg conforms to UTType.image. These conformance relationships allow a consumer to request a more general type, which may be fulfilled by a more specific type by the provider. Representations registered with NSItemProvider should conform to UTType.data or UTType.directory.

New content types are declared in the Info.plist of an application. The operating system adds new definitions to its database when the application is launched for the first time.

Representations are registered in order of fidelity

Representations should be registered in order of fidelity: the first registered representation should have the highest fidelity—usually whatever file format the provider uses to save its data to nonvolatile storage—followed by representations of increasingly lower fidelity. Using our image example, a provider would register its native file type (e.g. psd for Photoshop), followed by lossless formats (png), and finally lossy ones (jpeg).

Consequently, consumers should examine the ordered list returned by the registeredContentTypes property of incoming NSItemProviders, and load the first registered representation that the app supports to get the highest possible fidelity.

Content types specify binary encoding of data

When registering representations, choose UTTypes or Uniform Type Identifier strings that precisely define the binary format of the promised data. For example, choose UTType.jpeg to identify a JPEG-encoded image, or UTType.utf8PlainText for UTF8-encoded text. Avoid non-specific types such as UTType.image or UTType.plainText, which only specify what the data contains, but not how the data should be parsed.

Consumers examine the registered content types of incoming NSItemProvider objects before loading data. Registering precise content types helps consumers select the best representations they can consume.

Consumers should use conformance checks to look for eligible representations. API such as registeredContentTypes(conformingTo contentType:) help to streamline this task.

Register UTType.url representations only to provide URLs to network resources such as web pages. Registering UTType.url representations of URLs to local files is almost always an error; you should use a content type matching the content of the local file instead.

Some older extension points require host apps to provide URLs to local files, in which case you should continue to do so. But avoid doing this in other cases.

MacOS programmers may recall that AppKit API traditionally require apps to share NSURL objects across processes using the Pasteboard or drag an drop as a way to share files as open-in-place. This convention is not used with NSItemProvider; always register a content type that reflects the content of the file.

Data can be provided three ways

A provider can fulfill its promise to provide data for each of its representations in three different ways:

  1. By handing over a Data (NSData) object
  2. By handing over a URL pointing to a file to be copied into the consumer’s data container
  3. By handing over a URL pointing to a file to be opened-in-place by the consumer

Providing a Data object works pretty simply: the system makes a copy of the bytes in the object and makes it available to the consumer.

Providing a file to be copied causes the the system to copy the file and make that copy available to the consumer.

Providing an opened-in-place file causes the file to be shared by both the provider and consumer.

Not all files can be opened-in-place. On iOS, only files stored in iCloud Drive or a File Provider container can be shared this way. On macOS, most of the user’s files can be opened-in-place.

After sharing a file for opening in-place, both ends of the transaction must cooperate when modifying the file, and anticipate that changes or deletion may be performed by some process other than itself. Practically, this means only accessing the file through NSFileCoordinator, and using an NSFilePresenter to respond to changes.

Providers may attempt to offer any file as openable-in-place; the system will determine if the file is indeed eligible for open-in-place. If the file is not eligible, NSItemProvider automatically converts the operation to a file copy.

Data can be consumed three ways

Similarly, a consumer may load data for each representation in three ways: as a Data object, as a copied file, and as a file opened-in-place.

When a provider provides data in the same way that the consumer requests it, the steps taken by the system to hand the data from provider to consumer is simple: the Data object is copied, a file copy is created, or the shared file URL is passed to the consumer.

If there is a mismatch, NSItemProvider helps by converting the way the provider offers the data so that the consumer can receive the data in the way it requested.

Consumers may attempt to load any representation as open-in-place; the system will determine if the representation’s file is indeed eligible for open-in-place. The consumer’s completion block will receive a parameter that tells it whether open-in-place succeeded, or if it received a copy of the data instead.

NSItemProvider bridges data between provider and consumer

A valuable service provided by NSItemProvider is its ability to bridge load requests when providers and consumers decide on different ways to provide and consume data.

For instance, a provider could provide data for a representation as a Data object. If a consumer happens to request the data as a file copy, NSItemProvider will write a temporary copy of the Data object to nonvolatile storage, and provide that file’s URL to the consumer.

The following matrix shows what NSItemProvider does to bridge data between providers and consumers.

When creating copies of files, NSItemProvider does its best to preserve file names. A provider may set the suggestedName property to override this behavior, or to provide a name for a Data object if it were to be written to nonvolatile storage. As a fallback, NSItemProvider uses the localized description of a representation’s content type as a file name. Additionally, NSItemProvider ensures that the appropriate file extension is present in the file name; a file extension will be added when necessary.

  Consumer loads Data Consumer loads file copy Consumer loads file to open in place
Provider registers Data Consumer receives Data Data written to nonvolatile storage, consumer receives URL Data written to nonvolatile storage, consumer receives URL (open-in-place is false)
Provider registers file to copy Consumer receives Data containing contents of file File is copied, consumer receives URL File is copied, consumer receives URL (open-in-place is false)
Provider registers file to open in place Consumer receives Data containing contents of file File is copied, consumer receives URL Original file URL is given to consumer (open-in-place is true)

Providers may provide directories instead of files

Providers may hand a URL pointing to a directory instead of a file to fulfill its promise. In such cases, a consumer loading a copy of the file will receive a copy of the entire directory tree. If a consumer loads the representation as Data, it will receive a Data containing a zip archive of the directory, with the directory as the only entry at the root of the archive. It’s not possible to share a directory as open-in-place; the consumer will always receive a copy.

Providing directories is appropriate for representations conforming to UTType.directory, such as UTType.rtfd. In other cases, it is advisable to provide data as files to maximize compatibility.

Providing and consuming data is inherently asynchronous

Providing and consuming data is asynchronous. After all, the data you requested may need to be retrieved over the network, or generated on demand.

When providing data, loader blocks will be called whenever a consumer requests data, on an arbitrary queue. When consuming data, the completion block is called on an arbitrary queue after the data has completely arrived. Design your apps to account for potentially long delays when loading data. Use the Progress objects returned by the loading functions to display progress, and to offer a way for the user to cancel a long-running operation. Some system UI, like Drag and Drop on iOS, automatically display modal alerts that show the user the progress of a long-running drag and drop operation.

Use modern NSItemProvider API

If you open the NSItemProvider.h header in Foundation, you will find that it is divided into two sections: a Binary interface containing a more modern API, and a Coercing interface containing the original API used in iOS 8.0. Do not use the coercing interface. I repeat: Do not use the coercing interface ❌ (I added that big red X emoji so you don’t miss it). The coercing interface remains a supported API because some extension points still use them, but they rely on Objective-C runtime introspection and they do not work in Swift. The coercing interface should be considered obsolete, and I will not cover it in this post.

If you target iOS 16.0 or later, the API you should use mostly lives in the NSItemProvider+UTType.h header in the UniformTypeIdentifiers framework. This category provides the most modern interface, and works well with Swift.

So which API should you use? Take a look at the header files for details, but here is a short list for you:

class NSItemProvider: NSObject {
    // Initialization
    
    init()
    
    convenience init(
        contentsOf fileURL: URL,
        contentType: UTType?,
        openInPlace: Bool = false,
        coordinated: Bool = false,
        visibility: NSItemProviderRepresentationVisibility = .all
    )
    
    convenience init(object: NSItemProviderWriting)

    // Metadata query
    
    var registeredContentTypes: [UTType] { get }
    var registeredContentTypesForOpenInPlace: [UTType] { get }
    func registeredContentTypes(conformingTo contentType: UTType) -> [UTType]
    var suggestedName: String? { get set }
    func canLoadObject(ofClass aClass: NSItemProviderReading.Type) -> Bool

    // Providing data
    
    func registerDataRepresentation(
        for contentType: UTType,
        visibility: NSItemProviderRepresentationVisibility = .all,
        loadHandler: @escaping @Sendable (@escaping (Data?, Error?) -> Void) -> Progress?
    )

    func registerFileRepresentation(
        for contentType: UTType,
        visibility: NSItemProviderRepresentationVisibility = .all,
        openInPlace: Bool = false,
        loadHandler: @escaping @Sendable (@escaping (URL?, Bool, Error?) -> Void) -> Progress?
    )

    func registerObject(
        _ object: NSItemProviderWriting,
        visibility: NSItemProviderRepresentationVisibility
    )

    func registerObject(
        ofClass aClass: NSItemProviderWriting.Type,
        visibility: NSItemProviderRepresentationVisibility,
        loadHandler: @escaping @Sendable (@escaping (NSItemProviderWriting?, Error?) -> Void) -> Progress?
    )

    func register<T>(_ transferable: @autoclosure @escaping @Sendable () -> T) where T : Transferable

    // Consuming data
    
    func loadDataRepresentation(
        for contentType: UTType,
        completionHandler: @escaping @Sendable (Data?, Error?) -> Void
    ) -> Progress

    func loadFileRepresentation(
        for contentType: UTType,
        openInPlace: Bool = false,
        completionHandler: @escaping @Sendable (URL?, Bool, Error?) -> Void
    ) -> Progress
    
    func loadObject(
        ofClass aClass: NSItemProviderReading.Type,
        completionHandler: @escaping @Sendable (NSItemProviderReading?, Error?) -> Void
    ) -> Progress

    func loadTransferable<T>(
        type transferableType: T.Type,
        completionHandler: @escaping @Sendable (Result<T, Error>) -> Void
    ) -> Progress where T : Transferable
}
class NSItemProvider {
    // Initialization
    
    init()

    convenience init?(contentsOf fileURL: URL!)

    convenience init(object: NSItemProviderWriting)

    // Metadata query
    
    var suggestedName: String? { get set }
    func canLoadObject(ofClass aClass: NSItemProviderReading.Type) -> Bool
    func hasItemConformingToTypeIdentifier(_ typeIdentifier: String) -> Bool
    func hasRepresentationConforming(
        toTypeIdentifier typeIdentifier: String,
        fileOptions: NSItemProviderFileOptions = []
    ) -> Bool
    var registeredTypeIdentifiers: [String] { get }
    func registeredTypeIdentifiers(fileOptions: NSItemProviderFileOptions = []) -> [String]

    // Providing data

    func registerDataRepresentation(
        forTypeIdentifier typeIdentifier: String,
        visibility: NSItemProviderRepresentationVisibility,
        loadHandler: @escaping @Sendable (@escaping (Data?, Error?) -> Void) -> Progress?
    )

    func registerFileRepresentation(
        forTypeIdentifier typeIdentifier: String,
        fileOptions: NSItemProviderFileOptions = [],
        visibility: NSItemProviderRepresentationVisibility,
        loadHandler: @escaping @Sendable (@escaping (URL?, Bool, Error?) -> Void) -> Progress?
    )

    func registerObject(
        _ object: NSItemProviderWriting,
        visibility: NSItemProviderRepresentationVisibility
    )
    
    func registerObject(
        ofClass aClass: NSItemProviderWriting.Type,
        visibility: NSItemProviderRepresentationVisibility,
        loadHandler: @escaping @Sendable (@escaping (NSItemProviderWriting?, Error?) -> Void) -> Progress?
    )
    
    // Consuming data
    
    func loadDataRepresentation(
        forTypeIdentifier typeIdentifier: String,
        completionHandler: @escaping @Sendable (Data?, Error?) -> Void
    ) -> Progress

    func loadFileRepresentation(
        forTypeIdentifier typeIdentifier: String,
        completionHandler: @escaping @Sendable (URL?, Error?) -> Void
    ) -> Progress

    func loadInPlaceFileRepresentation(
        forTypeIdentifier typeIdentifier: String,
        completionHandler: @escaping @Sendable (URL?, Bool, Error?) -> Void
    ) -> Progress

    func loadObject(
        ofClass aClass: NSItemProviderReading.Type,
        completionHandler: @escaping @Sendable (NSItemProviderReading?, Error?) -> Void
    ) -> Progress
}

Example walkthrough

The following section walks you through creating an NSItemProvider, registering a representation, and loading it.

A provider creates an NSItemProvider and registers representations

Let’s begin our exploration by creating an NSItemProvider that provides data for an image. In our first example, the provider creates an item provider that provides a PNG representation from a file in nonvolatile storage, which we want NSItemProvider to provide to the consumer as openable-in-place. We assume that the provider already has the png file saved in an iCloud Drive directory, so our code looks like this:

let itemProvider = NSItemProvider() // 1

// iOS 16.0 or later
itemProvider.registerFileRepresentation(for: .png, openInPlace: true) { completion in // 2
    // Loader block
    completion(pngFileURL, true, nil) // 3
    return nil // 4
}

// iOS 11.0 or later
itemProvider.registerFileRepresentation(forTypeIdentifier: kUTTypePNG as String, fileOptions: .openInPlace, visibility: .all) { completion in // 2
    // Loader block
    completion(pngFileURL, true, nil) // 3
    return nil // 4
}

Line 1 constructs a new NSItemProvider with no registered representations.

Line 2 registers a data representation for the .png content type. As part of the registration, the provider passes in a block which will be called only when this representation is loaded by a consumer. The visibility parameter for the iOS 11 API will be covered later in this post.

Line 3 fulfills the promise by calling the completion block, passing in the requested png file’s URL. The second parameter tells NSItemProvider whether it needs to use file coordination to access the file. We pass in true to make sure the provider does not make changes to the file while it might potentially be copied to the consumer. The third parameters allows the loader block to return an error.

Line 4 The loader block may return an optional Progress object which will report progress on the data loading (e.g. if the provider needs time to download or generate the representation on the fly), and respond to requests to cancel loading the data. In this case, it returns nil, which will cause NSItemProvider to synthesize a Progress object for us that automatically goes from 0 to 100% when the completion block is called.

In addition, we want to register a jpeg representation which creates its data on demand.

// iOS 16.0 or later
itemProvider.registerDataRepresentation(for: .jpeg) { completion in
    let progress = createJpegDataFromPngFileAsync(pngURL) { // 5
        jpegData, error in
        completion(jpegData, error)
    }
    return progress // 6
}

// iOS 11.0 or later
itemProvider.registerDataRepresentation(forTypeIdentifier: kUTTypeJPEG as String, visibility: .all) { completion in
    let progress = createJpegDataFromPngFileAsync(pngURL) { // 5
        jpegData, error in
        completion(jpegData, error)
    }
    return progress // 6
}

Line 5 reencodes the png file into a jpeg on demand and asynchronously, and hands the resulting Data object off (or an error) when the job is done, to fulfill this promise. This block will only be called if a consumer requests the jpeg representation.

Line 6 returns a Progress object that tracks the conversion process and allows the loader to cancel the operation if the user thinks it takes too long. The code in createJpegDataFromPngFileAsync is responsible for updating the progress amount and periodically checking for cancellation.

A consumer loads representations

This example shows a receiver loading the png representation as a file opened-in-place.

// iOS 16.0 or later
if let repr = itemProvider.registeredContentTypes(conformingTo: .png).first { // 1
    let progress = itemProvider.loadFileRepresentation(for: repr) { URL, openedInPlace, error in // 2
        if let URL { // 3
            if openedInPlace { // 4
                // File was successfully opened in place. Save URL.
            } else {
                let fm = FileManager.default
                do {
                    // We received a copy of the file. Copy it to a safe
                    // location, because it will be deleted when we return
                    // from this block.
                    try fm.copyItem(at: URL, to: storageURL)  // 5
                } catch {
                    // ...
                }
            }
        }
    }
}

// iOS 11.0 or later
if let repr = itemProvider.registeredTypeIdentifiers.filter(
    { UTTypeConformsTo($0 as CFString, kUTTypePNG) }).first { // 1
    let progress = itemProvider.loadInPlaceFileRepresentation(forTypeIdentifier: repr) { URL, openedInPlace, error in // 2
        if let URL { // 3
            if openedInPlace { // 4
                // File was successfully opened in place. Save URL.
            } else {
                let fm = FileManager.default
                do {
                    // We received a copy of the file. Copy it to a safe
                    // location, because it will be deleted when we return
                    // from this block.
                    try fm.copyItem(at: URL, to: storageURL) // 5
                } catch {
                    // ...
                }
            }
        }
    }
}

Line 1 attempts to finds the first representation that conforms to something the consumer can handle, in this case, png.

Line 2 loads the representation . The consumer passes in a completion block that will be called when the file is loaded, or an error is returned. A Progress object is returned immediately, which the consumer can use to track or cancel the loading progress.

Line 3 runs when the loading has finished. If URL is non-nil, the load has succeeded. Otherwise, an error has occurred.

Line 4 tests whether the file was successfully opened in place. If so, the consumer saves the URL (see note below).

Line 5 runs when the consumer did not successfully open the file in place. In this case, the URL points to a temporary file (containing a copy of the data) that NSItemProvider has made available for the duration of the block. The consumer must copy this file into a safe location, because the temporary file will be deleted.

When NSItemProvider makes a file copy available to the consumer, the URL passed into the completion block is valid only during the lifetime of the block. The file will be deleted immediately after the control returns from the block.

Saving an open-in-place URL involves creating bookmarks that capture the security scope (data which allows a process to regain access to a file outside its container). Here’s how to do it:

// Save URL as bookmark
let bookmarkData = URL.bookmarkData(options: [.withSecurityScope])
// Store `bookmarkData` in nonvolatile storage

// Load URL from bookmark
// Load `bookmarkData` from nonvolatile storage
let bookmarkData = ...
var bookmarkWasStale = false
let URL = URL(resolvingBookmarkData: bookmarkData, options: [.withSecurityScope], &bookmarkWasStale)
// If bookmarkWasStale is `true`, recreate the bookmarkData with this new URL

// Start using the URL
let didStartAccessingSecurityScope = URL.startAccessingSecurityScopedResource()

// URL is now ready to use
// Remember to use `NSFileCoordinator` to access the file

// Stop using the URL
if (didStartAccessingSecurityScope) {
    URL.stopAccessingSecurityScopedResource() // Free resources
}

Saving URLs as bookmarks that capture the security scope ensures that your app continues to have access to the underlying file after your app has been terminated and relaunched.

NSItemProviderWriting and …Reading automate registration and loading

The task of selecting the appropriate representation to load can be boilerplate-heavy. To automate this chore for in-memory objects, Foundation offers the NSItemProviderReading and NSItemProviderWriting protocols.

These protocols can be implemented by model classes, which makes registration as simple as:

// Construct an item provider that registers all content types exportable from
// the model object
let itemProvider = NSItemProvider(object: modelObject)

And loading as simple as:

// Check to see if a model object can be created from a compatible representation
if itemProvider.canLoadObject(ofClass: ModelObject.self) {
    let progress = itemProvider.loadObject(ofClass: ModelObject.self) { 
        object, error in
        if let modelObject = object as? ModelObject {
            // ...
        }
    }
}

NSItemProviderWriting automates registration

The NSItemProviderWriting protocol allows NSItemProvider to query your model object for the content types it supports. The writableTypeIdentifiers property should return an array of type identifiers—in decreasing order of fidelity—that the object can offer, which NSItemProvider can then automatically register. When a consumer requests data, the loadData(withTypeIdentifier:, forItemProvider:, completionHandler:) method is called. Call the completion handler with the requested data to fulfill the promise.

Note that there are two versions of writableTypeIdentifiers: an instance property, and a class property. If implemented, the instance property is queried when you register an existing object using the registerObject(:, visibility:) function, otherwise, the class property is queried. Only the class property is queried when you register a promised object using the registerObject(ofClass:, visibility:, loadHandler:) function.

Always implement the class property; only implement the instance property when an override is needed.

For backward-compatibility reasons, the NSItemProviderReading and NSItemProviderWriting protocols only support Uniform Type Identifier strings, not UTType objects.

NSItemProviderReading automates loading

The NSItemProviderReading protocol allows NSItemProvider to select the best representation to use for constructing a model object of a specified class. Exact content type matches are preferred, but NSItemProvider will fall back to a conformance match if no exact match is found.

Your model class should implement the readableTypeIdentifiersForItemProvider class property to return an array of Uniform Type Identifier strings—in decreasing order of fidelity—that NSItemProvider can use to select the representation that will be used to construct an instance of the class. After loading is done, the object(withItemProviderData: typeIdentifier:) class method is called, and your implementation can instantiate an object from the incoming data.

Many SDK classes already implement NSItemProviderWriting and …Reading

Many built-in classes in the SDK already support NSItemProviderWriting and NSItemProviderReading. These classes include NSString, NSAttributedString, NSURL, NSImage, UIImage, UIColor, CNContact, MKMapItem, and many more. Before writing your own implementations, check to see if a class you’re using already supports these protocols.

NSItemProviderWriting and …Reading only support Data representations

NSItemProviderWriting and NSItemProviderReading only support providing and consuming representations as Data objects. To return the contents of files, your model object should read the file contents and return them as Data objects.

Fun fact: the NSAttributedString class can offer a file representation with the UTType.rtfd content type through NSItemProviderWriting conformance; but this facility is not available in the public API.

If your use case requires providing and consuming files, especially open-in-place files, you should manually register and load your representations.

UIKit and AppKit add functionality to NSItemProvider

UIKit provides a category for NSItemProvider in NSItemProvider+UIKitAdditions.h to better support Drag and Drop. AppKit provides a similar category in its NSItemProvider.h header for animating Share Sheet transitions. See those headers for more information.

UIKit uses the visibility parameter of registered representations to filter out representations during Drag and Drop. You can use this parameter to make certain representations visible to only your process, other apps belonging to your development team, or all apps.

Conclusion

That wraps up a comprehensive look at NSItemProvider, what it does, and how to use it. Here are a few key takeaways and best practices that you should remember:

  • An NSItemProvider object represents a promise for single item. Do not combine multiple items in one NSItemProvider.
  • An NSItemProvider makes available multiple representations for the item.
    • Each representation promises binary data, tagged with a content type (UTType) or a Uniform Type Identifier string. UTType and type identifier strings are mappable one-to-one.
    • The highest fidelity representation is registered first, followed by increasingly lower fidelity representations.
    • Register content types that specify byte encoding (i.e. UTType.jpeg, not UTType.image).
    • Representations can be provided and consumed in these ways:
      • Data
      • A file to be copied
      • A file to be opened-in-place
    • NSItemProvider bridges providers and consumers such that each representation can be provided in any of those ways, and consumed in any of those ways.
    • Providers can offer folders instead of files.
  • Loading data is an asynchronous operation.
    • Use Progress objects to report and monitor progress, and to cancel load requests midway.
    • Loader blocks and completion handler blocks are called on arbitrary queues.
  • Use NSItemProviderWriting and NSItemProviderReading conformance to make model objects easy to share and load using NSItemProvider.
    • Many commonly-used classes in the SDK already conform to these protocols.
  • Do not use the coercing interface ❌. Use the more modern interfaces introduced in iOS 11.0 and iOS 16.0.

I hope that helps. I know it was a bit of a trek. If you have any comments or questions, please feel free to contact me.