Setup guide for iOS

This page shows you how to set up your development environment to use MediaPipe Tasks in your iOS applications with Swift or Objective-C.

Supported devices and platforms

To create iOS applications with MediaPipe Tasks, your development environment requires the following:

macOS Ventura 13.0 or greater.
Xcode 14.3 or greater.
iOS device or simulator with at least iOS 15.0. An iOS simulator can run applications that don't require the physical device camera.

Developer environment setup

Before running a MediaPipe task in an iOS application, open an existing Xcode project or create a new one on your Mac.

MediaPipe Tasks is distributed through Swift Package Manager (SPM) in Xcode.

Example code setup

The MediaPipe Samples repository contains example iOS applications for MediaPipe Tasks.

The following steps build the Text Classifier sample app:

Open Terminal, clone the MediaPipe Samples repository, and navigate to the directory containing TextClassifier.xcodeproj:

git clone https://github.com/google-ai-edge/mediapipe-samples
cd mediapipe-samples/examples/text_classification/ios

Open TextClassifier.xcodeproj in Xcode. Xcode automatically resolves and downloads the required Swift package dependencies.
Select the TextClassifier scheme and choose a physical iOS device or simulator from the toolbar.
Click the Run button in your project's toolbar.

For more instructions on running an app in Xcode, refer to Building and running an app.

MediaPipe Tasks dependencies

MediaPipe Tasks provides five prebuilt Swift Package Manager products covering vision, text, audio, retrieval, and decision tasks. Bundle .task, .tflite, or .litertlm model files in your iOS app target resources (or load them from the app container).

Adding dependencies in Xcode

To add the MediaPipe Tasks Swift package to your Xcode project:

With your project open in Xcode, select File > Add Package Dependencies....
In the search field in the upper right, enter the repository URL:
```
https://github.com/google-ai-edge/mediapipe
```
Select the mediapipe package and set Dependency Rule to Branch with master (or Exact Commit).
Click Add Package.
In the package products dialog, select the libraries required for your application and add them to your application target:
- Vision tasks: MediaPipeTasksVision (see Prepare vision input data)
- Text tasks: MediaPipeTasksText
- Audio tasks: MediaPipeTasksAudio (see Prepare audio input data)
- Retrieval tasks: MediaPipeTasksRetrieval
- Decision tasks: MediaPipeTasksDecision
Note: MediaPipeTasksCommon is automatically included as a dependency.

Adding dependencies in `Package.swift`

If you are developing a Swift package, add the MediaPipe package and selected domain products to your Package.swift:

dependencies: [
    .package(url: "https://github.com/google-ai-edge/mediapipe", branch: "master"),
],
targets: [
    .target(
        name: "MyAppTarget",
        dependencies: [
            .product(name: "MediaPipeTasksVision", package: "mediapipe"),
            .product(name: "MediaPipeTasksText", package: "mediapipe"),
            .product(name: "MediaPipeTasksAudio", package: "mediapipe"),
            .product(name: "MediaPipeTasksRetrieval", package: "mediapipe"),
            .product(name: "MediaPipeTasksDecision", package: "mediapipe"),
        ]
    ),
]

BaseOptions configuration

BaseOptions allows general configuration of MediaPipe Task APIs:

Option name	Description	Accepted values
`modelAssetPath`	The path to a model file in the iOS application bundle or app container.	`String`
`delegate`	Hardware acceleration delegate used to execute the MediaPipe pipeline. Default value: `.CPU`.	`Delegate.CPU` \| `Delegate.GPU`

Hardware acceleration

MediaPipe Tasks runs on CPU (Delegate.CPU) by default on iOS and supports Metal GPU acceleration (Delegate.GPU) on compatible tasks and models:

import Foundation
import MediaPipeTasksVision

guard let modelPath = Bundle.main.path(forResource: "hand_landmarker", ofType: "task") else { return }

let options = HandLandmarkerOptions()
options.baseOptions.modelAssetPath = modelPath
options.baseOptions.delegate = .GPU

Troubleshooting

For help with technical questions related to MediaPipe, visit the discussion group or Stack Overflow for support from the community. To report bugs or make feature requests, file an issue on GitHub.

For help setting up your iOS development environment, visit the Apple Developer Documentation.