A wrapper class for input audio samples used in on-device machine learning using MediaPipe Task library.

Internally this class uses a ring buffer to hold input audio data. Clients could feed input audio data via load method and access the aggregated audio samples via buffer property.

Note that this class can only handle input audio in Float. TODO: Investigate if AVAudioFormatInt32 should be supported. Only necessary if you can change the input format of AVAudioEngine or load audio files using AVAudioFormatInt32.

Wraps properties describing the format of the incoming audio samples, namely number of channels and the sample rate.

A utility class to load audio files from file paths on iOS.

Audio part representing the audio content block of a multi-modal block.

A wrapper class to record the device's microphone continuously. Currently this class only supports recording up to 2 channels. If the number of channels is 2, then the mono microphone input is duplicated to provide dual channel data.

Internally the class manages an instance of AVAudioEngine for tapping the microphone samples.

Recording sets the following parameters on the AVAudioSession.sharedInstance():

1. category = .playAndRecord 2. mode = .default 3. categoryOptions = [.overrideMutedMicrophoneInterruption]

If you need more control, use the native AVAudioEngine directly for tapping the microphone. AudioRecord does not actively monitor audio interruptions, route changes, resetting of media services etc. They can be handled in the app using the notifications provided by AVFAudio. Refer to the official docs for the following: 1. Handling audio interruptions https://developer.apple.com/documentation/avfaudio/handling_audio_interruptions

2. Responding to audio route changes https://developer.apple.com/documentation/avfaudio/responding_to_audio_route_change

3. Responding to resetting of media services https://developer.apple.com/documentation/avfaudio/avaudiosessionmediaserviceswereresetnotification

iOS may automatically stop the audio engine in some cases. A few examples of such events are a system interrupt, route change, etc. In such cases, read(offset:length:) returns TasksErrorCode.audioRecordNotTappingMicError. You can restart recording using startRecording().

Note: Avoid setting the AVAudioSession.sharedInstance() category, mode, options, channel configuration (enabling stereo) or other parameters while audio is being actively recorded using the audio record. Setting these parameters or changing the configuration of AVAudioSession.sharedInstance() while a recording is in progress will result in undefined behaviour.

Holds the base options that is used for creation of any type of task. It has fields with important information acceleration configuration, TFLite model source etc.

A default text chunking implementation using native C++ word-based chunking with overlap or character sliding window.

Represents the embedding for a given embedder head. Typically used in embedding tasks.

One and only one of the two 'floatEmbedding' and 'quantizedEmbedding' will contain data, based on whether or not the embedder was configured to perform scala quantization.

Represents the embedding results of a model. Typically used as a result for embedding tasks.

Image part representing the image content block of a multi-modal block.

An image used in on-device machine learning using MediaPipe Task library.

A wrapper class to store pointer to a float array and its size.

Represents a single record retrieved from the SemanticRetriever.

Represents a single result retrieved from the SemanticRetriever, including similarity/relevance score.

Performs on-device semantic search, vector storage, and query retrieval on text, image, and audio files.

A configuration container class for initializing MPPSemanticRetriever.

A vector store implementation backed by SQLite database using native SQLite vector engine.

MediaPipe Tasks options base class. Any MediaPipe task-specific options class should extend this class.

Represents a part of a multi-modal content block.

MediaPipe Tasks result base class. Any MediaPipe task result class should extend this class.

Text part representing the text content block of a multi-modal block.

Performs multi-modal embedding extraction on text, image, audio, or custom content blocks.

This API wraps the native LiteRT-LM EmbeddingEngine directly, providing real on-device multimodal embeddings.

Options for setting up a MPPUniversalEmbedder.