Class that performs classification on audio data.

This API expects a TFLite model with mandatory TFLite Model Metadata that contains the mandatory AudioProperties of the solo input audio tensor and the optional (but recommended) category labels as AssociatedFiles with type TENSOR_AXIS_LABELS per output classification tensor.

Input tensor (kTfLiteFloat32) - input audio buffer of size [batch * samples]. - batch inference is not supported (batch is required to be 1). - for multi-channel models, the channels must be interleaved.

At least one output tensor with: (kTfLiteFloat32) - [1 x N] array with N represents the number of categories. - optional (but recommended) category labels as AssociatedFiles with type TENSOR_AXIS_LABELS, containing one label per line. The first such AssociatedFile (if any) is used to fill the category_name field of the results. The display_name field is filled from the AssociatedFile (if any) whose locale matches the display_names_locale field of the AudioClassifierOptions used at creation time ("en" by default, i.e. English). If none of these are available, only the index field of the results will be filled.

Options for setting up a AudioClassifier.

Represents the classification results generated by AudioClassifier. *

A wrapper class for input audio samples used in on-device machine learning using MediaPipe Task library.

Internally this class uses a ring buffer to hold input audio data. Clients could feed input audio data via load method and access the aggregated audio samples via buffer property.

Note that this class can only handle input audio in Float. TODO: Investigate if AVAudioFormatInt32 should be supported. Only necessary if you can change the input format of AVAudioEngine or load audio files using AVAudioFormatInt32.

Wraps properties describing the format of the incoming audio samples, namely number of channels and the sample rate.

Audio part representing the audio content block of a multi-modal block.

A wrapper class to record the device's microphone continuously. Currently this class only supports recording up to 2 channels. If the number of channels is 2, then the mono microphone input is duplicated to provide dual channel data.

Internally the class manages an instance of AVAudioEngine for tapping the microphone samples.

Recording sets the following parameters on the AVAudioSession.sharedInstance():

1. category = .playAndRecord 2. mode = .default 3. categoryOptions = [.overrideMutedMicrophoneInterruption]

If you need more control, use the native AVAudioEngine directly for tapping the microphone. AudioRecord does not actively monitor audio interruptions, route changes, resetting of media services etc. They can be handled in the app using the notifications provided by AVFAudio. Refer to the official docs for the following: 1. Handling audio interruptions https://developer.apple.com/documentation/avfaudio/handling_audio_interruptions

2. Responding to audio route changes https://developer.apple.com/documentation/avfaudio/responding_to_audio_route_change

3. Responding to resetting of media services https://developer.apple.com/documentation/avfaudio/avaudiosessionmediaserviceswereresetnotification

iOS may automatically stop the audio engine in some cases. A few examples of such events are a system interrupt, route change, etc. In such cases, read(offset:length:) returns TasksErrorCode.audioRecordNotTappingMicError. You can restart recording using startRecording().

Note: Avoid setting the AVAudioSession.sharedInstance() category, mode, options, channel configuration (enabling stereo) or other parameters while audio is being actively recorded using the audio record. Setting these parameters or changing the configuration of AVAudioSession.sharedInstance() while a recording is in progress will result in undefined behaviour.

Performs automatic speech recognition (ASR) on streaming mono PCM float audio samples.

Options for configuring a MediaPipe MPPAudioTranscriber task.

Represents the transcription results generated by MPPAudioTranscriber.

A single transcribed segment emitted by MPPAudioTranscriber.

Holds the base options that is used for creation of any type of task. It has fields with important information acceleration configuration, TFLite model source etc.

Represents the classification results of a model. Typically used as a result for classification tasks.

Represents the list of classification for a given classifier head. Typically used as a result for classification tasks.

Represents the embedding for a given embedder head. Typically used in embedding tasks.

One and only one of the two 'floatEmbedding' and 'quantizedEmbedding' will contain data, based on whether or not the embedder was configured to perform scala quantization.

Represents the embedding results of a model. Typically used as a result for embedding tasks.

Image part representing the image content block of a multi-modal block.

A wrapper class to store pointer to a float array and its size.

Category is a util class that contains a label, its display name, a float value as score, and the index of the label in the corresponding label file. Typically it's used as the result of classification tasks.

Bidirectional / pull-based streaming synthesis handle supporting Level 2 incremental LLM text pushing (pushText + finishInput) alongside readNext() and cancel().

MediaPipe Tasks options base class. Any MediaPipe task-specific options class should extend this class.

Represents a part of a multi-modal content block.

MediaPipe Tasks result base class. Any MediaPipe task result class should extend this class.

Text part representing the text content block of a multi-modal block.

MediaPipe TextToSpeech task interface for Swift (iOS / macOS).