MediaPipeTasksRetrieval Framework Reference

AudioData

class AudioData: NSObject

A wrapper class for input audio samples used in on-device machine learning using MediaPipe Task library.

Internally this class uses a ring buffer to hold input audio data. Clients could feed input audio data via load method and access the aggregated audio samples via buffer property.

Note that this class can only handle input audio in Float. TODO: Investigate if AVAudioFormatInt32 should be supported. Only necessary if you can change the input format of AVAudioEngine or load audio files using AVAudioFormatInt32.

  • Initializes a new instance of AudioData with the given AudioFormat and sample count.

    The AudioData stores data in a ring buffer of length sampleCount * MPPAudioFormat.channelCount.

    Declaration

    Swift

    init(format: AudioDataFormat, sampleCount: UInt)

    Parameters

    format

    An instance of AudioFormat that defines the number of channels and sample rate required by the TFLite model.

    sampleCount

    The number of samples AudioData can store at any given time. The sampleCount provided will be used to calculate the buffer length of AudioData (with bufferLength = format.channelCount * sampleCount.

    Return Value

    A new instance of AudioData with the given audio format and sample count.

  • A copy of all the internal buffer elements in order with the most recent elements appearing at the end of the buffer.

    Declaration

    Swift

    var buffer: MPPFloatBuffer { get }

  • Capacity of the AudioData buffer in number of elements.

    Declaration

    Swift

    var bufferLength: UInt { get }

  • Audio format specifying the number of channels and sample rate supported.

    Declaration

    Swift

    var format: AudioDataFormat { get }

  • Loads the internal buffer of AudioData with the audio samples currently held by the audio record.

    New data from the audio record is appended to the end of the buffer by shifting out any old data from the beginning of the buffer if needed to make space. If the length of the new data to be copied is more than the capacity of the buffer, only the most recent data of the AudioData's buffer will be copied from the input buffer.

    Declaration

    Swift

    func load(audioRecord: AudioRecord) throws

    Parameters

    audioRecord

    MPPAudioRecord whose values are to be loaded into AudioData. For multi-channel input, the audio record must hold interleaved data.

    Return Value

    A boolean indicating if the load operation succeeded.

  • Loads the internal buffer of AudioData with a slice of the audio samples contained in the provided FloatBuffer.

    New data from the input buffer is appended to the end of the buffer by shifting out any old data from the beginning of the buffer if needed to make space. If the length of the new data to be copied is more than the capacity of the buffer, only the most recent data of the AudioData's buffer will be copied from the input buffer.

    Declaration

    Swift

    func load(buffer: MPPFloatBuffer, offset: UInt, length: UInt) throws

    Parameters

    sourceBuffer

    A float buffer whose values are to be loaded into AudioData. For multi-channel input, the array must be interleaved.

    offset

    Starting index in the source buffer from which elements should be copied.

    length

    The number of elements to be copied.

    Return Value

    A boolean indicating if the load operation succeeded.