SpeechRecognizer

class SpeechRecognizer : Closeable


Provides an interface for performing on-device speech recognition.

It supports audio input from microphone, files, URIs, and file descriptors via AudioSource.

Typical usage:

Example 1: Recognizing from Microphone

// 1. Create a SpeechRecognizer with desired options.
val options: SpeechRecognizerOptions =
speechRecognizerOptions {
locale = Locale.US
preferredMode = SpeechRecognizerOptions.Mode.MODE_ADVANCED
}
val speechRecognizer: SpeechRecognizer = SpeechRecognition.getClient(options)

// 2. Check if the recognition model is available or needs downloading.
launch {
val status: Int = speechRecognizer.checkStatus()
if (status == FeatureStatus.DOWNLOADABLE) {
// 3. If needed, download the model.
speechRecognizer.download().collect { downloadStatus ->
when (downloadStatus) {
is DownloadStatus.DownloadCompleted -> {
// Model is ready, start recognition.
startMyRecognition(speechRecognizer)
}
is DownloadStatus.DownloadFailed -> {
// Handle download failure.
}
is DownloadStatus.DownloadProgress -> {
// Handle download progress.
}
else -> {}
}
}
} else if (status == FeatureStatus.AVAILABLE) {
// Model is ready, start recognition.
startMyRecognition(speechRecognizer)
} else {
// Handle other statuses like DOWNLOADING or UNAVAILABLE.
}
}

// 4. Define your recognition logic.
suspend fun startMyRecognition(recognizer: SpeechRecognizer) {
// Create a request and collect the responses.
val request = speechRecognizerRequest {
audioSource = AudioSource.fromMic()
}
recognizer.startRecognition(request).collect { response ->
// Process the responses.
when (response) {
is SpeechRecognizerResponse.PartialTextResponse -> { /* ... */}
is SpeechRecognizerResponse.FinalTextResponse -> { /* ... */}
else -> {}
}
}
}

// 5. Stop recognition when needed.
launch {
speechRecognizer.stopRecognition()
speechRecognizer.close()
}

Example 2: Recognizing from a File Descriptor in Streaming Mode (using a Pipe)

When using a ParcelFileDescriptor as the audio source in streaming mode, you must pace the writes to the file descriptor to match real-time playback (e.g., 3200 bytes every 100ms for 16kHz, 16-bit, mono PCM). Writing too fast may overwhelm the buffer.

suspend fun recognizeFromFile(recognizer: SpeechRecognizer, audioData: ByteArray) {
val (readFd, writeFd) = ParcelFileDescriptor.createPipe()

// Launch a coroutine to write to the pipe in a background thread (Dispatchers.IO)
val writeJob = CoroutineScope(Dispatchers.IO).launch {
ParcelFileDescriptor.AutoCloseOutputStream(writeFd).use { outputStream ->
// Example for STREAMING mode: pace the writes to simulate real-time.
// (3200 bytes is 100ms of 16kHz 16-bit mono PCM: 16000 samples/sec * 2 bytes/sample * 0.1 sec)
val chunkSize = 3200
var offset = 0
while (offset < audioData.size && isActive) {
val len = minOf(chunkSize, audioData.size - offset)
outputStream.write(audioData, offset, len)
offset += len
delay(100) // Pace the stream
}
}
}

val audioSource = AudioSource.fromPfd(readFd, AudioSource.Mode.STREAMING)
val request = speechRecognizerRequest {
this.audioSource = audioSource
}

try {
recognizer.startRecognition(request).collect { response ->
when (response) {
is SpeechRecognizerResponse.PartialTextResponse -> {
println("Partial: ${response.text}")
}
is SpeechRecognizerResponse.FinalTextResponse -> {
println("Final: ${response.text}")
}
else -> {}
}
}
} finally {
// Ensure the write job is cancelled and the audioSource is closed
writeJob.cancel()
audioSource.close()
}
}

Example 3: Recognizing Pre-Recorded Audio in One-Shot Mode

For pre-recorded audio files or batch audio buffers, provide an AudioSource in one-shot mode (e.g. via AudioSource.fromFile). In one-shot mode, the recognizer processes the entire audio input at once before producing the final result.

// 1. Create a SpeechRecognizer.
val options = speechRecognizerOptions {
locale = Locale.US
preferredMode = SpeechRecognizerOptions.Mode.MODE_BASIC
}
val speechRecognizer = SpeechRecognition.getClient(options)

// 2. Recognize from a File in ONE_SHOT mode.
suspend fun recognizeFromFile(recognizer: SpeechRecognizer, audioFile: File) {
val audioSource = AudioSource.fromFile(audioFile)
val request = speechRecognizerRequest {
this.audioSource = audioSource
}

try {
recognizer.startRecognition(request).collect { response ->
when (response) {
is SpeechRecognizerResponse.FinalTextResponse -> {
println("Final: ${response.text}")
}
else -> {}
}
}
} finally {
audioSource.close()
}
}

// 3. Close the recognizer when no longer needed.
speechRecognizer.close()

Summary

Public constructors

Public functions

suspend @FeatureStatus Int

Checks the current availability status of the speech recognition feature.

open Unit

Releases resources associated with the speech recognition engine.

Flow<DownloadStatus>
@OptIn(markerClass = [ExperimentalAtomicApi])
download()

Downloads the required model assets for the speech recognition feature if they are not already available.

@FeatureStatus Int
Flow<SpeechRecognizerResponse>

Starts a speech recognition session when the returned Flow is collected.

suspend Unit

Stops the current speech recognition session.

suspend Unit

Prepares the speech recognition engine by loading required resources in advance to reduce latency for subsequent recognition requests.

Public constructors

SpeechRecognizer

@VisibleForTesting
SpeechRecognizer(
    options: SpeechRecognizerOptions,
    mlKitStatsLogger: MLKitStatsLogger,
    coroutineStub: GoogleAsrServiceGrpcKt.GoogleAsrServiceCoroutineStub? = null
)

Public functions

checkStatus

suspend fun checkStatus(): @FeatureStatus Int

Checks the current availability status of the speech recognition feature.

Returns
@FeatureStatus Int

FeatureStatus indicating feature readiness.

close

open fun close(): Unit

Releases resources associated with the speech recognition engine.

This should be called when the speech recognizer is no longer needed.

download

@OptIn(markerClass = [ExperimentalAtomicApi])
fun download(): Flow<DownloadStatus>

Downloads the required model assets for the speech recognition feature if they are not already available.

Use this method to proactively download models before inference. A flow of DownloadStatus reports progress and completion status.

Returns
Flow<DownloadStatus>

Flow for collecting to start the download and receive DownloadStatus.

startRecognition

@OptIn(markerClass = [ExperimentalAtomicApi])
fun startRecognition(request: SpeechRecognizerRequest): Flow<SpeechRecognizerResponse>

Starts a speech recognition session when the returned Flow is collected.

Downstream collectors receive streaming updates such as SpeechRecognizerResponse.PartialTextResponse, SpeechRecognizerResponse.FinalTextResponse, and SpeechRecognizerResponse.AudioLevelResponse. Runtime recognition errors are delivered as SpeechRecognizerResponse.ErrorResponse items within the flow rather than thrown as exceptions during collection, and SpeechRecognizerResponse.CompletedResponse is emitted upon normal completion.

Parameters
request: SpeechRecognizerRequest

the SpeechRecognizerRequest containing the audio source and other configuration.

Throws
IllegalArgumentException

if word timing is enabled with AudioSource.Mode.ONE_SHOT.

stopRecognition

suspend fun stopRecognition(): Unit

Stops the current speech recognition session.

This method gracefully terminates the recognition job. The SpeechRecognizerResponse.CompletedResponse will be sent after the job is fully closed.

warmup

suspend fun warmup(): Unit

Prepares the speech recognition engine by loading required resources in advance to reduce latency for subsequent recognition requests.