Record and play back an AR session on Android NDK

The Recording & Playback API enables you to record video and AR data once within a given environment and use that content to replace a live camera session.

Prerequisites

Make sure that you understand fundamental AR concepts and how to configure an ARCore session before proceeding.

Compatibility with other ARCore APIs

Due to the way session data is processed, ARCore APIs may produce different results during playback than observed during recording. They may also produce different results during subsequent playback sessions. For example, the number of detected trackables, the precise timing of their detection, and their poses over time may be different during playback.

Compatibility with Cloud Anchors

You can host and resolve Cloud Anchors while recording or playing back a session.

Recording

Start, stop, and check the status of an ARCore session recording.

Record an ARCore session

To record an ARCore session, configure the session and provide an MP4 URI for the recording. Call ArSession_startRecording() before the first call to ArSession_resume(). Recording automatically starts when the session resumes. To automatically stop recording when the session is paused, call ArRecordingConfig_setAutoStopOnPause(). To record a partial session, call ArSession_startRecording() while the session is running.

ArRecordingConfig* recording_config = nullptr;
ArRecordingConfig_create(ar_session, &recording_config);
ArRecordingConfig_setMp4DatasetUri(ar_session, recording_config,
                                   mp4_dataset_uri);
ArRecordingConfig_setAutoStopOnPause(ar_session, recording_config, true);

CHECK(ArSession_startRecording(ar_session, recording_config));
// …
// Resume ARCore session to start recording.
CHECK(ArSession_resume(ar_session));
// …
// Recording ends.
CHECK(ArSession_pause(ar_session));

Stop a recording

To stop recording without pausing the currently running AR session, call ArSession_stopRecording() and ArRecordingConfig_destroy().

ArStatus status = ArSession_stopRecording(ar_session);
ArRecordingConfig_destroy(recording_config);

Check recording status

ArSession_getRecordingStatus() can be used at any time to determine the current ArRecordingStatus.

ArRecordingStatus recording_status;
// Can be called at any time.
ArSession_getRecordingStatus(ar_session, &recording_status);
if (recording_status == AR_RECORDING_NONE) {
  // The dataset recorder is not recording.
} else if (recording_status == AR_RECORDING_OK) {
  // The dataset recorder is recording normally.
} else if (recording_status == AR_RECORDING_IO_ERROR) {
  // The dataset recorder encountered an error while recording.
}

Playback

Play back previously recorded AR sessions. Sessions play back in real time, and session playback or speed cannot be adjusted.

Play back a previously recorded session

To play back a previously recorded session, call ArSession_setPlaybackDatasetUri() before the first call to ArSession_resume().

Once playback has started due to the first call to ArSession_resume(), pausing the session by calling ArSession_pause() will suspend processing of all camera image frames and any other recorded sensor data in the dataset. Camera image frames and sensor frame data that is discarded in this way will not be reprocessed when the session is again resumed by calling ArSession_resume(). AR tracking for the session will generally suffer due to the gap in processed data.

// Specify previously recorded MP4 file.
CHECK(ArSession_setPlaybackDatasetUri(ar_session, mp4_dataset_uri));
// …
// Playback starts from the beginning of the dataset.
CHECK(ArSession_resume(ar_session));
// …
// Pause AR session, but allow playback to silently continue.
CHECK(ArSession_pause(ar_session));
// …
// Resume AR session. Playback continues with gap to paused session.
CHECK(ArSession_resume(ar_session));

Restart playback from the beginning

To restart a playback from the beginning of the dataset, pause the session and call ArSession_setPlaybackDatasetUri(), specifying the same MP4 recording, before resuming the session.

CHECK(ArSession_pause(ar_session));
// Pause and specify the *same* dataset:
CHECK(ArSession_setPlaybackDatasetUri(ar_session, mp4_dataset_uri));
// Playback starts from the *beginning* of the dataset.
CHECK(ArSession_resume(ar_session));

Play back a different session

To play back a different dataset, pause the session and specify the new dataset before resuming the session.

CHECK(ArSession_pause(ar_session));
// Pause and specify a *different* dataset:
CHECK(ArSession_setPlaybackDatasetUri(ar_session, other_mp4_dataset_uri));
// Playback starts from the *beginning* of the new dataset.
CHECK(ArSession_resume(ar_session));

Check playback status

Use ArSession_getPlaybackStatus() at any time to determine the current ArPlaybackStatus.

ArPlaybackStatus playback_status;
// Can be called at any time.
ArSession_getPlaybackStatus(ar_session, &playback_status);
if (playback_status == AR_PLAYBACK_NONE) {
  // The session is not playing back an MP4 dataset file.
} else if (playback_status == AR_PLAYBACK_OK) {
  // Playback is in process without issues.
} else if (playback_status == AR_PLAYBACK_IO_ERROR) {
  // Playback has stopped due to an error.
} else if (playback_status == AR_PLAYBACK_FINISHED) {
  // Playback has finished successfully.
}

What’s next