TFRecord and Earth Engine

TFRecord is a binary format for efficiently encoding long sequences of tf.Example protos. TFRecord files are easily loaded by TensorFlow through the tf.data package as described here and here. This page describes how Earth Engine converts between ee.FeatureCollection or ee.Image and TFRecord format.

Exporting data to TFRecord

You can export tables (ee.FeatureCollection) or images (ee.Image) to TFRecord files in Google Drive or Cloud Storage. Configuration of the export depends on what you are exporting as described below. All numbers exported from Earth Engine to TFRecord are coerced to float type.

Exporting tables

When exporting an ee.FeatureCollection to a TFRecord file, there is a 1:1 correspondence between each ee.Feature in the table and each tf.train.Example (i.e. each record) in the TFRecord file. Each property of the ee.Feature is encoded as a tf.train.Feature with a list of floats corresponding to the number or ee.Array stored in the property. If you export a table with arrays in the properties, you need to tell TensorFlow the shape of the array when it is read. A table exported to a TFRecord file will always be compressed with the GZIP compression type. You always get exactly one TFRecord file for each export.

The following example demonstrates parsing data from an exported table of scalar properties ('B2',...,'B7', 'landcover'). Note that the dimension of the float lists is [1] and the type is tf.float32:

dataset = tf.data.TFRecordDataset(exportedFilePath)

featuresDict = {
  'B2': tf.io.FixedLenFeature(shape=[1], dtype=tf.float32),
  'B3': tf.io.FixedLenFeature(shape=[1], dtype=tf.float32),
  'B4': tf.io.FixedLenFeature(shape=[1], dtype=tf.float32),
  'B5': tf.io.FixedLenFeature(shape=[1], dtype=tf.float32),
  'B6': tf.io.FixedLenFeature(shape=[1], dtype=tf.float32),
  'B7': tf.io.FixedLenFeature(shape=[1], dtype=tf.float32),
  'landcover': tf.io.FixedLenFeature(shape=[1], dtype=tf.float32)
}

parsedDataset = dataset.map(lambda example: tf.io.parse_single_example(example, featuresDict))
        

Note that this example illustrates reading scalar features (i.e. shape=[1]). If you are exporting 2D or 3D arrays (e.g. image patches), then you would specify the shape of your patches at parse time, for example shape=[16, 16] for a 16x16 pixel patch.

Exporting images

When you export an image, the data are ordered as channels, height, width (CHW). The export may be split into multiple TFRecord files with each file containing one or more patches of size patchSize, which is user specified in the export. The size of the files in bytes is user specified in the maxFileSize parameter. There is a 1:1 correspondence between each patch and each tf.train.Example in the resulting TFRecord file. Each band of the image is stored as a separate tf.train.Feature in each tf.train.Example, where the length of the float list stored in each feature is the patch width * height. The flattened lists can be split into multiple individual pixels as shown in this example. Or the shape of the exported patch can be recoved as in this example.

To help reduce edge effects, the exported patches can overlap. Specifically, you can specify a kernelSize which will result in tiles of size:

[patchSize[0] + kernelSize[0], patchSize[1] + kernelSize[1]]
    

Each tile overlaps adjacent tiles by [kernelSize[0]/2, kernelSize[1]/2]. As a result, a kernel of size kernelSize centered on an edge pixel of a patch of size patchSize contains entirely valid data. The spatial arrangement of the patches in space is illustrated by Figure 1, where Padding Dimension corresponds to the part of the kernel that overlaps the adjacent image:

formatOptions

The patchSize, maxFileSize, and kernelSize parameters are passed to the ee.Export (JavaScript) or ee.batch.Export (Python) call through a formatOptions dictionary, where keys are the names of additional parameters passed to Export. Possible formatOptions for an image exported to TFRecord format are:

The TFRecord “mixer” file

When you export to TFRecord, Earth Engine will generate a sidecar with your TFRecord files called the “mixer.” This is a simple JSON file used to define the spatial arrangement of the patches (i.e. georeferencing). This file is needed for uploading predicions made on the imagery as described in the next section.

Exporting Time Series

Image exports to both Examples and SequenceExamples are supported. When you export to Examples, the export region is cut into patches and those patches are exported in row-major order to some number of .tfrecord files with each band its own feature (unless you specify collapseBands). When you export to SequenceExamples, a SequenceExample per-pixel will be exported, with those SequenceExamples in row-major order within a patch, and then in row-major order of patches in the original export region (if you’re ever unsure, always assume things will be in row-major order in some capacity). Note: any scalar bands of an image will be packed into the context of a SequenceExample, while the array bands will become the actual sequence data.

Array Bands

Array bands are exportable when an image is exported to TFRecord format. Export of array bands provides a means to populate the “FeatureLists” of SequenceExamples, and a way to create 3D tensors when exporting to regular Examples. For information on how the lengths/depths of array bands are managed, see collapseBands and/or tensorDepths in the table above. Note: usage of collapseBands and export to SequenceExamples (so setting the parameter sequenceData) will result in all bands being collapsed to a single time series per-pixel.

Uploading TFRecords to Earth Engine

You can upload tables (command line only) and images to Earth Engine as TFRecord files. For tables, the 1:1 relationship described previously applies in the reverse direction (i.e. tf.train.Example -> ee.Feature).

Uploading imagery

If you generate predictions on exported imagery, supply the mixer when you upload the predictions (as TFRecord files) to obtain georeferenced imagery. Note that the overlapping portion of the patches (Padding Dimension in Figure 1) will be discarded to result in conterminous coverage of the exported region. The predictions should be arranged as a tf.train.Example sequence of the same number and order as your originally exported image examples (even between an arbitrary number of files).