Your custom trained model must be configured such that Earth Engine can send well-formed, interpretable inference requests to the model as it is hosted on Vertex AI.
Supported Model Inputs
Earth Engine constructs requests on your behalf when performing inference.
Specify the payload format that EE will send requests with using the
payloadFormat parameter when you instantiate the model connector with
ee.Model.fromVertexAi.
gRPC Prediction Payloads
All hosted TensorFlow models can send predictions over the gRPC protocol. This is the preferred way to connect hosted models with Earth Engine as it will result in lower prediction latency and higher reliability.
GRPC_TF_TENSORS
Use the GRPC_TF_TENSORS payload format to use gRPC with TensorFlow models.
All properties and or bands will be encoded in a single
PredictRequest.
This PredictRequest will be converted to a dictionary of tensors for your
model to use.
GRPC_SERIALIZED_TF_TENSORS
Use the GRPC_SERIALIZED_TF_TENSORS format if you want to
migrate a Cloud AI Platform model that was previously already integrated with
Earth Engine without having to modify the model. You will need to re-upload and
re-deploy if container_grpc_ports is not set on your model in Vertex AI.
GRPC_SERIALIZED_TF_EXAMPLES
Use the GRPC_SERAILZED_TF_EXAMPLES for models that support tf.Example protocol
buffers. Earth Engine will send a single tensor named "input" that contains the
utf-8 encoded proto ByteString of an Example proto.
HTTP API Payloads
Vertex AI supports connecting to HTTP inference endpoints. Earth Engine supports several of the common HTTP payload formats. By default all Vertex AI custom models support the HTTP inference API.
SERIALIZED_TF_TENSORS
This is the default payloadFormat when connecting to a hosted model in Vertex
AI. This payload format is the most efficient of the HTTP payload formats when
using TensorFlow models.
Earth Engine will construct the inputs in the following way: for every band and
property required for your inference request will be a single key-value pair in
the instances object sent to your hosted model.
Each key will be the band or property name and each value will be a
Base64 encoded
TensorProto ByteString as a string_val.
RAW_JSON
For other model frameworks the most flexible format we can send is a JSON dictionary of named inputs and values. This payload format works well with PyTorch and AutoML models by default.
However do note that all numerical values will be converted into JSON strings. For example, to represent the number we encode 12.345 this as the string "12.345". Large inference payloads are not well supported with this payload format.
ND_ARRAYS
This is similar to RAW_JSON payload format but will omit the keys and only
pass in a list of numbers in the same format of calling
to_list()
on a NumPy ndarray. This payload format works well with PyTorch natively.