The wrapper class for MediaPipe Image objects. Images are stored as ImageData, ImageBitmap or WebGLTexture objects. You can convert the underlying type to any other type by passing the desired type to getAs...(). As type conversions can be expensive, it is recommended to limit these conversions. You can verify what underlying types are already available by invoking has...(). Images that are returned from a MediaPipe Tasks are owned by by the underlying C++ Task. If you need to extend the lifetime of these objects, you can invoke the clone() method. To free up the resources obtained during any clone or type conversion operation, it is important to invoke close() on the MPImage instance. Converting to and from ImageBitmap requires that the MediaPipe task is initialized with an OffscreenCanvas. As we require WebGL2 support, this places some limitations on Browser support as outlined here: https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas/getContext
Signature:
export declare class MPImage
Properties
| Property | Modifiers | Type | Description |
|---|---|---|---|
| canvas | HTMLCanvasElement | OffscreenCanvas | undefined | Returns the canvas element that the image is bound to. | |
| width | number | Returns the width of the image. | |
| height | number | Returns the height of the image. |
Methods
| Method | Modifiers | Description |
|---|---|---|
| hasImageData() | Returns whether this MPImage contains a mask of type ImageData. |
|
| hasImageBitmap() | Returns whether this MPImage contains a mask of type ImageBitmap. |
|
| hasWebGLTexture() | Returns whether this MPImage contains a mask of type WebGLTexture. |
|
| getAsImageData() | Returns the underlying image as an ImageData object. Note that this involves an expensive GPU to CPU transfer if the current image is only available as an ImageBitmap or WebGLTexture. |
|
| getAsImageBitmap() | Returns the underlying image as an ImageBitmap. Note that conversions to ImageBitmap are expensive, especially if the data currently resides on CPU. Processing with ImageBitmaps requires that the MediaPipe Task was initialized with an OffscreenCanvas with WebGL2 support. See https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas/getContext for a list of supported platforms. |
|
| getAsWebGLTexture() | Returns the underlying image as a WebGLTexture object. Note that this involves a CPU to GPU transfer if the current image is only available as an ImageData object. The returned texture is bound to the current canvas (see .canvas). |
|
| clone() | Creates a copy of the resources stored in this MPImage. You can invoke this method to extend the lifetime of an image returned by a MediaPipe Task. Note that performance critical applications should aim to only use the MPImage within the MediaPipe Task callback so that copies can be avoided. |
|
| close() | Frees up any resources owned by this MPImage instance. Note that this method does not free images that are owned by the C++ Task, as these are freed automatically once you leave the MediaPipe callback. Additionally, some shared state is freed only once you invoke the Task's close() method. |
MPImage.canvas
Returns the canvas element that the image is bound to.
Signature:
readonly canvas: HTMLCanvasElement | OffscreenCanvas | undefined;
MPImage.width
Returns the width of the image.
Signature:
readonly width: number;
MPImage.height
Returns the height of the image.
Signature:
readonly height: number;
MPImage.hasImageData()
Returns whether this MPImage contains a mask of type ImageData.
Signature:
hasImageData(): boolean;
Returns:
boolean
MPImage.hasImageBitmap()
Returns whether this MPImage contains a mask of type ImageBitmap.
Signature:
hasImageBitmap(): boolean;
Returns:
boolean
MPImage.hasWebGLTexture()
Returns whether this MPImage contains a mask of type WebGLTexture.
Signature:
hasWebGLTexture(): boolean;
Returns:
boolean
MPImage.getAsImageData()
Returns the underlying image as an ImageData object. Note that this involves an expensive GPU to CPU transfer if the current image is only available as an ImageBitmap or WebGLTexture.
Signature:
getAsImageData(): ImageData;
Returns:
ImageData
MPImage.getAsImageBitmap()
Returns the underlying image as an ImageBitmap. Note that conversions to ImageBitmap are expensive, especially if the data currently resides on CPU. Processing with ImageBitmaps requires that the MediaPipe Task was initialized with an OffscreenCanvas with WebGL2 support. See https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas/getContext for a list of supported platforms.
Signature:
getAsImageBitmap(): ImageBitmap;
Returns:
ImageBitmap
MPImage.getAsWebGLTexture()
Returns the underlying image as a WebGLTexture object. Note that this involves a CPU to GPU transfer if the current image is only available as an ImageData object. The returned texture is bound to the current canvas (see .canvas).
Signature:
getAsWebGLTexture(): WebGLTexture;
Returns:
WebGLTexture
MPImage.clone()
Creates a copy of the resources stored in this MPImage. You can invoke this method to extend the lifetime of an image returned by a MediaPipe Task. Note that performance critical applications should aim to only use the MPImage within the MediaPipe Task callback so that copies can be avoided.
Signature:
clone(): MPImage;
Returns:
MPImage
MPImage.close()
Frees up any resources owned by this MPImage instance. Note that this method does not free images that are owned by the C++ Task, as these are freed automatically once you leave the MediaPipe callback. Additionally, some shared state is freed only once you invoke the Task's close() method.
Signature:
close(): void;
Returns:
void