PromptPrefix
data class PromptPrefix
A data class representing a prompt prefix.
By providing the shared prefix, the system can cache the processing of this prefix on supported devices, potentially reducing inference time. To enable prefix caching, use PromptPrefix for the static part of your prompt and TextPart for the dynamic part.
Note: Pre-processing the prefix may introduce a small amount of one-time latency. It's recommended to only use this for prefixes that are expected to be used multiple times.
Summary
Public constructors |
|---|
PromptPrefix(textString: String) |
Public properties |
|
|---|---|
String |