Compass 解决方案
使用生成式 AI 构建由客服人员提供支持的旅行规划应用
应用架构
下图简要介绍了如何结合使用一流的 Google 开发者服务来构建 AI 驱动的应用。
构建 AI 赋能的精美应用,针对移动设备和 Web 应用
您可以使用 Flutter 和 Firebase Genkit 构建可与 AI 无缝集成的多平台应用。
从您的 Flutter 应用中无缝调用 Firebase Genkit
利用 Genkit,您可以定义能够验证 LLM 输出的架构,让您的应用能够放心地使用来自 LLM 的数据。在 Flutter 中,您可以使用 Dart 对请求进行序列化,并将响应反序列化,以便使用标准 HTTP 请求与 Genkit 架构匹配。
在 IDX 中开始使用 Flutter
借助 IDX,您无需安装任何软件即可创建 Flutter 应用。这样,您就可以在浏览器中开发和测试 Android 应用和 Web 应用了。
使用 Firebase Data Connect 连接数据库
Firebase Data Connect 是一项适用于移动应用和 Web 应用的关系型数据库服务,可让您使用由 Cloud SQL 提供支持的全托管式 PostgreSQL 数据库来构建和扩缩。它使用与 Firebase Authentication 完美集成的 GraphQL 提供安全架构以及查询和变更管理。Data Connect 包含对 Kotlin Android 和 Web 的 SDK 支持。
使用 Firebase Genkit 构建代理
代理使用 AI 编排框架来接收用户输入并生成响应。
--- model: googleai/gemini-1.5-flash-latest config: temperature: 1.0 safetySettings: - category: HARM_CATEGORY_HATE_SPEECH threshold: BLOCK_LOW_AND_ABOVE - category: HARM_CATEGORY_DANGEROUS_CONTENT threshold: BLOCK_ONLY_HIGH - category: HARM_CATEGORY_HARASSMENT threshold: BLOCK_LOW_AND_ABOVE - category: HARM_CATEGORY_SEXUALLY_EXPLICIT threshold: BLOCK_LOW_AND_ABOVE input: schema: request: string, The users request for where they want to travel to. place: string, The place that closely represents the users request. placeDescription: string, A description of that place. activities(array, a stringify list of activities that can be found at the specified place): string restaurants?(array, a stringify list of all the restaurants found at that location): string output: schema: place: string, The place the user is traveling to. itineraryName: string, a catchy itinerary name that encapsulates the spirit of the trip and includes the place name startDate: string, the start date of the trip endDate: string, the end date of the trip tags(array, relevant tags for the trip): string itinerary(array): day: number date: string planForDay(array): activityRef: string, the reference value for the activity - this comes from the available activities JSON. If no value is present use a ref value of restaurant. activityTitle: string, a catchy title for the activity activityDesc: string, a six word description of the activity photoUri?: string, set the photo uri value for restaurants only. googleMapsUri?: string, if this is a restaurant include the googleMapsUri --- Generate an itinerary for a tourist planning on traveling to the location specified based in their request. If there is something that does not exist within the list of activities, do not include it in your answer. Feel free to relate the activitiy to the request in a meaningful way. In the plan for day array, put activities as a travel brouchure might do. Come up with a catchy name for the itinerary. Pick three activities per day, minimum of three day trip unless otherwise specified in the request. Output schema should not include the properties type or object. Pick a date after 2024-05-14 but before 2024-12-31. The output date must be in the format year-month-day. Give each activity a unique title and description. Limit activity descriptions to 6 words. If no restaurants are supplied, do not recommend any restaurants to eat at. {{#if restaurants}} Find a restaurant to eat at each day. Include a restaurant to visit in the itinerary for each day from the available restaurants. The restaurant should be the only activity with a photoUri. The photoUri for the restaurant should be from the photoUri property from the restaurant array. If there are no restaurants to pick from, do not include it in the list. The photoUri from the restaurantFinder should be in the format of places/${placeId}/photos/${photoId} Each restaurant should be unique to the overall itinerary. Each restaurant must contain a photoUri in their output JSON schema. Each restaurant must also include an activitiyRef, activityTitle, and activityDesc in their output {{/if}} Output must be in JSON format. REQUEST : {{request}} PLACE : {{place}} PLACE DESCRIPTION : {{placeDescription}} AVAILABLE ACTIVITIES : {{activities}} RESTAURANTS : {{restaurants}}
在 Firebase Genkit 中创建您的第一个流程
使用生成式 AI 时,请务必撰写有效的提示,以便模型返回高质量的回答。Firebase Genkit 提供 Dotprompt 插件和文本格式,可帮助您编写和整理生成式 AI 提示。该格式可将提示、输入和输出架构、模型和配置全部封装在一个文件中。
以下代码示例展示了旅行应用中使用的 Dotprompt 文件。架构基于用户在描述梦想行程时提供的信息。
以下代码示例展示了旅行应用中使用的 Dotprompt 文件。架构基于用户在描述梦想行程时提供的信息。
import {defineTool} from '@genkit-ai/ai/tool'; ... { name: 'restaurantFinder', description: `Used when needing to find a restaurant based on a users location. The location should be used to find nearby restaurants to a place. You can also selectively find restaurants based on the users preferences, but you should default to 'Local' if there are no indications of restaurant types in the users request. `, inputSchema: z.object({ place: z.string(), typeOfRestaurant: z.string().optional() }), outputSchema: z.unknown(), }, ... async (input) => { if (input.typeOfRestaurant == undefined) { input.typeOfRestaurant = "Local"; } const geocodeEndpoint = "https://places.googleapis.com/v1/places:searchText"; const textQuery = {textQuery: `${input.typeOfRestaurant} restaurants in ${input.place}`}; const response = await axios.post( geocodeEndpoint, JSON.stringify(textQuery), { headers: { "Content-Type": "application/json", "X-Goog-Api-Key": MAPS_API_KEY, "X-Goog-FieldMask": "places.displayName,places.formattedAddress,places.priceLevel,places.photos.name,places.editorialSummary,places.googleMapsUri" } } ); console.log(response.data); let data = (response.data as PlaceResponse); for(let i = 0; i < data.places.length; i++) { if (data.places[i].photos) { data.places[i].photos = [data.places[i].photos[0]]; } } return data as PlaceResponse; }
通过添加与外部服务进行交互的工具来扩展代理的功能
您可以使用 Genkit 中的函数调用来扩展代理的功能,以便代理可以进一步优化响应并完成其他任务。旅行应用定义了一个工具,该工具可根据用户所需的行程,通过 Places API 返回餐厅信息。代码使用 Zod 定义输入和输出架构,以便验证查询结果。
... export const textRefinement = defineFlow( { name: 'textRefinement', inputSchema: z.string(), outputSchema: z.unknown(), }, async (userRequest) => { const refinementPrompt = await prompt('textRefinement') const result = await refinementPrompt.generate({ input: { request: userRequest }, }); return result.output(); });
优化搜索查询
为了帮助为用户提供更加个性化的搜索体验,在用户描述梦想行程后,Gemini 会根据旅行应用提供的提示判断是否需要更多信息,并在认为需要更多信息时向应用发出信号。然后,应用提示用户输入该信息,并将其附加到后端的请求中。
import 'package:http:http.dart' as http; ... Future<List<Trip>> generateTrips(String description, List<Image> images) async { final uri = Uri.parse('.../generateTrips'); final request = http.MultipartRequest('POST', uri); request.fields['description'] = description; request.files.add(http.MultipartFile.fromData( images.name, images.bytes, contentType: MediaType('image', 'png'), )); var response = await request.send(); if (response.statusCode == 200) { final body = await response.body.text(); final items = jsonDecode(body) as List<dynamic>; return items.map(Trip.fromJson).toList(); } ... import { imagen2, geminiProVision } from '@genkit-ai/vertexai'; import { generate } from '@genkit-ai/ai'; const imageResult = await generate({ model: imagen2, prompt: 'Generate an image of a very specific historical time and place.', }); const generatedImage = imageResult.media(); const descriptionResult = await generate({ model: geminiProVision, prompt: [ { text: 'What is the historical time and place represented in this picture?', }, { media: generatedImage }, ], }); console.log(descriptionResult.text()); }
生成建议行程概览列表
该旅行应用要求用户通过使用文本输入或点按麦克风按钮来激活语音转文字功能,从而定义他们梦想中的行程。用户也可以选择上传图片。
该应用利用 pub.dev 中的 Dart 软件包集成了适用于每个平台的原生语音转文字功能,并使用 Firebase Genkit 中的 Gemini API 来处理图片或视频等多模态输入。Gemini API 使用检索增强生成 (RAG) 技术返回一组建议的行程,然后使用 Firebase Data Connect 和嵌入执行最邻近对象搜索。
该应用利用 pub.dev 中的 Dart 软件包集成了适用于每个平台的原生语音转文字功能,并使用 Firebase Genkit 中的 Gemini API 来处理图片或视频等多模态输入。Gemini API 使用检索增强生成 (RAG) 技术返回一组建议的行程,然后使用 Firebase Data Connect 和嵌入执行最邻近对象搜索。
将应用扩展到生产环境
遵循所有云服务的生产指南
Firebase Hosting 可与 Flutter 等流行的现代 Web 框架集成。通过将 Firebase Hosting 和 Cloud Functions for Firebase 与这些框架搭配使用,您可以在自己偏好的框架环境中开发应用和微服务,然后在安全的代管式服务器环境中进行部署。在发布正式版之前,请先了解应用中所有服务的安全性和性能。如需了解详情,请参阅 Firebase 发布核对清单。
Vertex AI
这款旅行应用使用 Google AI 快速迭代测试数据,非常适合拥有极少 AI 用例且无需扩缩的应用。Vertex AI 拥有更高的配额,有助于扩大生产应用规模,并拥有更强大的隐私权政策来保护用户数据。Genkit 内置了可轻松切换模型的功能,因此您无需重写提示或 API 调用。