solution: metadata: id: compass title: Build an agent-powered travel planning app with Generative AI description: Learn how to use Flutter and Genkit to build multi-platform apps that can seamlessly integrate with AI. fallbackCta: title: Launch in Firebase Studio icon: firebase-studio url: https://idx.google.com/import?url=https%3A%2F%2Fgithub.com%2FFirebaseExtended%2Fcompass-ai-travel-planning-sample-flutter accentColor: foreground: "white" background: "#141e2a" sections: - type: watch youtubeId: vLHwDTNfxdE title: Build an agent-powered travel planning app with Generative AI description: Learn how to use Flutter and Genkit to build multi-platform apps that can seamlessly integrate with AI. technologies: - icon: /external-assets/flutter.svg label: Flutter url: https://flutter.dev - icon: /external-assets/firebase.svg label: Firebase url: https://firebase.google.com/ - icon: /external-assets/vertex-ai.svg label: Google AI url: https://cloud.google.com/vertex-ai - icon: /external-assets/gemini.svg label: Gemini url: https://ai.google.dev/ - icon: /external-assets/places-api.svg label: Places API url: https://developers.google.com/maps/documentation/places/web-service - type: explore mode: video subsections: - id: video-1 title: Querying for pre-generated suggestions and image data description: | As the user initiates a search, the app queries a pre-generated list of suggestions and image data. The images were generated ahead of time using the Imagen API. videoPath: external-assets/compass/compass-1.mp4 orientation: landscape frame: laptop layout: sidebyside logs: - timestamp: 7000 summary: Flutter app queries the pre-generated list of suggestions detail: The suggestions and imagery have been generated ahead of time with Gemini and Imagen. imageData: "" sequence: - flutter - firebase-genkit - places-api inspect: genkit-gemini - timestamp: 14000 summary: Flutter retrieves pre-generated suggestions and images detail: Images were generated ahead of time using Imagen. imageData: /external-assets/compass/temple.png sequence: - flutter - firebase-genkit - google-ai - gemini inspect: genkit-places - id: video-2 title: Sending text and images to Gemini for inspiration description: | Our app's Flutter code sends a multimodal prompt of text and image data to Gemini via Genkit and Vertex AI to generate travel suggestions. videoPath: external-assets/compass/compass-2.mp4 orientation: portrait frame: androidphone layout: sidebyside logs: - timestamp: 10000 summary: Flutter sends text and image prompt data to Gemini via Genkit and Vertex AI detail: Prompt text "Traveling in China to see Shanghai and Sichuan" imageData: /external-assets/compass/buddha.jpeg # file: restaurantFinder.ts sequence: - flutter - firebase-genkit - google-ai - gemini inspect: flutter-genkit - timestamp: 15000 summary: Flutter retrieves pre-generated suggestions and images detail: Images were generated ahead of time using Imagen. imageData: /external-assets/compass/lanterns.jpeg file: restaurantFinder.ts sequence: - flutter - firebase-genkit - vertex-ai - gemini inspect: genkit-places - type: inspect subsections: - id: genkit-dataconnect title: Genkit & Data Connect API examples: - title: Connect your database with Firebase Data Connect icons: - /external-assets/firebase-genkit.svg - /external-assets/firebase-connect.svg links: - label: View Data Connect quickstart url: https://firebase.google.com/docs/data-connect/quickstart mode: markdown info: | Firebase Data Connect is a relational database service for mobile and web apps that lets you build and scale using a fully-managed PostgreSQL database powered by Cloud SQL. It provides a secure schema and query and mutation management using GraphQL that integrates well with Firebase Authentication. Data Connect includes SDK support for Kotlin Android and web. code: language: typescript file: external-assets/compass/genkit-dataconnect.ts region_tag: main_body - title: Defining your data connect query icons: - /external-assets/firebase-genkit.svg - /external-assets/firebase-connect.svg - /external-assets/places-api.svg links: - type: github url: https://github.com/FirebaseExtended/compass-ai-travel-planning-sample-flutter/blob/main/dataconnect/connector/queries.gql#L1-L12 - label: Learn more about vector similarity search url: https://firebase.google.com/docs/data-connect/solutions-vector-similarity-search mode: markdown info: | The getNearestPlaces() function is driven by a Firebase data connect query which is defined in the GQL format, here is its definition. Rather than using Google AI's embedding engine, you can use the Google Vertex AI extension for PostGRES SQL for embeddings generation and search. A production application would benefit from this database integration. code: language: typescript file: external-assets/compass/queries.gql - id: genkit-places title: Genkit & Places API examples: - title: Use tool calling to gather data from external sources icons: - /external-assets/firebase-genkit.svg links: - type: github url: https://github.com/FirebaseExtended/compass-ai-travel-planning-sample-flutter/blob/4cf8ea780ccee5e98bf19d1af94836c029d5df55/genkit/src/tools/restaurantFinder.ts#L26-L38 - label: View Gemini API Docs url: https://ai.google.dev/gemini-api/docs/function-calling - label: View Genkit docs url: https://firebase.google.com/docs/genkit/devtools mode: markdown info: | We use tool calling in Genkit to extend the agent’s functionality so that the agent can further refine responses and complete additional tasks. We start with defining the tools attributes as the first argument in defining a tool. The travel app defines a tool that can return restaurant information from the Places API based on the user's desired trip. The code uses Zod to define the input and output schema so that the query results can be validated. code: language: typescript file: external-assets/compass/genkit-places.ts - title: Calling the Places API from your code icons: - /external-assets/places-api.svg links: - type: github url: https://github.com/FirebaseExtended/compass-ai-travel-planning-sample-flutter/blob/main/genkit/src/tools/restaurantFinder.ts - label: View Places API docs url: https://developers.google.com/maps/documentation/places/web-service/overview mode: markdown info: | The second argument to the defineTool() call is the logic of what the tool actually does. Here we have the input object which is derived in the inputSchema from above and makes a call to the Places API to gather a list of suitable restaurants or grocery stores in the area. It returns an unfiltered list of Places to the model from this tool definition. code: language: javascript file: external-assets/compass/genkit-places-2.ts - id: genkit-gemini title: Genkit & Gemini examples: - title: Generate list of suggested trip overviews icons: - /external-assets/firebase-genkit.svg - /external-assets/flutter.svg mode: markdown info: | The travel app asks the user to define their dream trip by using text input or by tapping the microphone button to activate speech-to-text. The user can also optionally upload images. The app leverages a Dart package from pub.dev to integrate with native speech-to-text capabilities for each platform, and uses the Gemini API inside of Genkit to handle the multi-modal inputs such as images or videos. The Gemini API uses Retrieval-augmented generation (RAG) to return a set of suggested trips using Firebase Data Connect and embeddings to perform a nearest neighbor search. Flows are functions that are strongly typed, streamable, locally and remotely callable, and fully observable. Genkit provides a command-line interface and developer UI for working with flows, such as running or debugging flows. This code sample is an example of how this could be done. code: language: dart file: external-assets/compass/genkit-gemini.dart - title: Pre-generating the suggestions and images icons: - /external-assets/googlecloud.svg - /external-assets/vertex-ai.svg - /external-assets/gemini.svg mode: markdown links: - label: Learn more about Imagen url: https://cloud.google.com/vertex-ai/generative-ai/docs/image/overview info: | To simplify the demo, the suggested trip locations and images associated with those trips were pre-generated using a combination of Gemini and Imagen. This code sample is an example of how this could be done. code: language: typescript file: external-assets/compass/genkit-gemini-2.ts - id: flutter-genkit title: Flutter & Genkit examples: - title: Flutter communicates to Genkit icons: - /external-assets/flutter.svg mode: markdown links: - type: github url: https://github.com/FirebaseExtended/compass-ai-travel-planning-sample-flutter/blob/4cf8ea780ccee5e98bf19d1af94836c029d5df55/lib/ai/itineraries_feature/models/itinerary.dart#L57-L80 info: | You can use Genkit to enable your app to confidently consume data from LLMs by defining a schema that can validate the LLM's output. In Flutter, you can use Dart to serialize the request and deserialize the response to match the Genkit schema using standard HTTP requests. code: language: dart file: external-assets/compass/flutter-genkit-2.dart - title: Defining your schema with Dotprompt icons: - /external-assets/firebase-genkit.svg mode: markdown links: - label: View Flutter docs url: https://docs.flutter.dev/ - label: View Genkit docs url: https://firebase.google.com/docs/genkit info: | When working with generative AI, it's important to craft effective prompts so that the model returns high-quality responses. Genkit provides the Dotprompt plugin and text format to help you write and organize your generative AI prompts. The format encapsulates the prompt, input and output schema, model, and configuration all in a single file. The following code example shows a Dotprompt file used in the travel app. The schema is based on the information the user provides when they describe their dream trip. code: language: yaml file: external-assets/compass/flutter-genkit.ts - id: flutter-firebase title: Flutter & Firebase Hosting examples: - title: Deploying Flutter to Firebase Hosting Integration icons: - /external-assets/flutter.svg - /external-assets/firebase-hosting.svg mode: markdown links: - label: View Firebase launch checklist url: https://firebase.google.com/support/guides/launch-checklist - label: Integrate Flutter Web url: https://firebase.google.com/docs/hosting/frameworks/flutter info: | Firebase Hosting integrates with popular modern web frameworks including Flutter. Using Firebase Hosting and Cloud Functions for Firebase with these frameworks, you can develop apps and microservices in your preferred framework environment and then deploy them in a managed, secure server environment. Before going to production, understand the security and performance of all the services in your app. See the Firebase launch checklist for more information. code: language: typescript file: external-assets/compass/flutter-firebase.ts - type: quiz questions: - title: Which Firebase service is described as a relational database service that uses GraphQL for schema and query management and integrates with Firebase Authentication? answers: - answer: Firebase Hosting - answer: Firebase Cloud Functions - answer: Firebase Data Connect correct: true - answer: Firebase Places API - title: What technology is used in Genkit to validate the input and output schema for data received from the Places API? answers: - answer: Dart - answer: Zod correct: true - answer: GraphQL - answer: HTTP Requests - title: Which API does the travel app leverage within Genkit to handle multi-modal inputs like images or videos and uses RAG to provide suggested trips? answers: - answer: Gemini API correct: true - answer: Firebase Places API - answer: Firebase Hosting API - answer: Firebase Data Connect API - title: What tool within Genkit helps in writing and organizing generative AI prompts by encapsulating the prompt, schema, model, and configuration in a single file? answers: - answer: Zod - answer: Cloud Functions - answer: Dotprompt plugin correct: true - answer: GraphQL - type: build promoType: firebase-studio links: - url: https://github.com/FirebaseExtended/compass-ai-travel-planning-sample-flutter label: Open in GitHub - url: https://developers.google.com/solutions/catalog label: Explore more Solutions architecture: # zoom: 0.85 entities: - id: flutter label: Flutter icon: /external-assets/flutter.svg x: 0 y: 3 connections: - from: flutter to: firebase-genkit inspect: flutter-genkit - id: firebase-genkit icon: /external-assets/firebase-genkit.svg label: Genkit x: 2 y: 3 connections: - from: firebase-genkit to: google-ai inspect: genkit-gemini - from: firebase-genkit to: places-api inspect: genkit-places - from: firebase-genkit to: firebase-connect inspect: genkit-dataconnect - id: firebase-connect icon: /external-assets/firebase-connect.svg label: Firebase Data Connect x: 4 y: 5 connections: - from: firebase-connect to: cloud-sql - id: places-api icon: /external-assets/places-api.svg label: Places API x: 4 y: 3 - id: google-ai icon: /external-assets/vertex-ai.svg label: Google AI x: 4 y: 1 connections: - from: google-ai to: gemini - from: google-ai to: gecko - id: cloud-sql icon: /external-assets/cloud-sql.svg label: Cloud SQL x: 6 y: 5 - id: gecko icon: /external-assets/gecko.svg label: Embeddings x: 6 y: 3 - id: gemini icon: /external-assets/gemini.svg label: Google Gemini x: 6 y: 1 - id: firebase-hosting icon: /external-assets/firebase-hosting.svg label: Firebase Hosting x: 0 y: 5 connections: - from: firebase-hosting to: flutter inspect: flutter-firebase badges: startBadge: https://developers.google.com/profile/badges/playlists/solutions/compass/view exploreBadge: https://developers.google.com/profile/badges/playlists/solutions/compass/learn quizBadge: https://developers.google.com/profile/badges/playlists/solutions/compass/quiz buildBadge: https://developers.google.com/profile/badges/playlists/solutions/compass/action