solution:
metadata:
id: compass
title: Build an agent-powered travel planning app with Generative AI
description: Learn how to use Flutter and Genkit to build multi-platform apps that can seamlessly integrate with AI.
fallbackCta:
title: Launch in Firebase Studio
icon: firebase-studio
url: https://idx.google.com/import?url=https%3A%2F%2Fgithub.com%2FFirebaseExtended%2Fcompass-ai-travel-planning-sample-flutter
accentColor:
foreground: "white"
background: "#141e2a"
sections:
- type: watch
youtubeId: vLHwDTNfxdE
title: Build an agent-powered travel planning app with Generative AI
description: Learn how to use Flutter and Genkit to build multi-platform apps that can seamlessly integrate with AI.
technologies:
- icon: /external-assets/flutter.svg
label: Flutter
url: https://flutter.dev
- icon: /external-assets/firebase.svg
label: Firebase
url: https://firebase.google.com/
- icon: /external-assets/vertex-ai.svg
label: Google AI
url: https://cloud.google.com/vertex-ai
- icon: /external-assets/gemini.svg
label: Gemini
url: https://ai.google.dev/
- icon: /external-assets/places-api.svg
label: Places API
url: https://developers.google.com/maps/documentation/places/web-service
- type: explore
mode: video
subsections:
- id: video-1
title: Querying for pre-generated suggestions and image data
description: |
As the user initiates a search, the app queries a pre-generated list of suggestions and image data. The images were generated ahead of time using the Imagen API.
videoPath: external-assets/compass/compass-1.mp4
orientation: landscape
frame: laptop
layout: sidebyside
logs:
- timestamp: 7000
summary: Flutter app queries the pre-generated list of suggestions
detail: The suggestions and imagery have been generated ahead of time with Gemini and Imagen.
imageData: ""
sequence:
- flutter
- firebase-genkit
- places-api
inspect: genkit-gemini
- timestamp: 14000
summary: Flutter retrieves pre-generated suggestions and images
detail: Images were generated ahead of time using Imagen.
imageData: /external-assets/compass/temple.png
sequence:
- flutter
- firebase-genkit
- google-ai
- gemini
inspect: genkit-places
- id: video-2
title: Sending text and images to Gemini for inspiration
description: |
Our app's Flutter code sends a multimodal prompt of text and image data to Gemini via Genkit and Vertex AI to generate travel suggestions.
videoPath: external-assets/compass/compass-2.mp4
orientation: portrait
frame: androidphone
layout: sidebyside
logs:
- timestamp: 10000
summary: Flutter sends text and image prompt data to Gemini via Genkit and Vertex AI
detail: Prompt text "Traveling in China to see Shanghai and Sichuan"
imageData: /external-assets/compass/buddha.jpeg
# file: restaurantFinder.ts
sequence:
- flutter
- firebase-genkit
- google-ai
- gemini
inspect: flutter-genkit
- timestamp: 15000
summary: Flutter retrieves pre-generated suggestions and images
detail: Images were generated ahead of time using Imagen.
imageData: /external-assets/compass/lanterns.jpeg
file: restaurantFinder.ts
sequence:
- flutter
- firebase-genkit
- vertex-ai
- gemini
inspect: genkit-places
- type: inspect
subsections:
- id: genkit-dataconnect
title: Genkit & Data Connect API
examples:
- title: Connect your database with Firebase Data Connect
icons:
- /external-assets/firebase-genkit.svg
- /external-assets/firebase-connect.svg
links:
- label: View Data Connect quickstart
url: https://firebase.google.com/docs/data-connect/quickstart
mode: markdown
info: |
Firebase Data Connect is a relational database service for mobile and web apps that lets you build and scale using a fully-managed PostgreSQL database powered by Cloud SQL. It provides a secure schema and query and mutation management using GraphQL that integrates well with Firebase Authentication. Data Connect includes SDK support for Kotlin Android and web.
code:
language: typescript
file: external-assets/compass/genkit-dataconnect.ts
region_tag: main_body
- title: Defining your data connect query
icons:
- /external-assets/firebase-genkit.svg
- /external-assets/firebase-connect.svg
- /external-assets/places-api.svg
links:
- type: github
url: https://github.com/FirebaseExtended/compass-ai-travel-planning-sample-flutter/blob/main/dataconnect/connector/queries.gql#L1-L12
- label: Learn more about vector similarity search
url: https://firebase.google.com/docs/data-connect/solutions-vector-similarity-search
mode: markdown
info: |
The getNearestPlaces() function is driven by a Firebase data connect query which is defined in the GQL format, here is its definition.
Rather than using Google AI's embedding engine, you can use the Google Vertex AI extension for PostGRES SQL for embeddings generation and search. A production application would benefit from this database integration.
code:
language: typescript
file: external-assets/compass/queries.gql
- id: genkit-places
title: Genkit & Places API
examples:
- title: Use tool calling to gather data from external sources
icons:
- /external-assets/firebase-genkit.svg
links:
- type: github
url: https://github.com/FirebaseExtended/compass-ai-travel-planning-sample-flutter/blob/4cf8ea780ccee5e98bf19d1af94836c029d5df55/genkit/src/tools/restaurantFinder.ts#L26-L38
- label: View Gemini API Docs
url: https://ai.google.dev/gemini-api/docs/function-calling
- label: View Genkit docs
url: https://firebase.google.com/docs/genkit/devtools
mode: markdown
info: |
We use tool calling in Genkit to extend the agentโs functionality so that the agent can further refine responses and complete additional tasks. We start with defining the tools attributes as the first argument in defining a tool. The travel app defines a tool that can return restaurant information from the Places API based on the user's desired trip. The code uses Zod to define the input and output schema so that the query results can be validated.
code:
language: typescript
file: external-assets/compass/genkit-places.ts
- title: Calling the Places API from your code
icons:
- /external-assets/places-api.svg
links:
- type: github
url: https://github.com/FirebaseExtended/compass-ai-travel-planning-sample-flutter/blob/main/genkit/src/tools/restaurantFinder.ts
- label: View Places API docs
url: https://developers.google.com/maps/documentation/places/web-service/overview
mode: markdown
info: |
The second argument to the defineTool() call is the logic of what the tool actually does. Here we have the input object which is derived in the inputSchema from above and makes a call to the Places API to gather a list of suitable restaurants or grocery stores in the area. It returns an unfiltered list of Places to the model from this tool definition.
code:
language: javascript
file: external-assets/compass/genkit-places-2.ts
- id: genkit-gemini
title: Genkit & Gemini
examples:
- title: Generate list of suggested trip overviews
icons:
- /external-assets/firebase-genkit.svg
- /external-assets/flutter.svg
mode: markdown
info: |
The travel app asks the user to define their dream trip by using text input or by tapping the microphone button to activate speech-to-text. The user can also optionally upload images.
The app leverages a Dart package from pub.dev to integrate with native speech-to-text capabilities for each platform, and uses the Gemini API inside of Genkit to handle the multi-modal inputs such as images or videos. The Gemini API uses Retrieval-augmented generation (RAG) to return a set of suggested trips using Firebase Data Connect and embeddings to perform a nearest neighbor search.
Flows are functions that are strongly typed, streamable, locally and remotely callable, and fully observable. Genkit provides a command-line interface and developer UI for working with flows, such as running or debugging flows.
This code sample is an example of how this could be done.
code:
language: dart
file: external-assets/compass/genkit-gemini.dart
- title: Pre-generating the suggestions and images
icons:
- /external-assets/googlecloud.svg
- /external-assets/vertex-ai.svg
- /external-assets/gemini.svg
mode: markdown
links:
- label: Learn more about Imagen
url: https://cloud.google.com/vertex-ai/generative-ai/docs/image/overview
info: |
To simplify the demo, the suggested trip locations and images associated with those trips were pre-generated using a combination of Gemini and Imagen.
This code sample is an example of how this could be done.
code:
language: typescript
file: external-assets/compass/genkit-gemini-2.ts
- id: flutter-genkit
title: Flutter & Genkit
examples:
- title: Flutter communicates to Genkit
icons:
- /external-assets/flutter.svg
mode: markdown
links:
- type: github
url: https://github.com/FirebaseExtended/compass-ai-travel-planning-sample-flutter/blob/4cf8ea780ccee5e98bf19d1af94836c029d5df55/lib/ai/itineraries_feature/models/itinerary.dart#L57-L80
info: |
You can use Genkit to enable your app to confidently consume data from LLMs by defining a schema that can validate the LLM's output. In Flutter, you can use Dart to serialize the request and deserialize the response to match the Genkit schema using standard HTTP requests.
code:
language: dart
file: external-assets/compass/flutter-genkit-2.dart
- title: Defining your schema with Dotprompt
icons:
- /external-assets/firebase-genkit.svg
mode: markdown
links:
- label: View Flutter docs
url: https://docs.flutter.dev/
- label: View Genkit docs
url: https://firebase.google.com/docs/genkit
info: |
When working with generative AI, it's important to craft effective prompts so that the model returns high-quality responses. Genkit provides the Dotprompt plugin and text format to help you write and organize your generative AI prompts. The format encapsulates the prompt, input and output schema, model, and configuration all in a single file.
The following code example shows a Dotprompt file used in the travel app. The schema is based on the information the user provides when they describe their dream trip.
code:
language: yaml
file: external-assets/compass/flutter-genkit.ts
- id: flutter-firebase
title: Flutter & Firebase Hosting
examples:
- title: Deploying Flutter to Firebase Hosting Integration
icons:
- /external-assets/flutter.svg
- /external-assets/firebase-hosting.svg
mode: markdown
links:
- label: View Firebase launch checklist
url: https://firebase.google.com/support/guides/launch-checklist
- label: Integrate Flutter Web
url: https://firebase.google.com/docs/hosting/frameworks/flutter
info: |
Firebase Hosting integrates with popular modern web frameworks including Flutter. Using Firebase Hosting and Cloud Functions for Firebase with these frameworks, you can develop apps and microservices in your preferred framework environment and then deploy them in a managed, secure server environment. Before going to production, understand the security and performance of all the services in your app. See the Firebase launch checklist for more information.
code:
language: typescript
file: external-assets/compass/flutter-firebase.ts
- type: quiz
questions:
- title: Which Firebase service is described as a relational database service that uses GraphQL for schema and query management and integrates with Firebase Authentication?
answers:
- answer: Firebase Hosting
- answer: Firebase Cloud Functions
- answer: Firebase Data Connect
correct: true
- answer: Firebase Places API
- title: What technology is used in Genkit to validate the input and output schema for data received from the Places API?
answers:
- answer: Dart
- answer: Zod
correct: true
- answer: GraphQL
- answer: HTTP Requests
- title: Which API does the travel app leverage within Genkit to handle multi-modal inputs like images or videos and uses RAG to provide suggested trips?
answers:
- answer: Gemini API
correct: true
- answer: Firebase Places API
- answer: Firebase Hosting API
- answer: Firebase Data Connect API
- title: What tool within Genkit helps in writing and organizing generative AI prompts by encapsulating the prompt, schema, model, and configuration in a single file?
answers:
- answer: Zod
- answer: Cloud Functions
- answer: Dotprompt plugin
correct: true
- answer: GraphQL
- type: build
promoType: firebase-studio
links:
- url: https://github.com/FirebaseExtended/compass-ai-travel-planning-sample-flutter
label: Open in GitHub
- url: https://developers.google.com/solutions/catalog
label: Explore more Solutions
architecture:
# zoom: 0.85
entities:
- id: flutter
label: Flutter
icon: /external-assets/flutter.svg
x: 0
y: 3
connections:
- from: flutter
to: firebase-genkit
inspect: flutter-genkit
- id: firebase-genkit
icon: /external-assets/firebase-genkit.svg
label: Genkit
x: 2
y: 3
connections:
- from: firebase-genkit
to: google-ai
inspect: genkit-gemini
- from: firebase-genkit
to: places-api
inspect: genkit-places
- from: firebase-genkit
to: firebase-connect
inspect: genkit-dataconnect
- id: firebase-connect
icon: /external-assets/firebase-connect.svg
label: Firebase Data Connect
x: 4
y: 5
connections:
- from: firebase-connect
to: cloud-sql
- id: places-api
icon: /external-assets/places-api.svg
label: Places API
x: 4
y: 3
- id: google-ai
icon: /external-assets/vertex-ai.svg
label: Google AI
x: 4
y: 1
connections:
- from: google-ai
to: gemini
- from: google-ai
to: gecko
- id: cloud-sql
icon: /external-assets/cloud-sql.svg
label: Cloud SQL
x: 6
y: 5
- id: gecko
icon: /external-assets/gecko.svg
label: Embeddings
x: 6
y: 3
- id: gemini
icon: /external-assets/gemini.svg
label: Google Gemini
x: 6
y: 1
- id: firebase-hosting
icon: /external-assets/firebase-hosting.svg
label: Firebase Hosting
x: 0
y: 5
connections:
- from: firebase-hosting
to: flutter
inspect: flutter-firebase
badges:
startBadge: https://developers.google.com/profile/badges/playlists/solutions/compass/view
exploreBadge: https://developers.google.com/profile/badges/playlists/solutions/compass/learn
quizBadge: https://developers.google.com/profile/badges/playlists/solutions/compass/quiz
buildBadge: https://developers.google.com/profile/badges/playlists/solutions/compass/action