{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ijGzTHJJUCPY"
      },
      "outputs": [],
      "source": [
        "# Copyright 2024 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VEqbX8OhE8y9"
      },
      "source": [
        "# Building and Deploying a Google Maps API Agent with Agent Engine\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/gemini/agent-engine/tutorial_google_maps_agent.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fgemini%2Fagent-engine%2Ftutorial_google_maps_agent.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Run in Colab Enterprise\n",
        "    </a>\n",
        "  </td>      \n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/agent-engine/tutorial_google_maps_agent.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/agent-engine/tutorial_google_maps_agent.ipynb\">\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br> Open in Vertex AI Workbench\n",
        "    </a>\n",
        "  </td>\n",
        "</table>\n",
        "\n",
        "<div style=\"clear: both;\"></div>\n",
        "\n",
        "<b>Share to:</b>\n",
        "\n",
        "<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/agent-engine/tutorial_google_maps_agent.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/agent-engine/tutorial_google_maps_agent.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/agent-engine/tutorial_google_maps_agent.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/agent-engine/tutorial_google_maps_agent.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/agent-engine/tutorial_google_maps_agent.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
        "</a>            \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "84e7e432e6ff"
      },
      "source": [
        "| | |\n",
        "|-|-|\n",
        "|Author(s) | [Kristopher Overholt](https://github.com/koverholt) |"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "CkHPv2myT2cx"
      },
      "source": [
        "## Overview\n",
        "\n",
        "### Gemini\n",
        "\n",
        "Gemini is a family of Generative AI models developed by Google DeepMind that is designed for multimodal use cases.\n",
        "\n",
        "### Function Calling in Gemini\n",
        "\n",
        "[Function calling](https://cloud.google.com/vertex-ai/docs/generative-ai/multimodal/function-calling) lets developers create a description of a function in their code, then pass that description to a language model in a request. The response from the model includes the name of a function that matches the description and the arguments to call it with.\n",
        "\n",
        "### Agent Engine in Vertex AI\n",
        "\n",
        "[Agent Engine](https://cloud.google.com/vertex-ai/generative-ai/docs/agent-engine/overview) is a managed service that helps you to build and deploy agent frameworks. It gives you the flexibility to choose how much reasoning you want to delegate to the LLM and how much you want to handle with customized code. You can define Python functions that get used as tools via [Gemini Function Calling](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling). Agent Engine integrates closely with the Python SDK for the Gemini model in Vertex AI, and it can manage prompts, agents, and examples in a modular way. Agent Engine is compatible with LangChain, LlamaIndex, or other Python frameworks."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DrkcqHrrwMAo"
      },
      "source": [
        "### Objectives\n",
        "\n",
        "In this tutorial, you will build and deploy an agent (model, tools, and reasoning) using the Vertex AI SDK for Python.\n",
        "\n",
        "You'll build and deploy an agent that uses different components of the Google Maps API to help with various tasks related to planning and building a community solar panel project:\n",
        "\n",
        "- Install the Vertex AI SDK for Python\n",
        "- Define a model for your agent\n",
        "- Define Python functions as tools so that our agent can:\n",
        "   - Geocode addresses to lat/lon coordinates using the Maps Geocoding API\n",
        "   - Get information about places using the Maps Places API\n",
        "   - Generate satellite map images using the Maps Static API\n",
        "   - Generate solar potential map images using the Maps Solar API\n",
        "- Use the LangChain agent template provided in the Vertex AI SDK for Agent Engine\n",
        "- Test your agent locally before deploying\n",
        "- Deploy and test your agent on Agent Engine in Vertex AI"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C9nEPojogw-g"
      },
      "source": [
        "### Enable APIs and Services\n",
        "\n",
        "This tutorial uses the following billable components of Google Cloud, which you'll need to enable for this tutorial:\n",
        "\n",
        "- [Enable Vertex AI API](https://console.cloud.google.com/apis/api/aiplatform.googleapis.com/overview)\n",
        "- [Enable Maps Geocoding API](https://console.cloud.google.com/apis/api/geocoding-backend.googleapis.com/overview)\n",
        "- [Enable Maps Places API](https://console.cloud.google.com/apis/api/places-backend.googleapis.com/overview)\n",
        "- [Enable Maps Static API](https://console.cloud.google.com/apis/api/static-maps-backend.googleapis.com/overview)\n",
        "- [Enable Maps Solar API](https://console.cloud.google.com/apis/api/solar.googleapis.com/overview)\n",
        "- [Enable Resource Manager API](https://console.cloud.google.com/apis/api/cloudresourcemanager.googleapis.com/overview)\n",
        "\n",
        "Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing) and use the [Pricing Calculator](https://cloud.google.com/products/calculator/) to generate a cost estimate based on your projected usage."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r11Gu7qNgx1p"
      },
      "source": [
        "## Getting Started\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "No17Cw5hgx12"
      },
      "source": [
        "### Install Vertex AI SDK for Python\n",
        "\n",
        "Install the latest version of the Vertex AI SDK for Python as well as extra dependencies related to Agent Engine and LangChain:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "tFy3H3aPgx12"
      },
      "outputs": [],
      "source": [
        "%pip install --upgrade --quiet \\\n",
        "    \"google-cloud-aiplatform[langchain,agent_engines]\" \\\n",
        "    cloudpickle==3.0.0 \\\n",
        "    \"pydantic>=2.10\" \\\n",
        "    googlemaps \\\n",
        "    google-cloud-storage \\\n",
        "    google-cloud-resource-manager \\\n",
        "    matplotlib \\\n",
        "    rasterio \\\n",
        "    requests"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R5Xep4W9lq-Z"
      },
      "source": [
        "### Restart current runtime\n",
        "\n",
        "To use the newly installed packages in this Jupyter runtime, you must restart the runtime. You can do this by running the cell below, which will restart the current kernel."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "XRvKdaPDTznN"
      },
      "outputs": [],
      "source": [
        "# Restart kernel after installs so that your environment can access the new packages\n",
        "import IPython\n",
        "\n",
        "app = IPython.Application.instance()\n",
        "app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SbmM4z7FOBpM"
      },
      "source": [
        "<div class=\"alert alert-block alert-warning\">\n",
        "<b>⚠️ The kernel is going to restart. Please wait until it is finished before continuing to the next step. ⚠️</b>\n",
        "</div>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dmWOrTJ3gx13"
      },
      "source": [
        "### Authenticate your notebook environment (Colab only)\n",
        "\n",
        "If you are running this notebook on Google Colab, run the following cell to authenticate your environment. This step is not required if you are using [Vertex AI Workbench](https://cloud.google.com/vertex-ai-workbench)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "NyKGtVQjgx13"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "    from google.colab import auth\n",
        "\n",
        "    auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DF4l8DTdWgPY"
      },
      "source": [
        "### Set Google Cloud project information and initialize Vertex AI SDK\n",
        "\n",
        "To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
        "\n",
        "Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "Nqwi-5ufWp_B"
      },
      "outputs": [],
      "source": [
        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
        "LOCATION = \"us-central1\"  # @param {type:\"string\"}\n",
        "STAGING_BUCKET = \"gs://[your-staging-bucket]\"  # @param {type:\"string\"}\n",
        "\n",
        "import vertexai\n",
        "\n",
        "vertexai.init(project=PROJECT_ID, location=LOCATION, staging_bucket=STAGING_BUCKET)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "750a7a24c514"
      },
      "source": [
        "## Example: Build and Deploy an Agent for Community Solar Project Planning\n",
        "\n",
        "In this tutorial, you'll build Python functions as tools that the Gemini model will use to help users plan for community solar panel projects.\n",
        "\n",
        "When designing a community solar project, you'll need to identify potential locations to build your solar project on, look up relevant government offices and suppliers to help you with the planning and implementation process, and review satellite images and solar potential of regions and buildings to find the optimal location to install your solar panels.\n",
        "\n",
        "Rather than doing all of this research work manually, it would be nice to build and deploy an agent powered by the Gemini model so to help us with these geocoding and mapping tasks along the way?\n",
        "\n",
        "Let's build an agent that does exactly that!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jXHfaVS66_01"
      },
      "source": [
        "### Import libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "lslYAvw37JGQ"
      },
      "outputs": [],
      "source": [
        "from IPython.display import Image, Markdown, display\n",
        "from google.cloud import resourcemanager_v3, storage\n",
        "import matplotlib.pyplot as plt\n",
        "from rasterio.io import MemoryFile\n",
        "from vertexai import agent_engines\n",
        "from vertexai.preview.reasoning_engines import LangchainAgent"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "275640d559e9"
      },
      "source": [
        "### Define generative model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "47f07c83a754"
      },
      "source": [
        "The first component of your agent involves the version of the generative model you want to use in your agent. Here you'll use the Gemini 2.0 model:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "921890fcb875"
      },
      "outputs": [],
      "source": [
        "model = \"gemini-2.0-flash\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a46b886f9232"
      },
      "source": [
        "### Input your Google Maps API key\n",
        "\n",
        "Once you've enable the relevant Google Maps APIs, you can [generate a Maps API key](https://developers.google.com/maps/documentation/geocoding/get-api-key) and paste it in the cell below.\n",
        "\n",
        "You'll use this Maps API key to work with the Maps Geocoding API, Maps Places API, and other Maps APIs using the functions in the next section:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "2da64a07a763"
      },
      "outputs": [],
      "source": [
        "MAPS_API_KEY = \"YOUR_MAPS_API_KEY\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "becf51ff0976"
      },
      "source": [
        "### Define Python functions as tools\n",
        "\n",
        "The second component of your agent involves Python functions as tools, which will enable the Gemini model to interact with external systems, databases, document stores, and other APIs so that the model can get the most up-to-date information or take action with those systems.\n",
        "\n",
        "In this example, you'll define four functions that work with different components of the Maps API to perform geocoding of addresses, search for places, and generate maps:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "796b5ed73b22"
      },
      "outputs": [],
      "source": [
        "def geocode_address(query: str):\n",
        "    \"\"\"Convert an address or location into latitude and longitude coordinates using the Google Maps Geocoding API\"\"\"\n",
        "    import googlemaps\n",
        "\n",
        "    gmaps = googlemaps.Client(key=MAPS_API_KEY)\n",
        "    response = gmaps.geocode(query)\n",
        "    return response[0][\"geometry\"][\"location\"]\n",
        "\n",
        "\n",
        "def search_places(query: str):\n",
        "    \"\"\"Search for places using the Google Maps Places API\"\"\"\n",
        "    import googlemaps\n",
        "\n",
        "    gmaps = googlemaps.Client(key=MAPS_API_KEY)\n",
        "    response = gmaps.places(query)\n",
        "    return response\n",
        "\n",
        "\n",
        "def create_satellite_map(location: str):\n",
        "    \"Create a satellite map of a specific location using the Google Maps Static API\"\n",
        "    from google.cloud import storage\n",
        "    import googlemaps\n",
        "\n",
        "    gmaps = googlemaps.Client(key=MAPS_API_KEY)\n",
        "    response = gmaps.static_map(\n",
        "        size=800,\n",
        "        maptype=\"hybrid\",\n",
        "        center=location,\n",
        "        zoom=18,\n",
        "    )\n",
        "    f = open(\"satellite_map.png\", \"wb\")\n",
        "    for chunk in response:\n",
        "        if chunk:\n",
        "            f.write(chunk)\n",
        "    f.close()\n",
        "\n",
        "    storage_client = storage.Client()\n",
        "    bucket = storage_client.bucket(STAGING_BUCKET.replace(\"gs://\", \"\"))\n",
        "    blob = bucket.blob(\"agent_engine/satellite_map.png\")\n",
        "    blob.upload_from_filename(\"satellite_map.png\")\n",
        "\n",
        "    return response\n",
        "\n",
        "\n",
        "def create_solar_potential_map(\n",
        "    latitude: str,\n",
        "    longitude: str,\n",
        "):\n",
        "    \"Get raw solar information as a TIFF image for an area surrounding a location using the Google Maps Solar API\"\n",
        "    from google.cloud import storage\n",
        "    import requests\n",
        "\n",
        "    response = requests.get(\n",
        "        \"https://solar.googleapis.com/v1/dataLayers:get\",\n",
        "        params={\n",
        "            \"location.latitude\": latitude,\n",
        "            \"location.longitude\": longitude,\n",
        "            \"radiusMeters\": 100,\n",
        "            \"view\": \"FULL_LAYERS\",\n",
        "            \"requiredQuality\": \"HIGH\",\n",
        "            \"pixelSizeMeters\": 0.5,\n",
        "            \"key\": MAPS_API_KEY,\n",
        "        },\n",
        "    )\n",
        "\n",
        "    solar_flux_url = response.json()[\"annualFluxUrl\"] + \"&key=\" + MAPS_API_KEY\n",
        "    response_solar_flux = requests.get(solar_flux_url)\n",
        "    with open(\"solar_flux_map.tiff\", \"wb\") as f:\n",
        "        f.write(response_solar_flux.content)\n",
        "\n",
        "    storage_client = storage.Client()\n",
        "    bucket = storage_client.bucket(STAGING_BUCKET.replace(\"gs://\", \"\"))\n",
        "    blob = bucket.blob(\"agent_engine/solar_flux_map.tiff\")\n",
        "    blob.upload_from_filename(\"solar_flux_map.tiff\")\n",
        "\n",
        "    return response"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0b473ac3af4e"
      },
      "source": [
        "Now you can test the above functions with sample inputs to ensure that they are working as expected:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "38112bbc905e"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'lat': 52.48211509999999, 'lng': 13.3891911}"
            ]
          },
          "execution_count": 9,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Test geocoding\n",
        "geocode_address(\"Tempelhof Airport Field in Berlin, Germany\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "174c92d8cf8e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "myParkplace - Parking at Tegel Airport Gewerbepark 30, 15745 Wildau, Germany\n",
            "Parkplätze 12101, Platz d. Luftbrücke, 12101 Berlin, Germany\n",
            "Easy Airport Parking Berlin Brandenburg Am Flughafen 1-5, 12529 Schönefeld, Germany\n",
            "[P] Parking Schoenefeld Airport - ESSO Thiem Waßmannsdorfer Ch 6, 12529 Schönefeld, Germany\n",
            "McParking Parkplatz Bohnsdorf Flughafen BER Berlin Brandenburg Gebrüder-Hirth-Straße 27, 12526 Berlin, Germany\n",
            "Dein Stellplatz - Parken Flughafen Berlin - Günstig Parken am BER Hufenweg 16, 12526 Berlin, Germany\n",
            "dam.Parkservice Schulzendorfer Str. 10, 12529 Schönefeld, Germany\n",
            "Parkplatz 10965 Berlin, Germany\n",
            "Tempelhofer Damm 120 Parking Tempelhofer Damm 120, 12099 Berlin, Germany\n",
            "123 Park & ​​Fly Schönefeld DE, Zeppelinring 21, 15749 Mittenwalde, Germany\n",
            "Parkhaus Tempelhofer Hafen Ordensmeisterstraße 1, 12099 Berlin, Germany\n",
            "BER Flughafen parken Mittelstraße 10, 12529 Schönefeld, Germany\n",
            "Airparks BER P2 Parkplatz Gebrüder-Hirth-Straße 27, 12526 Berlin, Germany\n",
            "Parktiger Flughafenparkplatz Berlin Gewerbepark 30, 15745 Wildau, Germany\n",
            "McParking Parkhaus Bohnsdorf Flughafen BER Berlin Brandenburg Gebrüder-Hirth-Straße 26, 12526 Berlin, Germany\n",
            "Einfach-gut-parken Diepenseer Str. 11, 12529 Schönefeld, Germany\n",
            "Parkplatz Baseball Spielfelder Tempelhofer Feld 12049 Berlin, Germany\n",
            "Parking prywatny Berlin lotnisko Kurzer Weg 3, 12529 Schönefeld, Germany\n",
            "THF Parkplatz Tempelhofer Damm 45, 12101 Berlin, Germany\n",
            "FlughafenTempelhof Columbiadamm 10, 10965 Berlin, Germany\n"
          ]
        }
      ],
      "source": [
        "# Test places search\n",
        "response = search_places(query=\"Parking lots near the Tempelhof Airport Field\")\n",
        "for result in response[\"results\"]:\n",
        "    print(result[\"name\"], result[\"formatted_address\"])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "03fa8b2cbb13"
      },
      "outputs": [
        {
          "data": {
            "image/png": "[encoded data, 378768 characters]",
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "execution_count": 11,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Test satellite map generation\n",
        "create_satellite_map(\"Tempelhof Airport Field\")\n",
        "\n",
        "# Retrieve and display the satellite image from your GCS bucket\n",
        "storage_client = storage.Client()\n",
        "bucket = storage_client.bucket(STAGING_BUCKET.replace(\"gs://\", \"\"))\n",
        "blob = bucket.blob(\"agent_engine/satellite_map.png\")\n",
        "image_content = blob.download_as_bytes()\n",
        "Image(image_content)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "479532d8cd48"
      },
      "outputs": [
        {
          "data": {
            "image/png": "[encoded data, 364996 characters]",
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# Test solar potential map\n",
        "create_solar_potential_map(latitude=\"52.482115\", longitude=\"13.389191\")\n",
        "\n",
        "# Retrieve and display the solar flux image from your GCS bucket\n",
        "storage_client = storage.Client()\n",
        "bucket = storage_client.bucket(STAGING_BUCKET.replace(\"gs://\", \"\"))\n",
        "blob = bucket.blob(\"agent_engine/solar_flux_map.tiff\")\n",
        "image_content = blob.download_as_bytes()\n",
        "with MemoryFile(image_content) as memfile:\n",
        "    with memfile.open() as dataset:\n",
        "        img = dataset.read()\n",
        "plt.imshow(img[0])\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "42820a8c31f2"
      },
      "source": [
        "### Define agent\n",
        "\n",
        "The third component of your agent involves adding a reasoning layer, which helps your agent use the tools that you provided to help the end user achieve a higher-level goal.\n",
        "\n",
        "If you were to use Gemini and Function Calling on their own without a reasoning layer, you would need to handle the process of calling functions and APIs in your application code, and you would need to implement retries and additional logic to ensure that your function calling code is resilient to failures and malformed requests.\n",
        "\n",
        "Here, you'll use the LangChain agent template provided in the Vertex AI SDK for Agent Engine, which brings together the model, tools, and reasoning that you've built up so far:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "c4be579b63dc"
      },
      "outputs": [],
      "source": [
        "agent = LangchainAgent(\n",
        "    model=model,\n",
        "    model_kwargs={\"temperature\": 0},\n",
        "    tools=[\n",
        "        geocode_address,\n",
        "        search_places,\n",
        "        create_satellite_map,\n",
        "        create_solar_potential_map,\n",
        "    ],\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d794214ba81e"
      },
      "source": [
        "### Test your agent locally\n",
        "\n",
        "Now you can test the model and agent behavior to ensure that it's working as expected before we deploy it:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "48843064771f"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "The closest government office to the Tokyo Big Sight Exhibition Center appears to be the **Tokyo Big Sight Information office**. It's located right on the premises, so it would be a convenient place to start. \n"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "response = agent.query(\n",
        "    input=\"\"\"I'd like to start a community effort to build a solar panel project\n",
        "    near the Tokyo Big Sight Exhibition Center. What are some nearby government\n",
        "    offices that might help me?\"\"\"\n",
        ")\n",
        "\n",
        "display(Markdown(response[\"output\"]))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "80671d3e1074"
      },
      "source": [
        "### Deploy your agent on Vertex AI\n",
        "\n",
        "Now that you've specified a model, tools, and reasoning for your agent and tested it out, you're ready to deploy your agent as a remote service in Vertex AI!\n",
        "\n",
        "You can re-define the agent to avoid any stateful information in the agent due to our testing in the previous cell:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "a79ed65e50d0"
      },
      "outputs": [],
      "source": [
        "agent = LangchainAgent(\n",
        "    model=model,\n",
        "    model_kwargs={\"temperature\": 0},\n",
        "    tools=[\n",
        "        geocode_address,\n",
        "        search_places,\n",
        "        create_satellite_map,\n",
        "        create_solar_potential_map,\n",
        "    ],\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e1027e9dab8f"
      },
      "source": [
        "Now you're ready to deploy your agent to Agent Engine in Vertex AI by calling `agent_engines.create()` along with the instance of your agent and the Python packages that your agent requires at runtime:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "659705cebbce"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Using bucket your-bucket-name\n",
            "Writing to gs://your-bucket-name/agent_engine/agent_engine.pkl\n",
            "Writing to gs://your-bucket-name/agent_engine/requirements.txt\n",
            "Creating in-memory tarfile of extra_packages\n",
            "Writing to gs://your-bucket-name/agent_engine/dependencies.tar.gz\n",
            "Creating AgentEngine\n",
            "Create AgentEngine backing LRO: projects/your-project-number/locations/us-central1/reasoningEngines/2553752094951604224/operations/2599704568526798848\n",
            "AgentEngine created. Resource name: projects/your-project-number/locations/us-central1/reasoningEngines/2553752094951604224\n",
            "To use this AgentEngine in another session:\n",
            "agent_engine = vertexai.agent_engines.get('projects/your-project-number/locations/us-central1/reasoningEngines/2553752094951604224')\n"
          ]
        }
      ],
      "source": [
        "remote_agent = agent_engines.create(\n",
        "    agent,\n",
        "    requirements=[\n",
        "        \"google-cloud-aiplatform[agent_engines,langchain]\",\n",
        "        \"cloudpickle==3.0.0\",\n",
        "        \"pydantic>=2.10\",\n",
        "        \"googlemaps\",\n",
        "        \"google-cloud-storage\",\n",
        "        \"rasterio\",\n",
        "        \"requests\",\n",
        "    ],\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3cf1e5491201"
      },
      "source": [
        "### Grant Storage Object User access to GCS bucket\n",
        "\n",
        "Before you send queries to your remote agent, you'll need to grant write permissions to your GCS bucket for the \"Storage Object User\" role so that the Agent Engine service can read and write image files in this tutorial to your bucket:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "id": "c7037edc160d"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<google.api_core.iam.Policy at 0x16a4fcbd0>"
            ]
          },
          "execution_count": 17,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Retrieve the project number associated with your project ID\n",
        "client = resourcemanager_v3.ProjectsClient()\n",
        "project = client.get_project(name=f\"projects/{PROJECT_ID}\")\n",
        "project_number = project.name.split(\"/\")[-1]  # Extract number from resource name\n",
        "\n",
        "# Grant the \"Storage Object User\" role on your staging bucket in GCS\n",
        "storage_client = storage.Client()\n",
        "bucket = storage_client.bucket(STAGING_BUCKET.replace(\"gs://\", \"\"))\n",
        "policy = bucket.get_iam_policy(requested_policy_version=3)\n",
        "policy.bindings.append(\n",
        "    {\n",
        "        \"role\": \"roles/storage.objectUser\",\n",
        "        \"members\": [\n",
        "            f\"serviceAccount:service-{project_number}@gcp-sa-aiplatform-re.iam.gserviceaccount.com\"\n",
        "        ],\n",
        "    }\n",
        ")\n",
        "bucket.set_iam_policy(policy)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "efee31ce8859"
      },
      "source": [
        "### Test your remotely deployed agent\n",
        "\n",
        "With all of the core components of your community solar planning agent in place, you can send prompts to your remotely deployed agent to perform different tasks and test that it's working as expected:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "id": "e9273f9b258a"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "The closest government office to the Tokyo Big Sight Exhibition Center appears to be the **Tokyo Big Sight Information office**. It looks like they're closed right now, but you can try going there when they open to see if they can help you with your solar panel project. \n"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "response = remote_agent.query(\n",
        "    input=\"\"\"I'd like to start a community effort to build a solar panel project\n",
        "    near the Tokyo Big Sight Exhibition Center. What are some nearby government\n",
        "    offices that might help me?\"\"\"\n",
        ")\n",
        "\n",
        "display(Markdown(response[\"output\"]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "id": "b80c4488d5b0"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "OK. I've created a static map centered around the Tokyo Big Sight Exhibition Center. Is there anything else I can help with?"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "response = remote_agent.query(\n",
        "    input=\"\"\"I'd like to start a community effort to build a solar panel project\n",
        "    near the Tokyo Big Sight Exhibition Center. Can you show me a satellite map\n",
        "    of the area?\"\"\"\n",
        ")\n",
        "\n",
        "display(Markdown(response[\"output\"]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "c6f612ef035e"
      },
      "outputs": [
        {
          "data": {
            "image/png": "[encoded data, 366660 characters]",
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "execution_count": 20,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Retrieve and display the satellite image from your GCS bucket\n",
        "bucket = storage_client.bucket(STAGING_BUCKET.replace(\"gs://\", \"\"))\n",
        "blob = bucket.blob(\"agent_engine/satellite_map.png\")\n",
        "image_content = blob.download_as_bytes()\n",
        "Image(image_content)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "22cf37070351"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": [
              "OK. I've generated a solar potential map for the area surrounding the Tokyo Big Sight Exhibition Center. The TIFF image is available at the URL in the response.\n"
            ],
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "response = remote_agent.query(\n",
        "    input=\"\"\"I'd like to start a community effort to build a solar panel project\n",
        "    near the Tokyo Big Sight Exhibition Center. Can you show me a map the solar\n",
        "    potential for this area?\"\"\"\n",
        ")\n",
        "\n",
        "display(Markdown(response[\"output\"]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "38c62541932c"
      },
      "outputs": [
        {
          "data": {
            "image/png": "[encoded data, 333052 characters]",
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# Retrieve and display the solar flux image from your GCS bucket\n",
        "bucket = storage_client.bucket(STAGING_BUCKET.replace(\"gs://\", \"\"))\n",
        "blob = bucket.blob(\"agent_engine/solar_flux_map.tiff\")\n",
        "image_content = blob.download_as_bytes()\n",
        "with MemoryFile(image_content) as memfile:\n",
        "    with memfile.open() as dataset:\n",
        "        img = dataset.read()\n",
        "plt.imshow(img[0])\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9f2f7d3ed7bd"
      },
      "source": [
        "### Querying your deployed agent\n",
        "\n",
        "You've now deployed your Agent Engine agent and can [interact with it in multiple ways](https://cloud.google.com/vertex-ai/generative-ai/docs/agent-engine/use), both within this notebook and from other applications or environments. The primary methods for accessing your deployed agent are via the Python client library or through REST API calls. Here's an overview of both methods:\n",
        "\n",
        "**Method 1: Reusing within this notebook or another Python environment**\n",
        "\n",
        "You can directly reuse and query the `remote_agent` instance you created in this notebook.\n",
        "\n",
        "Or, you can instantiate a new instance in another notebook or Python script. To do this, you'll need to retrieve your deployed agent's resource name that uniquely identifies your agent, which is a string that includes the project, location, and Agent Engine ID. You can retrieve it by running the following code in the notebook or environment where you created your agent:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "id": "fdaf8b91413f"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "'projects/your-project-number/locations/us-central1/reasoningEngines/223106848019812424'"
            ]
          },
          "execution_count": 23,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "remote_agent.resource_name"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "060f8369d113"
      },
      "source": [
        "Use the resource name to load the agent in your other notebook or Python script, then query the remote agent as usual:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "id": "78af4442827e"
      },
      "outputs": [],
      "source": [
        "# from vertexai.preview import agent_engines\n",
        "\n",
        "# AGENT_ENGINE_RESOURCE_NAME = \"YOUR_AGENT_ENGINE_RESOURCE_NAME\"  # Replace with the resource name of your deployed Agent Engine\n",
        "\n",
        "# remote_agent = agent_engines.get(AGENT_ENGINE_RESOURCE_NAME)\n",
        "# response = remote_agent.query(input=query)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "63ab06554fc0"
      },
      "source": [
        "**Method 2: Accessing from other environments via REST API**\n",
        "\n",
        "Beyond the Python client library, your deployed Vertex AI agent can be [queried using REST API calls](https://cloud.google.com/vertex-ai/generative-ai/docs/agent-engine/use), including:\n",
        "\n",
        "- Python: You can use Python's `requests` library or similar tools to make HTTP calls to the Vertex AI REST API.\n",
        "- cURL: A command-line tool, cURL allows you to send HTTP requests directly. This is useful for testing and debugging.\n",
        "- Other Programming Languages: If you prefer a different language for your application, you can use its native HTTP client library to make REST API calls.\n",
        "\n",
        "In summary, you have access to your deployed Agent Engine agent through the Python client library within Python environments, and more universally through its REST API via tools and programming languages of your choosing."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8e193ac8130f"
      },
      "source": [
        "## Cleaning up\n",
        "\n",
        "After you've finished experimenting, it's a good practice to clean up your cloud resources. You can delete the deployed Agent Engine instance to avoid any unexpected charges on your Google Cloud account."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "id": "95b1caa1ff1d"
      },
      "outputs": [],
      "source": [
        "remote_agent.delete()"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "tutorial_google_maps_agent.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
