{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "dRtCRT8gJxPf"
      },
      "outputs": [],
      "source": [
        "# Copyright 2025 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZPC2X_a9ErW7"
      },
      "source": [
        "# Gemini 2.0 Flash Image Generation in Vertex AI with REST API\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_0_image_gen_rest_api.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fgemini%2Fgetting-started%2Fintro_gemini_2_0_image_gen_rest_api.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/gemini/getting-started/intro_gemini_2_0_image_gen_rest_api.ipynb\">\n",
        "      <img src=\"https://www.gstatic.com/images/branding/gcpiconscolors/vertexai/v1/32px.svg\" alt=\"Vertex AI logo\"><br> Open in Vertex AI Workbench\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_0_image_gen_rest_api.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "</table>\n",
        "\n",
        "<div style=\"clear: both;\"></div>\n",
        "\n",
        "<b>Share to:</b>\n",
        "\n",
        "<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_0_image_gen_rest_api.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_0_image_gen_rest_api.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_0_image_gen_rest_api.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_0_image_gen_rest_api.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_2_0_image_gen_rest_api.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
        "</a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f0cc0f48513b"
      },
      "source": [
        "| Author(s) |\n",
        "| --- |\n",
        "| [Nikita Namjoshi](https://github.com/nikitamaia) |\n",
        "| [Katie Nguyen](https://github.com/katiemn) |"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "axauUzNXEl_R"
      },
      "source": [
        "## Overview\n",
        "\n",
        "Gemini 2.0 Flash supports image generation and editing. This enables you to converse with Gemini and create images with interwoven text.\n",
        "\n",
        "In this tutorial, you'll learn how to use Gemini 2.0 Flash's image generation features in Vertex AI using the REST API.\n",
        "\n",
        "You'll try out the following scenarios:\n",
        "* Image generation:\n",
        "  * Text to image\n",
        "  * Text to image and text (interleaved)\n",
        "* Image editing:\n",
        "  * Text and image to image\n",
        "  * Multi-turn image editing\n",
        "  * Images and text to image and text (interleaved)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D50ekWXjEl_S"
      },
      "source": [
        "## Get started"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jLJQdbgSbb4M"
      },
      "source": [
        "### Install required libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "SQ0qcEWuXNXs"
      },
      "outputs": [],
      "source": [
        "%%capture\n",
        "\n",
        "!sudo apt install -q jq"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dmWOrTJ3gx13"
      },
      "source": [
        "### Authenticate your notebook environment (Colab only)\n",
        "\n",
        "If you are running this notebook on Google Colab, run the following cell to authenticate your environment."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "NyKGtVQjgx13"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "    from google.colab import auth\n",
        "\n",
        "    auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TH9d_9D-UmIX"
      },
      "source": [
        "### Import libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 42,
      "metadata": {
        "id": "QI3Hak-wnTQ3"
      },
      "outputs": [],
      "source": [
        "import base64\n",
        "import json\n",
        "\n",
        "from IPython.display import Image, Markdown, display"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O6ZGaZlxP9L0"
      },
      "source": [
        "### Set Google Cloud project information\n",
        "\n",
        "To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
        "\n",
        "Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "u8IivOG5SqY6"
      },
      "outputs": [],
      "source": [
        "# Use the environment variable if the user doesn't provide Project ID.\n",
        "import os\n",
        "\n",
        "PROJECT_ID = \"[your-project-id]\"  # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
        "if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
        "    PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
        "\n",
        "LOCATION = \"global\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "854fbf388e2b"
      },
      "source": [
        "### Load the image model\n",
        "\n",
        "Gemini 2.0 Flash image generation: `gemini-2.0-flash-preview-image-generation`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "7eeb063ac6d4"
      },
      "outputs": [],
      "source": [
        "MODEL_ID = \"gemini-2.0-flash-preview-image-generation\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8s84m7h6HSTR"
      },
      "source": [
        "### Defining environment variables for cURL commands\n",
        "\n",
        "These environment variables are used to construct the cURL commands."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "krJ8UOHKoPn3"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "os.environ[\"PROJECT_ID\"] = PROJECT_ID\n",
        "os.environ[\"LOCATION\"] = LOCATION\n",
        "\n",
        "API_HOST = \"aiplatform.googleapis.com\"\n",
        "os.environ[\"API_ENDPOINT\"] = (\n",
        "    f\"{API_HOST}/v1/projects/{PROJECT_ID}/locations/{LOCATION}/publishers/google/models/{MODEL_ID}\"\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xgOucVQlVR4t"
      },
      "source": [
        "## Image generation\n",
        "\n",
        "First, send a text prompt to Gemini 2.0 Flash describing the image you want to generate.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MmA_RAbFwED4"
      },
      "source": [
        "### Text to image\n",
        "\n",
        "In the curl command below, you'll see that the payload includes the following keys:\n",
        "\n",
        "* `contents`: this is your prompt, in this case a text only user message\n",
        "*`generation_config`: this dictionary specifies the desired output modalities, in this case `TEXT` and `IMAGE`. If you do not specify `IMAGE`, you will not get image output and `IMAGE` only is not allowed\n",
        "* `safetySettings`: select your options from the categories below:\n",
        "    * `method`: HARM_BLOCK_METHOD_UNSPECIFIED, SEVERITY, PROBABILITY\n",
        "    * `category`: HARM_CATEGORY_UNSPECIFIED, HARM_CATEGORY_HATE_SPEECH, HARM_CATEGORY_DANGEROUS_CONTENT, HARM_CATEGORY_HARASSMENT, HARM_CATEGORY_SEXUALLY_EXPLICIT, HARM_CATEGORY_CIVIC_INTEGRITY\n",
        "    * `threshold`: HARM_BLOCK_THRESHOLD_UNSPECIFIED, BLOCK_LOW_AND_ABOVE, BLOCK_MEDIUM_AND_ABOVE, BLOCK_ONLY_HIGH, BLOCK_NONE, OFF\n",
        "\n",
        "The cell below writes the output of running the curl command to the file `response.json`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "metadata": {
        "id": "FOynZhO6u3lf"
      },
      "outputs": [],
      "source": [
        "%%bash\n",
        "\n",
        "curl -X POST \\\n",
        "  -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
        "  -H \"Content-Type: application/json\" \\\n",
        "  https://${API_ENDPOINT}:generateContent \\\n",
        "  -d '{\n",
        "    \"contents\": {\n",
        "      \"role\": \"USER\",\n",
        "      \"parts\": { \"text\": \"generate an image of a penguin driving a taxi in New York City\"},\n",
        "    },\n",
        "    \"generation_config\": {\n",
        "      \"response_modalities\": [\"TEXT\", \"IMAGE\"],\n",
        "    },\n",
        "    \"safetySettings\": {\n",
        "      \"method\": \"PROBABILITY\",\n",
        "      \"category\": \"HARM_CATEGORY_DANGEROUS_CONTENT\",\n",
        "      \"threshold\": \"BLOCK_MEDIUM_AND_ABOVE\"\n",
        "    },\n",
        "  }' 2>/dev/null >response.json"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xDXZgcTdw-RT"
      },
      "source": [
        "Let's examine the output in the `response.json` file.\n",
        "\n",
        "In `content` you can see the model has created an `image/png` which is the b64 encoded value to the `data` key."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 39,
      "metadata": {
        "id": "yki65L0KxE6a"
      },
      "outputs": [],
      "source": [
        "!cat response.json"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GysKM62jeSFi"
      },
      "source": [
        "Next, load in the data from the `response.json` file so it's easier to work with in Python."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 40,
      "metadata": {
        "id": "4BdxCWrRZnCk"
      },
      "outputs": [],
      "source": [
        "with open(\"response.json\") as f:\n",
        "    response_data = json.load(f)\n",
        "    print(response_data)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mAjVosqHec6F"
      },
      "source": [
        "Extract the image data from the response and visualize. All generated images include a [SynthID watermark](https://deepmind.google/technologies/synthid/), which can be verified via the Media Studio in [Vertex AI Studio](https://cloud.google.com/generative-ai-studio?hl=en)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 41,
      "metadata": {
        "id": "I6JAcTWdZqlI"
      },
      "outputs": [],
      "source": [
        "image_part = next(\n",
        "    filter(\n",
        "        lambda x: \"inlineData\" in x,\n",
        "        response_data[\"candidates\"][0][\"content\"][\"parts\"],\n",
        "    )\n",
        ")\n",
        "\n",
        "image_data = base64.b64decode(image_part[\"inlineData\"][\"data\"])\n",
        "display(Image(data=image_data, width=350, height=350))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5l4YLy8Vq_-v"
      },
      "source": [
        "### Text to image and text\n",
        "\n",
        "In addition to generating images, Gemini can generate multiple images and text in an interleaved fashion.\n",
        "\n",
        "For example, you could ask the model to generate a recipe for banana bread with images showing different stages of the cooking process. Or, you could ask the model to generate images of different wildflowers with accompanying titles and descriptions.\n",
        "\n",
        "Let's try out the interleaved text and image functionality by prompting Gemini 2.0 Flash to create a tutorial for assemblying a peanut butter and jelly sandwich.\n",
        "\n",
        "You'll notice that in the prompt we ask the model to generate both text and images. This will nudge the model to create text with images interleaved."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WdkrXmyIFtgv"
      },
      "source": [
        "⚠️ **Note:** we are asking the model to generate a lot of content in this prompt, so it will take a bit of time for this cell to finish executing."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "LgoqgDCVVR4u"
      },
      "outputs": [],
      "source": [
        "%%bash\n",
        "\n",
        "curl -X POST \\\n",
        "  -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
        "  -H \"Content-Type: application/json\" \\\n",
        "  https://${API_ENDPOINT}:generateContent \\\n",
        "  -d '{\n",
        "    \"contents\": {\n",
        "      \"role\": \"USER\",\n",
        "      \"parts\": { \"text\": \"Create a tutorial explaining how to make a peanut butter and jelly sandwich in three easy steps. For each step, provide a title with the number of the step, an explanation, and also generate an image, generate each image in a 1:1 aspect ratio.\"},\n",
        "    },\n",
        "    \"generation_config\": {\n",
        "      \"response_modalities\": [\"TEXT\", \"IMAGE\"],\n",
        "     },\n",
        "     \"safetySettings\": {\n",
        "      \"method\": \"PROBABILITY\",\n",
        "      \"category\": \"HARM_CATEGORY_DANGEROUS_CONTENT\",\n",
        "      \"threshold\": \"BLOCK_MEDIUM_AND_ABOVE\"\n",
        "    },\n",
        "  }' 2>/dev/null >response.json"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WvbxLZLqy2Kp"
      },
      "source": [
        "Let's visualize the response."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iUgEbabPHrVh"
      },
      "outputs": [],
      "source": [
        "with open(\"response.json\") as f:\n",
        "    response_data = json.load(f)\n",
        "\n",
        "for part in response_data[\"candidates\"][0][\"content\"][\"parts\"]:\n",
        "    if \"text\" in part.keys():\n",
        "        display(Markdown(part[\"text\"]))\n",
        "    if \"inlineData\" in part.keys():\n",
        "        content = part[\"inlineData\"][\"data\"]\n",
        "        image_data = base64.b64decode(content)\n",
        "        display(Image(data=image_data, width=350, height=350))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R3g5n23lDtsN"
      },
      "source": [
        "## Image editing\n",
        "\n",
        "You can pass text and an image to Gemini 2.0 Flash for use cases like product captions, information about a particular image, or to make edits or modifications to an existing image.\n",
        "\n",
        "### Text and image to image\n",
        "\n",
        "Let's try out a style transfer example and ask Gemini 2.0 Flash to create an image of this dog in a 3D cartoon rendering."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SPLCmWPXaZjA"
      },
      "source": [
        "Visualize the starting dog image by running this next cell."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "yuTg_ZGCn95N"
      },
      "outputs": [],
      "source": [
        "image_url = (\n",
        "    \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/dog-1.jpg\"\n",
        ")\n",
        "display(Image(url=image_url, width=350, height=350))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 53,
      "metadata": {
        "id": "UAiUyW1VMzPn"
      },
      "outputs": [],
      "source": [
        "%%bash\n",
        "\n",
        "curl -X POST \\\n",
        "  -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
        "  -H \"Content-Type: application/json\" \\\n",
        "  https://${API_ENDPOINT}:generateContent \\\n",
        "  -d '{\n",
        "    \"contents\": {\n",
        "      \"role\": \"USER\",\n",
        "      \"parts\": [\n",
        "        {\"file_data\": {\n",
        "          \"mime_type\": \"image/jpg\",\n",
        "          \"file_uri\": \"gs://cloud-samples-data/generative-ai/image/dog-1.jpg\"\n",
        "          }\n",
        "        },\n",
        "        {\"text\": \"Create a 3D cartoon style portrait of this dog, include rounded, exaggerated facial features, saturated colors, and realistic-looking textures. The dog is wearing a cowboy hat.\"},\n",
        "      ]\n",
        "\n",
        "    },\n",
        "    \"generation_config\": {\n",
        "      \"response_modalities\": [\"TEXT\", \"IMAGE\"],\n",
        "    },\n",
        "    \"safetySettings\": {\n",
        "      \"method\": \"PROBABILITY\",\n",
        "      \"category\": \"HARM_CATEGORY_DANGEROUS_CONTENT\",\n",
        "      \"threshold\": \"BLOCK_MEDIUM_AND_ABOVE\"\n",
        "    },\n",
        "  }' 2>/dev/null >response.json"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "53Q1jGTIf8oS"
      },
      "source": [
        "Extract the image data from the response and visualize."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4UuYoh92KJSA"
      },
      "outputs": [],
      "source": [
        "with open(\"response.json\") as f:\n",
        "    response_data = json.load(f)\n",
        "\n",
        "image_part = next(\n",
        "    filter(\n",
        "        lambda x: \"inlineData\" in x,\n",
        "        response_data[\"candidates\"][0][\"content\"][\"parts\"],\n",
        "    )\n",
        ")\n",
        "\n",
        "image_data = base64.b64decode(image_part[\"inlineData\"][\"data\"])\n",
        "display(Image(data=image_data, width=350, height=350))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SlGAQflXXL2w"
      },
      "source": [
        "### Multi-turn image editing\n",
        "\n",
        "In this next section, you supply a starting image and iteratively alter certain aspects of the image though chatting with Gemini 2.0 Flash.\n",
        "\n",
        "Visualize the starting image of a vase that's stored in Google Cloud Storage by running this next cell."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mLRjMqLnpVkU"
      },
      "outputs": [],
      "source": [
        "image_url = (\n",
        "    \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/vase.png\"\n",
        ")\n",
        "display(Image(url=image_url, width=350, height=350))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 56,
      "metadata": {
        "id": "q1n-AD82pak6"
      },
      "outputs": [],
      "source": [
        "%%bash\n",
        "\n",
        "curl -X POST \\\n",
        "  -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
        "  -H \"Content-Type: application/json\" \\\n",
        "  https://${API_ENDPOINT}:generateContent \\\n",
        "  -d '{\n",
        "    \"contents\": {\n",
        "      \"role\": \"USER\",\n",
        "      \"parts\": [\n",
        "        {\"file_data\": {\n",
        "          \"mime_type\": \"image/png\",\n",
        "          \"file_uri\": \"gs://cloud-samples-data/generative-ai/image/vase.png\"\n",
        "          }\n",
        "        },\n",
        "        {\"text\": \"add sunflowers to this vase\"},\n",
        "      ]\n",
        "    },\n",
        "    \"generation_config\": {\n",
        "      \"response_modalities\": [\"TEXT\", \"IMAGE\"],\n",
        "    }\n",
        "  }' 2>/dev/null >response.json"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6ixkSIAAa4mx"
      },
      "source": [
        "Extract the image data from the response and visualize.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "PCV4kWgmr1oy"
      },
      "outputs": [],
      "source": [
        "with open(\"response.json\") as f:\n",
        "    response_data = json.load(f)\n",
        "\n",
        "image_part = next(\n",
        "    filter(\n",
        "        lambda x: \"inlineData\" in x,\n",
        "        response_data[\"candidates\"][0][\"content\"][\"parts\"],\n",
        "    )\n",
        ")\n",
        "\n",
        "image_data = base64.b64decode(image_part[\"inlineData\"][\"data\"])\n",
        "display(Image(data=image_data, width=350, height=350))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2SDChAxBa_hE"
      },
      "source": [
        "Now, you'll add to the `contents` of the last request by including another `user` text prompt."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 58,
      "metadata": {
        "id": "Y2DmOotur9Ws"
      },
      "outputs": [],
      "source": [
        "%%bash\n",
        "\n",
        "curl -X POST \\\n",
        "  -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
        "  -H \"Content-Type: application/json\" \\\n",
        "  https://${API_ENDPOINT}:generateContent \\\n",
        "  -d '{\n",
        "    \"contents\": [\n",
        "    {\n",
        "      \"role\": \"user\",\n",
        "      \"parts\": [\n",
        "        {\"file_data\": {\n",
        "          \"mime_type\": \"image/png\",\n",
        "          \"file_uri\": \"gs://cloud-samples-data/generative-ai/image/vase.png\"\n",
        "          }\n",
        "        },\n",
        "        {\"text\": \"add sunflowers to this vase\"},\n",
        "      ]\n",
        "    },\n",
        "    {\n",
        "      \"role\": \"user\",\n",
        "      \"parts\": [\n",
        "        { \"text\": \"replace the sunflowers in the vase with pink and purple tulips\" },\n",
        "      ],\n",
        "    },\n",
        "    ],\n",
        "    \"generation_config\": {\n",
        "      \"response_modalities\": [\"TEXT\", \"IMAGE\"],\n",
        "    },\n",
        "  }' 2>/dev/null >response.json"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P91PCAb9bU8X"
      },
      "source": [
        "Extract the image data from the response and visualize.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6JMtRvBHuLGh"
      },
      "outputs": [],
      "source": [
        "with open(\"response.json\") as f:\n",
        "    response_data = json.load(f)\n",
        "\n",
        "image_part = next(\n",
        "    filter(\n",
        "        lambda x: \"inlineData\" in x,\n",
        "        response_data[\"candidates\"][0][\"content\"][\"parts\"],\n",
        "    )\n",
        ")\n",
        "\n",
        "image_data = base64.b64decode(image_part[\"inlineData\"][\"data\"])\n",
        "display(Image(data=image_data, width=350, height=350))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IzYiZyGzYZbM"
      },
      "source": [
        "### Images and text to image and text\n",
        "\n",
        "When editing images with Gemini 2.0 Flash, you can also supply multiple input images to create new ones. In this next example, you'll prompt Gemini with an image of a teacup and an outdoor table. You'll then ask Gemini to combine the objects from these images in order to create a new one. You'll also ask Gemini to supply text to accompany the image.\n",
        "\n",
        "Visualize the starting images by running this next cell."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8Xp4NndcRzey"
      },
      "outputs": [],
      "source": [
        "table_url = (\n",
        "    \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/table.png\"\n",
        ")\n",
        "display(Image(url=table_url, width=300, height=300))\n",
        "\n",
        "teacup_url = (\n",
        "    \"https://storage.googleapis.com/cloud-samples-data/generative-ai/image/teacup-1.png\"\n",
        ")\n",
        "display(Image(url=teacup_url, width=300, height=300))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 61,
      "metadata": {
        "id": "fVB5dt8sRKf3"
      },
      "outputs": [],
      "source": [
        "%%bash\n",
        "\n",
        "curl -X POST \\\n",
        "  -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
        "  -H \"Content-Type: application/json\" \\\n",
        "  https://${API_ENDPOINT}:generateContent \\\n",
        "  -d '{\n",
        "    \"contents\": {\n",
        "      \"role\": \"USER\",\n",
        "      \"parts\": [\n",
        "      { \"text\": \"Generate a side profile image of a person sitting at this table drinking out of this teacup in a 1:1 aspect ratio. Include a caption that could be used to post this image on social media.\"},\n",
        "      {\"file_data\": {\n",
        "          \"mime_type\": \"image/png\",\n",
        "          \"file_uri\": \"gs://cloud-samples-data/generative-ai/image/table.png\"\n",
        "          }\n",
        "        },\n",
        "        {\"file_data\": {\n",
        "          \"mime_type\": \"image/png\",\n",
        "          \"file_uri\": \"gs://cloud-samples-data/generative-ai/image/teacup-1.png\"\n",
        "          }\n",
        "        },]\n",
        "    },\n",
        "    \"generation_config\": {\n",
        "      \"response_modalities\": [\"TEXT\", \"IMAGE\"],\n",
        "     },\n",
        "     \"safetySettings\": {\n",
        "      \"method\": \"PROBABILITY\",\n",
        "      \"category\": \"HARM_CATEGORY_DANGEROUS_CONTENT\",\n",
        "      \"threshold\": \"BLOCK_MEDIUM_AND_ABOVE\"\n",
        "    },\n",
        "  }' 2>/dev/null >response.json"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PcwK0UQ7bk7f"
      },
      "source": [
        "Extract the text and image data from the response and visualize.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nG_3PKV6S3cY"
      },
      "outputs": [],
      "source": [
        "with open(\"response.json\") as f:\n",
        "    response_data = json.load(f)\n",
        "\n",
        "for part in response_data[\"candidates\"][0][\"content\"][\"parts\"]:\n",
        "    if \"text\" in part.keys():\n",
        "        display(Markdown(part[\"text\"]))\n",
        "    if \"inlineData\" in part.keys():\n",
        "        content = part[\"inlineData\"][\"data\"]\n",
        "        image_data = base64.b64decode(content)\n",
        "        display(Image(data=image_data, width=350, height=350))"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "intro_gemini_2_0_image_gen_rest_api.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
