{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ijGzTHJJUCPY"
      },
      "outputs": [],
      "source": [
        "# Copyright 2024 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NDsTUvKjwHBW"
      },
      "source": [
        "# Introduction to Multimodal Embeddings on Vertex AI\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/embeddings/intro_multimodal_embeddings.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fembeddings%2Fintro_multimodal_embeddings.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Run in Colab Enterprise\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/embeddings/intro_multimodal_embeddings.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/embeddings/intro_multimodal_embeddings.ipynb\">\n",
        "      <img src=\"https://www.gstatic.com/images/branding/gcpiconscolors/vertexai/v1/32px.svg\" alt=\"Vertex AI logo\"><br> Open in Vertex AI Workbench\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://goo.gle/4fVmfkB\">\n",
        "      <img width=\"32px\" src=\"https://cdn.qwiklabs.com/assets/gcp_cloud-e3a77215f0b8bfa9b3f611c0d2208c7e8708ed31.svg\" alt=\"Google Cloud logo\"><br> Open in  Cloud Skills Boost\n",
        "    </a>\n",
        "  </td>\n",
        "</table>\n",
        "\n",
        "<div style=\"clear: both;\"></div>\n",
        "\n",
        "<b>Share to:</b>\n",
        "\n",
        "<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/embeddings/intro_multimodal_embeddings.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/embeddings/intro_multimodal_embeddings.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/embeddings/intro_multimodal_embeddings.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/embeddings/intro_multimodal_embeddings.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/embeddings/intro_multimodal_embeddings.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
        "</a>            "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4uoRmYQsKBgl"
      },
      "source": [
        "| Authors |\n",
        "| --- |\n",
        "| [Lavi Nigam](https://github.com/lavinigam-gcp) |\n",
        "| [Kaz Sato](https://github.com/kazunori279) |"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RQT500QqVPIb"
      },
      "source": [
        "### Objectives\n",
        "\n",
        "In this notebook, you will explore:\n",
        "* Vertex AI Multimodal Embeddings API (Texts, Images & Video)\n",
        "* Building simple search with e-commerce data\n",
        "    - Find product based on text query\n",
        "    - Find product based on image\n",
        "    - Find Video based on video"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "76476d2db2c0"
      },
      "source": [
        "## Multimodal Embeddings\n",
        "\n",
        "[Vertex AI Multimodal Embeddings API](https://cloud.google.com/vertex-ai/generative-ai/docs/embeddings/get-multimodal-embeddings) generates [`128`, `256`, `512`, and `1408` (default)] -dimension vectors based on the input you provide, which can include a combination of image, text, and video data. The embedding vectors can then be used for subsequent tasks like image classification or video content moderation.\n",
        "\n",
        "The image embedding vector and text embedding vector generated with this API shares the semantic space. Consequently, these vectors can be used interchangeably for use cases like searching image by text, or searching video by image.\n",
        "\n",
        "**Use cases**\n",
        "\n",
        "**Image and text:**\n",
        "\n",
        "* Image classification: Takes an image as input and predicts one or more classes (labels).\n",
        "* Image search: Search relevant or similar images.\n",
        "* Recommendations: Generate product or ad recommendations based on images.\n",
        "\n",
        "**Image, text, and video:**\n",
        "\n",
        "* Recommendations: Generate product or advertisement recommendations based on videos (similarity search).\n",
        "* Video content search\n",
        "    * Using semantic search: Take a text as an input, and return a set of ranked frames matching the query.\n",
        "* Using similarity search:\n",
        "    * Take a video as an input, and return a set of videos matching the query.\n",
        "    * Take an image as an input, and return a set of videos matching the query.\n",
        "* Video classification: Takes a video as input and predicts one or more classes."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DXJpXzKrh2rJ"
      },
      "source": [
        "## Getting Started"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N5afkyDMSBW5"
      },
      "source": [
        "### Install Vertex AI SDK for Python and other dependencies"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "kc4WxYmLSBW5"
      },
      "outputs": [],
      "source": [
        "%pip install --upgrade --quiet google-cloud-aiplatform numpy pandas seaborn scikit-learn"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FtsU9Bw9h2rL"
      },
      "source": [
        "### Authenticate your notebook environment (Colab only)\n",
        "\n",
        "If you are running this notebook on Google Colab, run the following cell to authenticate your environment. This step is not required if you are using [Vertex AI Workbench](https://cloud.google.com/vertex-ai-workbench)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GpYEyLsOh2rL"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "# Additional authentication is required for Google Colab\n",
        "if \"google.colab\" in sys.modules:\n",
        "    # Authenticate user to Google Cloud\n",
        "    from google.colab import auth\n",
        "\n",
        "    auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O1vKZZoEh2rL"
      },
      "source": [
        "### Set Google Cloud project information and initialize Vertex AI SDK\n",
        "\n",
        "To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
        "\n",
        "Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gJqZ76rJh2rM"
      },
      "outputs": [],
      "source": [
        "# Use the environment variable if the user doesn't provide Project ID.\n",
        "import os\n",
        "\n",
        "PROJECT_ID = \"[your-project-id]\"  # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
        "if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
        "    PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
        "\n",
        "LOCATION = os.environ.get(\"GOOGLE_CLOUD_REGION\", \"us-central1\")\n",
        "\n",
        "# Initialize Vertex AI\n",
        "import vertexai\n",
        "\n",
        "vertexai.init(project=PROJECT_ID, location=LOCATION)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BuQwwRiniVFG"
      },
      "source": [
        "### Import libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rtMowvm-yQ97"
      },
      "outputs": [],
      "source": [
        "# for data processing\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import seaborn as sns\n",
        "\n",
        "pd.options.mode.chained_assignment = None  # default='warn'\n",
        "\n",
        "# for showing images and videos\n",
        "from IPython.display import HTML\n",
        "from IPython.display import Image as ImageByte\n",
        "from IPython.display import display\n",
        "from sklearn.metrics.pairwise import cosine_similarity\n",
        "\n",
        "# vertex ai sdk\n",
        "from vertexai.vision_models import Image as VMImage\n",
        "from vertexai.vision_models import MultiModalEmbeddingModel\n",
        "from vertexai.vision_models import Video as VMVideo\n",
        "from vertexai.vision_models import VideoSegmentConfig"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r-TX_R_xh2rM"
      },
      "source": [
        "### Load Vertex AI Multimodal Embeddings"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SvMwSRJJh2rM"
      },
      "outputs": [],
      "source": [
        "mm_embedding_model = MultiModalEmbeddingModel.from_pretrained(\"multimodalembedding\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "71bf88d9898a"
      },
      "source": [
        "### Helper functions"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2dd2e6b6cb97"
      },
      "outputs": [],
      "source": [
        "def get_text_embedding(\n",
        "    text: str = \"banana muffins\",\n",
        "    dimension: int | None = 1408,\n",
        ") -> list[float]:\n",
        "    embedding = mm_embedding_model.get_embeddings(\n",
        "        contextual_text=text,\n",
        "        dimension=dimension,\n",
        "    )\n",
        "    return embedding.text_embedding\n",
        "\n",
        "\n",
        "def get_image_embedding(\n",
        "    image_path: str,\n",
        "    dimension: int | None = 1408,\n",
        ") -> list[float]:\n",
        "    image = VMImage.load_from_file(image_path)\n",
        "    embedding = mm_embedding_model.get_embeddings(\n",
        "        image=image,\n",
        "        dimension=dimension,\n",
        "    )\n",
        "    return embedding.image_embedding\n",
        "\n",
        "\n",
        "def get_video_embedding(\n",
        "    video_path: str,\n",
        "    dimension: int | None = 1408,\n",
        "    video_segment_config: VideoSegmentConfig | None = None,\n",
        ") -> list[float]:\n",
        "    video = VMVideo.load_from_file(video_path)\n",
        "    embedding = mm_embedding_model.get_embeddings(\n",
        "        video=video,\n",
        "        dimension=dimension,\n",
        "        video_segment_config=video_segment_config,\n",
        "    )\n",
        "    return [video_emb.embedding for video_emb in embedding.video_embeddings]\n",
        "\n",
        "\n",
        "def get_public_url_from_gcs(gcs_uri: str) -> str:\n",
        "    return gcs_uri.replace(\"gs://\", \"https://storage.googleapis.com/\").replace(\n",
        "        \" \", \"%20\"\n",
        "    )\n",
        "\n",
        "\n",
        "def display_video_from_gcs(gcs_uri: str) -> None:\n",
        "    display(\n",
        "        HTML(\n",
        "            f\"\"\"\n",
        "    <video width=\"640\" height=\"480\" controls>\n",
        "        <source src=\"{get_public_url_from_gcs(gcs_uri)}\" type=\"video/mp4\">\n",
        "        Your browser does not support the video tag.\n",
        "    </video>\n",
        "    \"\"\"\n",
        "        )\n",
        "    )\n",
        "\n",
        "\n",
        "def print_similar_images(query_emb: list[float], data_frame: pd.DataFrame):\n",
        "    # calc dot product\n",
        "    image_embs = data_frame[\"image_embeddings\"]\n",
        "    scores = [np.dot(eval(image_emb), query_emb) for image_emb in image_embs]\n",
        "    data_frame[\"score\"] = scores\n",
        "    data_frame = data_frame.sort_values(by=\"score\", ascending=False)\n",
        "\n",
        "    # print results\n",
        "    print(data_frame.head()[[\"score\", \"title\"]])\n",
        "    for url in data_frame.head()[\"gcs_path\"]:\n",
        "        display(ImageByte(url=get_public_url_from_gcs(url), width=200, height=200))\n",
        "\n",
        "\n",
        "def print_similar_videos(query_emb: list[float], data_frame: pd.DataFrame):\n",
        "    # calc dot product\n",
        "    video_embs = data_frame[\"video_embeddings\"]\n",
        "    scores = [np.dot(eval(video_emb), query_emb) for video_emb in video_embs]\n",
        "    data_frame[\"score\"] = scores\n",
        "    data_frame = data_frame.sort_values(by=\"score\", ascending=False)\n",
        "\n",
        "    # print results\n",
        "    print(data_frame.head()[[\"score\", \"file_name\"]])\n",
        "    url = data_frame.iloc[0][\"gcs_path\"]\n",
        "    display_video_from_gcs(url)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2176030e88a0"
      },
      "source": [
        "## Generate Text Embeddings"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "beaMkDH9zlr2"
      },
      "outputs": [],
      "source": [
        "text_emb = get_text_embedding(text=\"What is life?\")\n",
        "print(\"length of embedding: \", len(text_emb))\n",
        "print(\"First five values are: \", text_emb[:5])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oXC0cEjDywIM"
      },
      "source": [
        "#### Embeddings and Pandas DataFrames"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oug-MkjFywzc"
      },
      "source": [
        "If your text is stored in a column of a DataFrame, you can create a new column with the embeddings with the example below."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "trpS0J6ByNd0"
      },
      "outputs": [],
      "source": [
        "text = [\n",
        "    \"i really enjoyed the movie last night\",\n",
        "    \"so many amazing cinematic scenes yesterday\",\n",
        "    \"had a great time writing my Python scripts a few days ago\",\n",
        "    \"huge sense of relief when my .py script finally ran without error\",\n",
        "    \"O Romeo, Romeo, wherefore art thou Romeo?\",\n",
        "]\n",
        "\n",
        "df = pd.DataFrame(text, columns=[\"text\"])\n",
        "df"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "luwdn0qA0QZN"
      },
      "source": [
        "Create a new column, `embeddings`, using the [`apply()`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.apply.html) function in pandas with the embeddings model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "etrempeSyQ9V"
      },
      "outputs": [],
      "source": [
        "df[\"embedding\"] = df.apply(lambda x: get_text_embedding(x.text), axis=1)\n",
        "df"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3_rtywdA0cv6"
      },
      "source": [
        "#### Comparing similarity of text examples using cosine similarity"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BXtUBuV9y4oX"
      },
      "outputs": [],
      "source": [
        "cos_sim_array = cosine_similarity(list(df.embedding.values))\n",
        "\n",
        "# display as DataFrame\n",
        "df = pd.DataFrame(cos_sim_array, index=text, columns=text)\n",
        "df"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BFE6uErz0gGe"
      },
      "source": [
        "To make this easier to understand, you can use a heatmap. Naturally, text is most similar when they are identical (score of 1.0). The next highest scores are when sentences are semantically similar. The lowest scores are when sentences are quite different in meaning."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6E4cfugj0gmC"
      },
      "outputs": [],
      "source": [
        "ax = sns.heatmap(df, annot=True, cmap=\"crest\")\n",
        "ax.xaxis.tick_top()\n",
        "ax.set_xticklabels(text, rotation=90)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7a2d7f3e3a33"
      },
      "source": [
        "## Generate Image Embeddings"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "10kzx4YoVjMU"
      },
      "outputs": [],
      "source": [
        "# Image embeddings with default 1408 dimension\n",
        "image_path = \"gs://github-repo/embeddings/getting_started_embeddings/gms_images/GGOEACBA104999.jpg\"\n",
        "print(get_public_url_from_gcs(image_path))\n",
        "\n",
        "image_emb = get_image_embedding(\n",
        "    image_path=image_path,\n",
        ")\n",
        "print(\"length of embedding: \", len(image_emb))\n",
        "print(\"First five values are: \", image_emb[:5])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "73CFiID6WNo8"
      },
      "source": [
        "### Find product images based on text search query"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nigd4xO7Vow5"
      },
      "outputs": [],
      "source": [
        "# get product list with pre-computed image embeddings\n",
        "product_image_list = pd.read_csv(\n",
        "    \"https://storage.googleapis.com/github-repo/embeddings/getting_started_embeddings/image_data_with_embeddings.csv\"\n",
        ")\n",
        "product_image_list.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SIiC_5JxWRMw"
      },
      "outputs": [],
      "source": [
        "# calc_scores for a text query\n",
        "query_emb = get_text_embedding(\"something related to dinosaurs theme\")\n",
        "print_similar_images(query_emb, product_image_list)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "B3Dui6Y1gM4G"
      },
      "outputs": [],
      "source": [
        "query_emb = get_text_embedding(\"Socks in checkered patterns\")\n",
        "print_similar_images(query_emb, product_image_list)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fdcec1fefb20"
      },
      "source": [
        "## Generate Video Embeddings"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QeGOaYxGwtdp"
      },
      "outputs": [],
      "source": [
        "# Video embeddings with 1408 dimension\n",
        "video_path = \"gs://github-repo/embeddings/getting_started_embeddings/UCF-101-subset/BrushingTeeth/v_BrushingTeeth_g01_c02.mp4\"\n",
        "display_video_from_gcs(video_path)\n",
        "\n",
        "video_emb = get_video_embedding(\n",
        "    video_path=video_path,\n",
        ")\n",
        "\n",
        "print(\"length of embedding: \", len(video_emb[0]))\n",
        "print(\"First five values of the first segment are: \", video_emb[0][:5])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "113e181c0cef"
      },
      "source": [
        "### Find videos based on text search query"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6a8c536bb20a"
      },
      "outputs": [],
      "source": [
        "video_list = pd.read_csv(\n",
        "    \"https://storage.googleapis.com/github-repo/embeddings/getting_started_embeddings/video_data_with_embeddings.csv\"\n",
        ")\n",
        "print(f\"Items in the video list: {len(video_list)}\")\n",
        "video_list.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "btCIJCh2OnSP"
      },
      "source": [
        "### Find Similar videos"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3a3c0614b7c3"
      },
      "outputs": [],
      "source": [
        "query_emb = get_text_embedding(\"A music concert\")\n",
        "print_similar_videos(query_emb, video_list)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ad004ac41e8f"
      },
      "outputs": [],
      "source": [
        "query_emb = get_text_embedding(\"A person playing a TaiChi\")\n",
        "print_similar_videos(query_emb, video_list)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IddiO0T-wdno"
      },
      "source": [
        "## What's next?\n",
        "\n",
        "- Learn how to store the vectors (embeddings) into Vertex AI Vector Search: [Notebook](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/embeddings/vector-search-quickstart.ipynb)\n",
        "- Learn how to tune the embeddings with your own data: [Notebook](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/embeddings/intro_embeddings_tuning.ipynb)\n",
        "- Learn how to use embeddings to do Text RAG and Multimodal RAG: [Notebook](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/use-cases/retrieval-augmented-generation/intro_multimodal_rag.ipynb)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "intro_multimodal_embeddings.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
