{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YJy8qKC5Zwmx"
      },
      "outputs": [],
      "source": [
        "# Copyright 2025 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A8dLtX2lZ6ah"
      },
      "source": [
        "# Vector Search 2.0 Public Preview Quickstart\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/embeddings/vector-search-2-quickstart.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fembeddings%2Fvector-search-2-quickstart.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Run in Colab Enterprise\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/embeddings/vector-search-2-quickstart.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "</table>\n",
        "\n",
        "<div style=\"clear: both;\"></div>\n",
        "\n",
        "<b>Share to:</b>\n",
        "\n",
        "<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/embeddings/vector-search-2-quickstart.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/embeddings/vector-search-2-quickstart.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/embeddings/vector-search-2-quickstart.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/embeddings/vector-search-2-quickstart.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/embeddings/vector-search-2-quickstart.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
        "</a>            "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3zQRr7StaHj3"
      },
      "source": [
        "## Objectives\n",
        "\n",
        "In this notebook, you will learn how to get started with the Vector Search 2.0 public preview API.\n",
        "\n",
        "### **Warning: delete your objects after the tutorial**\n",
        "\n",
        "In case you are using your own Cloud project, please make sure to delete all the Collection and any associated Indexes after finishing this tutorial. Otherwise the remaining assets would incur unexpected costs.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8rNNxNcaaQRb"
      },
      "source": [
        "## Prerequisites\n",
        "\n",
        "This tutorial requires a Google Cloud project that is linked with a billing account. To create a new project, take a look at [this document](https://cloud.google.com/vertex-ai/docs/start/cloud-environment) to create a project and setup a billing account for it.\n",
        "To get the permissions that you need to give a service account access to enable APIs and interact with Vertex AI resources, ask your administrator to grant you the [Security Admin](https://cloud.google.com/iam/docs/roles-permissions/iam#iam.securityAdmin) (`roles/iam.securityAdmin`) IAM role on your project. For more information about granting roles, see [Manage access to projects, folders, and organizations](https://cloud.google.com/iam/docs/granting-changing-revoking-access).[link text](https://)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZBbY9yJvbuL1"
      },
      "source": [
        "## Install the Vector Search SDK"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "69744e32"
      },
      "outputs": [],
      "source": [
        "%pip install google-cloud-vectorsearch"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bQ5q_KMkbXSi"
      },
      "source": [
        "## Environment variables\n",
        "\n",
        "Set environment variables for your project and location."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "VrZJvAovvN7D"
      },
      "outputs": [],
      "source": [
        "PROJECT_ID = \"your-project-id\"  # @param {type:\"string\"}\n",
        "LOCATION = \"us-central1\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nitj2X54bgQC"
      },
      "source": [
        "## Authentication\n",
        "\n",
        "On Colab, run the following to authenticate calls to the Vector Search APIs:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zQKHCit5vo-o"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "# Additional authentication is required for Google Colab\n",
        "if \"google.colab\" in sys.modules:\n",
        "    # Authenticate user to Google Cloud\n",
        "    from google.colab import auth\n",
        "\n",
        "    auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WycjwYB2Djp1"
      },
      "source": [
        "## Enable APIs\n",
        "\n",
        "Run the following commands to enable APIs for Vector Search and, if using Auto-Embeddings or Semantic Search, the Vertex AI API with this Google Cloud project.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "e3VZ0UNJDkg-"
      },
      "outputs": [],
      "source": [
        "! gcloud services enable vectorsearch.googleapis.com aiplatform.googleapis.com --project \"{PROJECT_ID}\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9gxfSP7sbIVU"
      },
      "source": [
        "## Initialize Clients"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zfiSJTIJvzI1"
      },
      "outputs": [],
      "source": [
        "from google.cloud import vectorsearch_v1beta\n",
        "\n",
        "vector_search_service_client = vectorsearch_v1beta.VectorSearchServiceClient()\n",
        "data_object_service_client = vectorsearch_v1beta.DataObjectServiceClient()\n",
        "data_object_search_service_client = vectorsearch_v1beta.DataObjectSearchServiceClient()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AOQkdB5mYtbU"
      },
      "source": [
        "## Create Collection"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "AOTKBWwjYvJj"
      },
      "outputs": [],
      "source": [
        "import getpass\n",
        "from datetime import datetime\n",
        "\n",
        "collection_id = f\"movies-demo-{getpass.getuser()}-{datetime.now().strftime('%m-%d-%y')}\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Nv1wUdXvY4kZ"
      },
      "outputs": [],
      "source": [
        "request = vectorsearch_v1beta.CreateCollectionRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}\",\n",
        "    collection_id=collection_id,\n",
        "    collection={\n",
        "        \"data_schema\": {\n",
        "            \"type\": \"object\",\n",
        "            \"properties\": {\n",
        "                \"year\": {\"type\": \"number\"},\n",
        "                \"genre\": {\"type\": \"string\"},\n",
        "                \"director\": {\"type\": \"string\"},\n",
        "                \"title\": {\"type\": \"string\"},\n",
        "            },\n",
        "        },\n",
        "        \"vector_schema\": {\n",
        "            \"plot_embedding\": {\"dense_vector\": {\"dimensions\": 3}},\n",
        "            \"soundtrack_embedding\": {\"dense_vector\": {\"dimensions\": 5}},\n",
        "            \"genre_embedding\": {\n",
        "                \"dense_vector\": {\n",
        "                    \"dimensions\": 4,\n",
        "                    \"vertex_embedding_config\": {\n",
        "                        # If a data object is created without a supplied value for genre_embedding, it will be\n",
        "                        # auto-generated based on this config.\n",
        "                        \"model_id\": \"text-embedding-004\",\n",
        "                        \"text_template\": (\"Movie: {title} Genre: {genre} Year: {year}\"),\n",
        "                        \"task_type\": \"RETRIEVAL_DOCUMENT\",\n",
        "                    },\n",
        "                }\n",
        "            },\n",
        "            \"sparse_embedding\": {\"sparse_vector\": {}},\n",
        "        },\n",
        "    },\n",
        ")\n",
        "operation = vector_search_service_client.create_collection(request=request)\n",
        "operation.result()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mBji6kevajeO"
      },
      "source": [
        "## Get Collection"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mHT5ef44akz3"
      },
      "outputs": [],
      "source": [
        "request = vectorsearch_v1beta.GetCollectionRequest(\n",
        "    name=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\"\n",
        ")\n",
        "vector_search_service_client.get_collection(request)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6t8c1DP-a_CX"
      },
      "source": [
        "## List Collections"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "b-6jO3ifv4M-"
      },
      "outputs": [],
      "source": [
        "request = vectorsearch_v1beta.ListCollectionsRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}\"\n",
        ")\n",
        "vector_search_service_client.list_collections(request)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c3n0ofiYbSOR"
      },
      "source": [
        "## Generate Sample Data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BE4UEvwbbU_t"
      },
      "outputs": [],
      "source": [
        "import math\n",
        "import random\n",
        "\n",
        "def normalize(v):\n",
        "    norm = math.sqrt(sum(x * x for x in v))\n",
        "    return [x / norm for x in v]\n",
        "\n",
        "\n",
        "# Sparse embedding generation\n",
        "\n",
        "\n",
        "VOCAB_SIZE = 50000  # Imagine a vocabulary of 50,000 possible items\n",
        "NON_ZERO_COUNT = 7  # We want to represent an item that has 7 active features\n",
        "VALUE_RANGE = (1, 10)  # Let's say the values represent ratings from 1 to 10\n",
        "\n",
        "\n",
        "def generate_sparse_embedding(\n",
        "    vocabulary_size: int, num_non_zero: int, value_range: tuple = (1, 100)\n",
        ") -> tuple[list[int], list[int]]:\n",
        "    \"\"\"Generates a random sparse representation with unique indices and corresponding values.\n",
        "\n",
        "    Args:\n",
        "        vocabulary_size (int): The total size of the embedding space (e.g.,\n",
        "          10000).\n",
        "        num_non_zero (int): The number of non-zero elements to generate.\n",
        "        value_range (tuple): A tuple (min, max) for the range of random integer\n",
        "          values.\n",
        "\n",
        "    Returns:\n",
        "        tuple[list[int], list[int]]: A tuple containing two lists:\n",
        "                                     - A sorted list of unique indices.\n",
        "                                     - A list of corresponding random values.\n",
        "    \"\"\"\n",
        "    # Ensure we don't try to pick more unique items than available\n",
        "    if num_non_zero > vocabulary_size:\n",
        "        raise ValueError(\n",
        "            \"Number of non-zero elements cannot exceed the vocabulary size.\"\n",
        "        )\n",
        "\n",
        "    # --- Generate Indices ---\n",
        "    # Pick a random sample of unique indices from the full range of the vocabulary.\n",
        "    # range(vocabulary_size) creates a sequence from 0 to vocabulary_size - 1.\n",
        "    indices = random.sample(range(vocabulary_size), num_non_zero)\n",
        "\n",
        "    # It's a common and good practice for sparse formats to have sorted indices.\n",
        "    indices.sort()\n",
        "\n",
        "    # --- Generate Values ---\n",
        "    # For each index, generate a corresponding random integer value.\n",
        "    min_val, max_val = value_range\n",
        "    values = [random.randint(min_val, max_val) for _ in range(num_non_zero)]\n",
        "\n",
        "    return indices, values\n",
        "\n",
        "\n",
        "movies = [\n",
        "    {\n",
        "        \"id\": \"the-shawshank-redemption\",\n",
        "        \"data\": {\n",
        "            \"title\": \"The Shawshank Redemption\",\n",
        "            \"genre\": \"Drama\",\n",
        "            \"year\": 1994,\n",
        "            \"director\": \"Frank Darabont\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"the-godfather\",\n",
        "        \"data\": {\n",
        "            \"title\": \"The Godfather\",\n",
        "            \"genre\": \"Crime\",\n",
        "            \"year\": 1972,\n",
        "            \"director\": \"Francis Ford Coppola\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"the-dark-knight\",\n",
        "        \"data\": {\n",
        "            \"title\": \"The Dark Knight\",\n",
        "            \"genre\": \"Action\",\n",
        "            \"year\": 2008,\n",
        "            \"director\": \"Christopher Nolan\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"pulp-fiction\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Pulp Fiction\",\n",
        "            \"genre\": \"Crime\",\n",
        "            \"year\": 1994,\n",
        "            \"director\": \"Quentin Tarantino\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"schindlers-list\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Schindler's List\",\n",
        "            \"genre\": \"Biography\",\n",
        "            \"year\": 1993,\n",
        "            \"director\": \"Steven Spielberg\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"twelve-angry-men\",\n",
        "        \"data\": {\n",
        "            \"title\": \"12 Angry Men\",\n",
        "            \"genre\": \"Drama\",\n",
        "            \"year\": 1957,\n",
        "            \"director\": \"Sidney Lumet\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"the-lord-of-the-rings-the-return-of-the-king\",\n",
        "        \"data\": {\n",
        "            \"title\": \"The Lord of the Rings: The Return of the King\",\n",
        "            \"genre\": \"Adventure\",\n",
        "            \"year\": 2003,\n",
        "            \"director\": \"Peter Jackson\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"spirited-away\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Spirited Away\",\n",
        "            \"genre\": \"Animation\",\n",
        "            \"year\": 2001,\n",
        "            \"director\": \"Hayao Miyazaki\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"parasite\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Parasite\",\n",
        "            \"genre\": \"Thriller\",\n",
        "            \"year\": 2019,\n",
        "            \"director\": \"Bong Joon-ho\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"the-matrix\",\n",
        "        \"data\": {\n",
        "            \"title\": \"The Matrix\",\n",
        "            \"genre\": \"Sci-Fi\",\n",
        "            \"year\": 1999,\n",
        "            \"director\": \"The Wachowskis\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"inception\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Inception\",\n",
        "            \"genre\": \"Sci-Fi\",\n",
        "            \"year\": 2010,\n",
        "            \"director\": \"Christopher Nolan\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"interstellar\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Interstellar\",\n",
        "            \"genre\": \"Sci-Fi\",\n",
        "            \"year\": 2014,\n",
        "            \"director\": \"Christopher Nolan\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"the-silence-of-the-lambs\",\n",
        "        \"data\": {\n",
        "            \"title\": \"The Silence of the Lambs\",\n",
        "            \"genre\": \"Thriller\",\n",
        "            \"year\": 1991,\n",
        "            \"director\": \"Jonathan Demme\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"psycho\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Psycho\",\n",
        "            \"genre\": \"Horror\",\n",
        "            \"year\": 1960,\n",
        "            \"director\": \"Alfred Hitchcock\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"the-green-mile\",\n",
        "        \"data\": {\n",
        "            \"title\": \"The Green Mile\",\n",
        "            \"genre\": \"Drama\",\n",
        "            \"year\": 1999,\n",
        "            \"director\": \"Frank Darabont\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"forrest-gump\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Forrest Gump\",\n",
        "            \"genre\": \"Drama\",\n",
        "            \"year\": 1994,\n",
        "            \"director\": \"Robert Zemeckis\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"fight-club\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Fight Club\",\n",
        "            \"genre\": \"Drama\",\n",
        "            \"year\": 1999,\n",
        "            \"director\": \"David Fincher\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"the-lion-king\",\n",
        "        \"data\": {\n",
        "            \"title\": \"The Lion King\",\n",
        "            \"genre\": \"Animation\",\n",
        "            \"year\": 1994,\n",
        "            \"director\": \"Roger Allers\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"beauty-and-the-beast\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Beauty and the Beast\",\n",
        "            \"genre\": \"Animation\",\n",
        "            \"year\": 1991,\n",
        "            \"director\": \"Gary Trousdale\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"toy-story\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Toy Story\",\n",
        "            \"genre\": \"Animation\",\n",
        "            \"year\": 1995,\n",
        "            \"director\": \"John Lasseter\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"goodfellas\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Goodfellas\",\n",
        "            \"genre\": \"Crime\",\n",
        "            \"year\": 1990,\n",
        "            \"director\": \"Martin Scorsese\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"seven\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Seven\",\n",
        "            \"genre\": \"Thriller\",\n",
        "            \"year\": 1995,\n",
        "            \"director\": \"David Fincher\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"se7en\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Se7en\",\n",
        "            \"genre\": \"Thriller\",\n",
        "            \"year\": 1995,\n",
        "            \"director\": \"David Fincher\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"city-of-god\",\n",
        "        \"data\": {\n",
        "            \"title\": \"City of God\",\n",
        "            \"genre\": \"Crime\",\n",
        "            \"year\": 2002,\n",
        "            \"director\": \"Fernando Meirelles\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"the-departed\",\n",
        "        \"data\": {\n",
        "            \"title\": \"The Departed\",\n",
        "            \"genre\": \"Crime\",\n",
        "            \"year\": 2006,\n",
        "            \"director\": \"Martin Scorsese\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"oldboy\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Oldboy\",\n",
        "            \"genre\": \"Thriller\",\n",
        "            \"year\": 2003,\n",
        "            \"director\": \"Park Chan-wook\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"memento\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Memento\",\n",
        "            \"genre\": \"Thriller\",\n",
        "            \"year\": 2000,\n",
        "            \"director\": \"Christopher Nolan\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"shutter-island\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Shutter Island\",\n",
        "            \"genre\": \"Thriller\",\n",
        "            \"year\": 2010,\n",
        "            \"director\": \"Martin Scorsese\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"the-usual-suspects\",\n",
        "        \"data\": {\n",
        "            \"title\": \"The Usual Suspects\",\n",
        "            \"genre\": \"Thriller\",\n",
        "            \"year\": 1995,\n",
        "            \"director\": \"Bryan Singer\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"gone-girl\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Gone Girl\",\n",
        "            \"genre\": \"Thriller\",\n",
        "            \"year\": 2014,\n",
        "            \"director\": \"David Fincher\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"the-sixth-sense\",\n",
        "        \"data\": {\n",
        "            \"title\": \"The Sixth Sense\",\n",
        "            \"genre\": \"Horror\",\n",
        "            \"year\": 1999,\n",
        "            \"director\": \"M. Night Shyamalan\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"the-others\",\n",
        "        \"data\": {\n",
        "            \"title\": \"The Others\",\n",
        "            \"genre\": \"Horror\",\n",
        "            \"year\": 2001,\n",
        "            \"director\": \"Alejandro Amenábar\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"the-ring\",\n",
        "        \"data\": {\n",
        "            \"title\": \"The Ring\",\n",
        "            \"genre\": \"Horror\",\n",
        "            \"year\": 2002,\n",
        "            \"director\": \"Gore Verbinski\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"the-exorcist\",\n",
        "        \"data\": {\n",
        "            \"title\": \"The Exorcist\",\n",
        "            \"genre\": \"Horror\",\n",
        "            \"year\": 1973,\n",
        "            \"director\": \"William Friedkin\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"singin-in-the-rain\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Singin' in the Rain\",\n",
        "            \"genre\": \"Musical\",\n",
        "            \"year\": 1952,\n",
        "            \"director\": \"Stanley Donen\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"the-sound-of-music\",\n",
        "        \"data\": {\n",
        "            \"title\": \"The Sound of Music\",\n",
        "            \"genre\": \"Musical\",\n",
        "            \"year\": 1965,\n",
        "            \"director\": \"Robert Wise\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"west-side-story\",\n",
        "        \"data\": {\n",
        "            \"title\": \"West Side Story\",\n",
        "            \"genre\": \"Musical\",\n",
        "            \"year\": 1961,\n",
        "            \"director\": \"Robert Wise\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"seven-samurai\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Seven Samurai\",\n",
        "            \"genre\": \"Adventure\",\n",
        "            \"year\": 1954,\n",
        "            \"director\": \"Akira Kurosawa\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"my-neighbor-totoro\",\n",
        "        \"data\": {\n",
        "            \"title\": \"My Neighbor Totoro\",\n",
        "            \"genre\": \"Animation\",\n",
        "            \"year\": 1988,\n",
        "            \"director\": \"Hayao Miyazaki\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"howls-moving-castle\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Howl's Moving Castle\",\n",
        "            \"genre\": \"Animation\",\n",
        "            \"year\": 2004,\n",
        "            \"director\": \"Hayao Miyazaki\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"ponyo\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Ponyo\",\n",
        "            \"genre\": \"Animation\",\n",
        "            \"year\": 2008,\n",
        "            \"director\": \"Hayao Miyazaki\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"the-secret-world-of-arrietty\",\n",
        "        \"data\": {\n",
        "            \"title\": \"The Secret World of Arrietty\",\n",
        "            \"genre\": \"Animation\",\n",
        "            \"year\": 2010,\n",
        "            \"director\": \"Hiromasa Yonebayashi\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"oklahoma\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Oklahoma!\",\n",
        "            \"genre\": \"Musical\",\n",
        "            \"year\": 1955,\n",
        "            \"director\": \"Fred Zinnemann\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"the-king-and-i\",\n",
        "        \"data\": {\n",
        "            \"title\": \"The King and I\",\n",
        "            \"genre\": \"Musical\",\n",
        "            \"year\": 1956,\n",
        "            \"director\": \"Walter Lang\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"my-fair-lady\",\n",
        "        \"data\": {\n",
        "            \"title\": \"My Fair Lady\",\n",
        "            \"genre\": \"Musical\",\n",
        "            \"year\": 1964,\n",
        "            \"director\": \"George Cukor\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"cabaret\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Cabaret\",\n",
        "            \"genre\": \"Musical\",\n",
        "            \"year\": 1972,\n",
        "            \"director\": \"Bob Fosse\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"grease\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Grease\",\n",
        "            \"genre\": \"Musical\",\n",
        "            \"year\": 1978,\n",
        "            \"director\": \"Randal Kleiser\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"chicago\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Chicago\",\n",
        "            \"genre\": \"Musical\",\n",
        "            \"year\": 2002,\n",
        "            \"director\": \"Rob Marshall\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"hairspray\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Hairspray\",\n",
        "            \"genre\": \"Musical\",\n",
        "            \"year\": 2007,\n",
        "            \"director\": \"Adam Shankman\",\n",
        "        },\n",
        "    },\n",
        "    {\n",
        "        \"id\": \"les-miserables\",\n",
        "        \"data\": {\n",
        "            \"title\": \"Les Misérables\",\n",
        "            \"genre\": \"Musical\",\n",
        "            \"year\": 2012,\n",
        "            \"director\": \"Tom Hooper\",\n",
        "        },\n",
        "    },\n",
        "]\n",
        "\n",
        "for movie in movies:\n",
        "    movie[\"vectors\"] = {}\n",
        "    movie[\"vectors\"][\"plot_embedding\"] = {\n",
        "        \"dense\": {\"values\": normalize([random.random() for _ in range(3)])}\n",
        "    }\n",
        "    movie[\"vectors\"][\"genre_embedding\"] = {\n",
        "        \"dense\": {\"values\": normalize([random.random() for _ in range(4)])}\n",
        "    }\n",
        "\n",
        "    # Create a cluster for \"Musical\" movies in soundtrack_embedding\n",
        "    if movie[\"data\"][\"genre\"] == \"Musical\":\n",
        "        movie[\"vectors\"][\"soundtrack_embedding\"] = {\n",
        "            \"dense\": {\n",
        "                \"values\": normalize([0.9 + random.random() * 0.1 for _ in range(5)])\n",
        "            }\n",
        "        }\n",
        "    else:\n",
        "        movie[\"vectors\"][\"soundtrack_embedding\"] = {\n",
        "            \"dense\": {\"values\": normalize([random.random() for _ in range(5)])}\n",
        "        }\n",
        "    indices, values = generate_sparse_embedding(\n",
        "        vocabulary_size=VOCAB_SIZE,\n",
        "        num_non_zero=NON_ZERO_COUNT,\n",
        "        value_range=VALUE_RANGE,\n",
        "    )\n",
        "    movie[\"vectors\"][\"sparse_embedding\"] = {\n",
        "        \"sparse\": {\"values\": values, \"indices\": indices}\n",
        "    }"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z8JMlDDgbkOB"
      },
      "source": [
        "## Populate Data Objects"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "VKBjHXWzb3bA"
      },
      "outputs": [],
      "source": [
        "movies[0]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O_u_MqM1bnJO"
      },
      "source": [
        "### Create Data Object"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Lyl9n6tSblgk"
      },
      "outputs": [],
      "source": [
        "request = vectorsearch_v1beta.CreateDataObjectRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    data_object_id=movies[0][\"id\"],\n",
        "    data_object={\n",
        "        \"data\": movies[0][\"data\"],\n",
        "        \"vectors\": movies[0][\"vectors\"],\n",
        "    },\n",
        ")\n",
        "data_object_service_client.create_data_object(request=request)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DZNvLmpkc_aZ"
      },
      "source": [
        "### Batch Create Data Objects"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Jrh2I_u2dEn9"
      },
      "outputs": [],
      "source": [
        "rest_of_movies_batch_request = []\n",
        "for movie in movies[1:-1]:\n",
        "    rest_of_movies_batch_request.append(\n",
        "        {\n",
        "            \"data_object_id\": movie[\"id\"],\n",
        "            \"data_object\": {\"data\": movie[\"data\"], \"vectors\": movie[\"vectors\"]},\n",
        "        }\n",
        "    )\n",
        "request = vectorsearch_v1beta.BatchCreateDataObjectsRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    requests=rest_of_movies_batch_request,\n",
        ")\n",
        "data_object_service_client.batch_create_data_objects(request)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i8yPvGcUTmys"
      },
      "source": [
        "### Import Data Objects from GCS file(s)\n",
        "\n",
        "The import will fail if the collection already has an ANN index."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8XlDRUHBTyiv"
      },
      "source": [
        "#### Prepare GCS data for import\n",
        "\n",
        "The example below writes one data object to a JSON file in the specified GCS bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fyMHek32lq4w"
      },
      "outputs": [],
      "source": [
        "import json\n",
        "import os\n",
        "\n",
        "IMPORT_BUCKET = f\"my-movie-demo-import-{PROJECT_ID}\"\n",
        "\n",
        "# Directory must only contain import data.\n",
        "IMPORT_DIRECTORY = \"import-data/\"\n",
        "IMPORT_FILE = \"movies.json\"\n",
        "\n",
        "# Error directory must be empty.\n",
        "IMPORT_ERROR_DIRECTORY = \"import-errors/\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "H40kxvcOFU_J"
      },
      "outputs": [],
      "source": [
        "import copy\n",
        "\n",
        "movie_to_import = copy.deepcopy(movies[-1])\n",
        "movie_to_import[\"vectors\"] = {\n",
        "    key: vector_info[\"dense\"][\"values\"]\n",
        "    if isinstance(vector_info, dict) and \"dense\" in vector_info\n",
        "    else vector_info\n",
        "    for key, vector_info in movie_to_import[\"vectors\"].items()\n",
        "}\n",
        "movie_string = json.dumps(movie_to_import)\n",
        "movie_string"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BCyLpyeZToET"
      },
      "outputs": [],
      "source": [
        "blob_name = os.path.join(IMPORT_DIRECTORY, IMPORT_FILE)\n",
        "gcs_uri = f\"gs://{IMPORT_BUCKET}/{blob_name}\"\n",
        "try:\n",
        "    from google.cloud import storage\n",
        "\n",
        "    storage_client = storage.Client(project=PROJECT_ID)\n",
        "    bucket = storage_client.bucket(IMPORT_BUCKET)\n",
        "    # To programmatically create the GCS bucket, uncomment the following line:\n",
        "    # storage_client.create_bucket(bucket)\n",
        "    blob = bucket.blob(blob_name)\n",
        "    blob.upload_from_string(movie_string)\n",
        "    print(f\"Successfully uploaded movie data to {gcs_uri}\")\n",
        "except Exception as e:\n",
        "    print(f\"Could not write to GCS using google-cloud-storage. Error: {e}\")\n",
        "    print(f\"Please manually copy the following JSON string to {gcs_uri}\")\n",
        "    print(movie_string)\n",
        "    print(\"Sample command:\")\n",
        "    print(f\"echo '{movie_string}' | gcloud storage cp - {gcs_uri}\")\n",
        "    print(\"(the bucket must already exist)\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jzmySijQT1wM"
      },
      "source": [
        "#### Perform the import"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8tjGeAs_T7oF"
      },
      "outputs": [],
      "source": [
        "contents_uri = f\"gs://{IMPORT_BUCKET}/{IMPORT_DIRECTORY}\"\n",
        "error_uri = f\"gs://{IMPORT_BUCKET}/{IMPORT_ERROR_DIRECTORY}\"\n",
        "print(f\"Importing from {contents_uri}\")\n",
        "print(f\"Errors will be written to {error_uri}\")\n",
        "request = vectorsearch_v1beta.ImportDataObjectsRequest(\n",
        "    name=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    gcs_import={\n",
        "        \"contents_uri\": contents_uri,\n",
        "        \"error_uri\": error_uri,\n",
        "    },\n",
        ")\n",
        "import_lro = vector_search_service_client.import_data_objects(request)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1caMIuc6mcSA"
      },
      "outputs": [],
      "source": [
        "print(f\"Waiting for import LRO: {import_lro.operation.name}\")\n",
        "import_lro.result()\n",
        "print(\"Import LRO complete.\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BILAS9zXHA6u"
      },
      "source": [
        "## Get Data Object"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "tSiB9WP8HCQi"
      },
      "outputs": [],
      "source": [
        "request = vectorsearch_v1beta.GetDataObjectRequest(\n",
        "        name=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}/dataObjects/{movies[0][\"id\"]}\",\n",
        "    )\n",
        "data_object_service_client.get_data_object(request=request)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zfFqCJEOINub"
      },
      "source": [
        "## Update Data Object"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "l5-hDOVVINDs"
      },
      "outputs": [],
      "source": [
        "request = vectorsearch_v1beta.UpdateDataObjectRequest(\n",
        "    data_object={\n",
        "        \"name\": f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}/dataObjects/{movies[0]['id']}\",\n",
        "        \"data\": {\"title\": f\"{movies[0]['data']['title']} (updated)\"},\n",
        "        \"vectors\": {\"plot_embedding\": {\"dense\": {\"values\": [1.0, 1.0, 1.0]}}},\n",
        "    }\n",
        ")\n",
        "data_object_service_client.update_data_object(request)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MNw5doXKIXsf"
      },
      "source": [
        "## Batch Update Data Object"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Rvg5EFsdIZtT"
      },
      "outputs": [],
      "source": [
        "movies[9][\"data\"][\"title\"] = movies[9][\"data\"][\"title\"] + \" updated\"\n",
        "movies[10][\"data\"][\"title\"] = movies[10][\"data\"][\"title\"] + \" updated\"\n",
        "movies[11][\"data\"][\"title\"] = movies[11][\"data\"][\"title\"] + \" updated\"\n",
        "collection_name = (\n",
        "    f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\"\n",
        ")\n",
        "requests = [\n",
        "    {\n",
        "        \"data_object\": {\n",
        "            \"name\": f\"{collection_name}/dataObjects/{movies[9]['id']}\",\n",
        "            \"data\": movies[9][\"data\"],\n",
        "            \"vectors\": movies[9][\"vectors\"],\n",
        "        }\n",
        "    },\n",
        "    {\n",
        "        \"data_object\": {\n",
        "            \"name\": f\"{collection_name}/dataObjects/{movies[10]['id']}\",\n",
        "            \"data\": movies[10][\"data\"],\n",
        "            \"vectors\": movies[10][\"vectors\"],\n",
        "        }\n",
        "    },\n",
        "    {\n",
        "        \"data_object\": {\n",
        "            \"name\": f\"{collection_name}/dataObjects/{movies[11]['id']}\",\n",
        "            \"data\": movies[11][\"data\"],\n",
        "            \"vectors\": movies[11][\"vectors\"],\n",
        "        }\n",
        "    },\n",
        "]\n",
        "request = vectorsearch_v1beta.BatchUpdateDataObjectsRequest(\n",
        "    parent=collection_name,\n",
        "    requests=requests,\n",
        ")\n",
        "data_object_service_client.batch_update_data_objects(request)\n",
        "# scifi = data_object_search_service_client.query_data_objects(\n",
        "#     request={\"parent\": collection_name, \"filter\": {\"genre\": {\"$eq\": \"Sci-Fi\"}}}\n",
        "# )\n",
        "# print([m.data[\"title\"] for m in scifi])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yvlHwQjgIs7v"
      },
      "source": [
        "## Create Data Object with Auto-Embeddings\n",
        "\n",
        "This requires the user has already enabled the Vertex Prediction API, as described in the `Enable APIs` section above."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "R2KXXql2Iue9"
      },
      "outputs": [],
      "source": [
        "movie_without_embedding = {\n",
        "    \"id\": \"the-matrix-2\",\n",
        "    \"data\": {\n",
        "        \"title\": \"The Matrix\",\n",
        "        \"genre\": \"Sci-Fi\",\n",
        "        \"year\": 2003,\n",
        "        \"director\": \"The Wachowskis\",\n",
        "    },\n",
        "}\n",
        "movies.append(movie_without_embedding)\n",
        "request = vectorsearch_v1beta.CreateDataObjectRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    data_object_id=movie_without_embedding[\"id\"],\n",
        "    data_object={\"data\": movie_without_embedding[\"data\"], \"vectors\": {}},\n",
        ")\n",
        "data_object_service_client.create_data_object(request=request)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B_-c9HImI6r9"
      },
      "source": [
        "## Query (list) data objects"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gq8lt1vwJZZg"
      },
      "outputs": [],
      "source": [
        "paged_response = data_object_search_service_client.query_data_objects(\n",
        "    vectorsearch_v1beta.QueryDataObjectsRequest(\n",
        "        parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "        page_size=2,\n",
        "        output_fields={\n",
        "            \"data_fields\": \"*\",\n",
        "            \"vector_fields\": \"*\",\n",
        "            \"metadata_fields\": \"*\",\n",
        "        },\n",
        "    )\n",
        ")\n",
        "page1 = next(paged_response.pages)\n",
        "next_page_token_1 = page1.next_page_token\n",
        "page1.data_objects"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iPlmRQgBJftf"
      },
      "outputs": [],
      "source": [
        "# Page 2\n",
        "paged_response_2 = data_object_search_service_client.query_data_objects(\n",
        "    vectorsearch_v1beta.QueryDataObjectsRequest(\n",
        "        parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "        page_size=2,\n",
        "        page_token=next_page_token_1,\n",
        "        output_fields={\n",
        "            \"data_fields\": \"*\",\n",
        "            \"vector_fields\": \"*\",\n",
        "            \"metadata_fields\": \"*\",\n",
        "        },\n",
        "    )\n",
        ")\n",
        "page2 = next(paged_response_2.pages)\n",
        "page2.data_objects"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4NYnmElNJkEk"
      },
      "source": [
        "## Query with filters"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "oHizcC4HJnMZ"
      },
      "outputs": [],
      "source": [
        "thrillers_request = vectorsearch_v1beta.QueryDataObjectsRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    filter={\"genre\": {\"$eq\": \"Thriller\"}},\n",
        "    output_fields=vectorsearch_v1beta.OutputFields(data_fields=[\"*\"]),\n",
        ")\n",
        "thrillers = data_object_search_service_client.query_data_objects(thrillers_request)\n",
        "print([m.data[\"title\"] for m in thrillers])\n",
        "\n",
        "thrillers_since_1995_request = vectorsearch_v1beta.QueryDataObjectsRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    filter={\"$and\": [{\"genre\": {\"$eq\": \"Thriller\"}}, {\"year\": {\"$gte\": 1995}}]},\n",
        "    output_fields=vectorsearch_v1beta.OutputFields(data_fields=[\"*\"]),\n",
        ")\n",
        "thrillers_since_1995 = data_object_search_service_client.query_data_objects(\n",
        "    thrillers_since_1995_request\n",
        ")\n",
        "list(thrillers_since_1995)\n",
        "\n",
        "nested_conditionals_request = vectorsearch_v1beta.QueryDataObjectsRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    filter={\n",
        "        \"$or\": [\n",
        "            {\"director\": {\"$eq\": \"Akira Kurosawa\"}},\n",
        "            {\n",
        "                \"$and\": [\n",
        "                    {\"director\": {\"$eq\": \"David Fincher\"}},\n",
        "                    {\"genre\": {\"$ne\": \"Thriller\"}},\n",
        "                ]\n",
        "            },\n",
        "        ]\n",
        "    },\n",
        "    output_fields=vectorsearch_v1beta.OutputFields(data_fields=[\"*\"]),\n",
        ")\n",
        "nested_conditionals = data_object_search_service_client.query_data_objects(\n",
        "    nested_conditionals_request\n",
        ")\n",
        "list(nested_conditionals)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VdQbEYvVKl5K"
      },
      "source": [
        "## Query with aggregates"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "daWCB6x7KpLE"
      },
      "outputs": [],
      "source": [
        "aggregate_request = vectorsearch_v1beta.AggregateDataObjectsRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    aggregate=\"COUNT\",\n",
        ")\n",
        "data_object_search_service_client.aggregate_data_objects(aggregate_request)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uf4IcjeDKryZ"
      },
      "source": [
        "## Search"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "l0IlxHlcKuT9"
      },
      "outputs": [],
      "source": [
        "search_request_1 = vectorsearch_v1beta.SearchDataObjectsRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    vector_search=vectorsearch_v1beta.VectorSearch(\n",
        "        search_field=\"genre_embedding\",\n",
        "        vector=vectorsearch_v1beta.DenseVector(values=normalize([0.1, 0.2, 0.3, 0.4])),\n",
        "        top_k=5,\n",
        "        output_fields=vectorsearch_v1beta.OutputFields(\n",
        "            data_fields=[\"*\"], vector_fields=[\"*\"], metadata_fields=[\"*\"]\n",
        "        ),\n",
        "    ),\n",
        ")\n",
        "results = data_object_search_service_client.search_data_objects(search_request_1)\n",
        "for result in results:\n",
        "    print(result.data_object)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1gEgArXcK-l4"
      },
      "outputs": [],
      "source": [
        "search_request_2 = vectorsearch_v1beta.SearchDataObjectsRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    vector_search=vectorsearch_v1beta.VectorSearch(\n",
        "        search_field=\"soundtrack_embedding\",\n",
        "        vector=vectorsearch_v1beta.DenseVector(\n",
        "            values=normalize([0.1, 0.1, 0.1, 0.1, 0.1])\n",
        "        ),\n",
        "        top_k=5,\n",
        "        output_fields=vectorsearch_v1beta.OutputFields(\n",
        "            data_fields=[\"*\"], vector_fields=[\"*\"], metadata_fields=[\"*\"]\n",
        "        ),\n",
        "    ),\n",
        ")\n",
        "results = data_object_search_service_client.search_data_objects(search_request_2)\n",
        "for result in results:\n",
        "    print(result.data_object)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v7woZlw5L_3w"
      },
      "source": [
        "## Search with filters"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fjvJiGrsMBw8"
      },
      "outputs": [],
      "source": [
        "search_filter_request_1 = vectorsearch_v1beta.SearchDataObjectsRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    vector_search=vectorsearch_v1beta.VectorSearch(\n",
        "        search_field=\"plot_embedding\",\n",
        "        vector=vectorsearch_v1beta.DenseVector(values=normalize([0.3, 0.4, 0.5])),\n",
        "        filter={\"genre\": {\"$eq\": \"Thriller\"}},\n",
        "        top_k=5,\n",
        "        output_fields=vectorsearch_v1beta.OutputFields(\n",
        "            data_fields=[\"*\"], vector_fields=[\"*\"], metadata_fields=[\"*\"]\n",
        "        ),\n",
        "    ),\n",
        ")\n",
        "results = data_object_search_service_client.search_data_objects(search_filter_request_1)\n",
        "list(results)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZZ4xQQyDMD-q"
      },
      "outputs": [],
      "source": [
        "search_filter_request_2 = vectorsearch_v1beta.SearchDataObjectsRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    vector_search=vectorsearch_v1beta.VectorSearch(\n",
        "        search_field=\"plot_embedding\",\n",
        "        vector=vectorsearch_v1beta.DenseVector(values=normalize([0.3, 0.4, 0.5])),\n",
        "        filter={\n",
        "            \"$and\": [\n",
        "                {\"genre\": {\"$eq\": \"Thriller\"}},\n",
        "                {\"director\": {\"$eq\": \"David Fincher\"}},\n",
        "            ]\n",
        "        },\n",
        "        top_k=5,\n",
        "        output_fields=vectorsearch_v1beta.OutputFields(\n",
        "            data_fields=[\"*\"], vector_fields=[\"*\"], metadata_fields=[\"*\"]\n",
        "        ),\n",
        "    ),\n",
        ")\n",
        "results = data_object_search_service_client.search_data_objects(search_filter_request_2)\n",
        "list(results)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QVCS25RBMNjc"
      },
      "outputs": [],
      "source": [
        "search_filter_request_3 = vectorsearch_v1beta.SearchDataObjectsRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    vector_search=vectorsearch_v1beta.VectorSearch(\n",
        "        search_field=\"plot_embedding\",\n",
        "        vector=vectorsearch_v1beta.DenseVector(values=normalize([0.3, 0.4, 0.5])),\n",
        "        filter={\n",
        "            \"$and\": [\n",
        "                {\"genre\": {\"$eq\": \"Thriller\"}},\n",
        "                {\"director\": {\"$eq\": \"David Fincher\"}},\n",
        "                {\"title\": {\"$ne\": \"Seven\"}},\n",
        "            ]\n",
        "        },\n",
        "        top_k=5,\n",
        "        output_fields=vectorsearch_v1beta.OutputFields(\n",
        "            data_fields=[\"*\"], vector_fields=[\"*\"], metadata_fields=[\"*\"]\n",
        "        ),\n",
        "    ),\n",
        ")\n",
        "results = data_object_search_service_client.search_data_objects(search_filter_request_3)\n",
        "list(results)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q-AbPHrEMQaf"
      },
      "source": [
        "## Semantic Search"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Qtpqr_MVMRrw"
      },
      "outputs": [],
      "source": [
        "semantic_search_request = vectorsearch_v1beta.SearchDataObjectsRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    semantic_search=vectorsearch_v1beta.SemanticSearch(\n",
        "        search_text=\"Wonderful genre of a Wonderful movie\",\n",
        "        search_field=\"genre_embedding\",\n",
        "        task_type=\"RETRIEVAL_QUERY\",\n",
        "        top_k=5,\n",
        "        output_fields=vectorsearch_v1beta.OutputFields(\n",
        "            data_fields=[\"*\"], vector_fields=[\"*\"], metadata_fields=[\"*\"]\n",
        "        ),\n",
        "    ),\n",
        ")\n",
        "results = data_object_search_service_client.search_data_objects(semantic_search_request)\n",
        "for result in results:\n",
        "    print(result.data_object)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6Nc9O5FHMvY2"
      },
      "source": [
        "## Text Search"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jx3lejMPMwVJ"
      },
      "outputs": [],
      "source": [
        "text_search_request = vectorsearch_v1beta.SearchDataObjectsRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    text_search=vectorsearch_v1beta.TextSearch(\n",
        "        search_text=\"king OR castle\",\n",
        "        data_field_names=[\"title\"],\n",
        "        top_k=5,\n",
        "        output_fields=vectorsearch_v1beta.OutputFields(\n",
        "            data_fields=[\"*\"], vector_fields=[\"*\"], metadata_fields=[\"*\"]\n",
        "        ),\n",
        "    ),\n",
        ")\n",
        "results = data_object_search_service_client.search_data_objects(text_search_request)\n",
        "for result in results:\n",
        "    print(result.data_object)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3X2TuDEnNvm2"
      },
      "source": [
        "## Batch Search"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5fM5JcoBNyAH"
      },
      "outputs": [],
      "source": [
        "batch_request_1 = vectorsearch_v1beta.BatchSearchDataObjectsRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    searches=[\n",
        "        vectorsearch_v1beta.Search(\n",
        "            vector_search=vectorsearch_v1beta.VectorSearch(\n",
        "                search_field=\"genre_embedding\",\n",
        "                vector=vectorsearch_v1beta.DenseVector(values=[0.1, 0.2, 0.3, 0.4]),\n",
        "                top_k=5,\n",
        "                output_fields=vectorsearch_v1beta.OutputFields(data_fields=[\"*\"]),\n",
        "            )\n",
        "        ),\n",
        "        vectorsearch_v1beta.Search(\n",
        "            vector_search=vectorsearch_v1beta.VectorSearch(\n",
        "                search_field=\"soundtrack_embedding\",\n",
        "                vector=vectorsearch_v1beta.DenseVector(\n",
        "                    values=[0.1, 0.1, 0.1, 0.1, 0.1]\n",
        "                ),\n",
        "                top_k=5,\n",
        "                output_fields=vectorsearch_v1beta.OutputFields(data_fields=[\"*\"]),\n",
        "            )\n",
        "        ),\n",
        "        vectorsearch_v1beta.Search(\n",
        "            vector_search=vectorsearch_v1beta.VectorSearch(\n",
        "                search_field=\"plot_embedding\",\n",
        "                vector=vectorsearch_v1beta.DenseVector(values=[0.3, 0.4, 0.5]),\n",
        "                top_k=5,\n",
        "                output_fields=vectorsearch_v1beta.OutputFields(data_fields=[\"*\"]),\n",
        "            )\n",
        "        ),\n",
        "    ],\n",
        ")\n",
        "data_object_search_service_client.batch_search_data_objects(batch_request_1)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bvhd0j-PN0u0"
      },
      "outputs": [],
      "source": [
        "batch_request_2 = vectorsearch_v1beta.BatchSearchDataObjectsRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    searches=[\n",
        "        vectorsearch_v1beta.Search(\n",
        "            vector_search=vectorsearch_v1beta.VectorSearch(\n",
        "                search_field=\"genre_embedding\",\n",
        "                vector=vectorsearch_v1beta.DenseVector(values=[0.1, 0.2, 0.3, 0.4]),\n",
        "                top_k=5,\n",
        "                output_fields=vectorsearch_v1beta.OutputFields(data_fields=[\"*\"]),\n",
        "            )\n",
        "        ),\n",
        "        vectorsearch_v1beta.Search(\n",
        "            vector_search=vectorsearch_v1beta.VectorSearch(\n",
        "                search_field=\"soundtrack_embedding\",\n",
        "                vector=vectorsearch_v1beta.DenseVector(\n",
        "                    values=[0.1, 0.1, 0.1, 0.1, 0.1]\n",
        "                ),\n",
        "                top_k=5,\n",
        "                output_fields=vectorsearch_v1beta.OutputFields(data_fields=[\"*\"]),\n",
        "            )\n",
        "        ),\n",
        "        vectorsearch_v1beta.Search(\n",
        "            vector_search=vectorsearch_v1beta.VectorSearch(\n",
        "                search_field=\"plot_embedding\",\n",
        "                vector=vectorsearch_v1beta.DenseVector(values=[0.3, 0.4, 0.5]),\n",
        "                top_k=5,\n",
        "                output_fields=vectorsearch_v1beta.OutputFields(data_fields=[\"*\"]),\n",
        "            )\n",
        "        ),\n",
        "    ],\n",
        "    combine=vectorsearch_v1beta.BatchSearchDataObjectsRequest.CombineResultsOptions(\n",
        "        ranker=vectorsearch_v1beta.Ranker(\n",
        "            rrf=vectorsearch_v1beta.ReciprocalRankFusion(weights=[1.0, 1.0, 1.0])\n",
        "        )\n",
        "    ),\n",
        ")\n",
        "data_object_search_service_client.batch_search_data_objects(batch_request_2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PHFGTj_5Q7uK"
      },
      "source": [
        "## Create ANN Index"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "C4zKyCJSQ-mM"
      },
      "outputs": [],
      "source": [
        "request = vectorsearch_v1beta.CreateIndexRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    index_id=\"plot_index\",\n",
        "    index={\n",
        "        \"index_field\": \"plot_embedding\",\n",
        "        \"filter_fields\": [\"year\", \"genre\"],\n",
        "        \"store_fields\": [\"title\"],\n",
        "    },\n",
        ")\n",
        "dense_index_lro = vector_search_service_client.create_index(request)\n",
        "dense_index_operation_name = dense_index_lro.operation.name\n",
        "dense_index_operation_name"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dyE1N0tgS1iG"
      },
      "source": [
        "## Create Sparse ANN Index"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "u5LVYZXdS4CF"
      },
      "outputs": [],
      "source": [
        "request = vectorsearch_v1beta.CreateIndexRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    index_id=\"sparse_index\",\n",
        "    index={\n",
        "        \"index_field\": \"sparse_embedding\",\n",
        "        \"filter_fields\": [\"year\", \"genre\"],\n",
        "        \"store_fields\": [\"title\"],\n",
        "    },\n",
        ")\n",
        "sparse_index_lro = vector_search_service_client.create_index(request)\n",
        "sparse_index_operation_name = sparse_index_lro.operation.name\n",
        "sparse_index_operation_name"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8kmxG_AHTLhI"
      },
      "source": [
        "## Poll LROs\n",
        "\n",
        "Index creation operations typically take several minutes or more to complete. The progress can be polled via the operation LROs:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8Fvw-J5fTEZP"
      },
      "outputs": [],
      "source": [
        "print(f\"Waiting for dense index LRO: {dense_index_lro.operation.name}\")\n",
        "dense_index_lro.result()\n",
        "print(\"Dense index ready.\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "No0wKmPaTBIe"
      },
      "outputs": [],
      "source": [
        "print(f\"Waiting for sparse index LRO: {sparse_index_lro.operation.name}\")\n",
        "sparse_index_lro.result()\n",
        "print(\"Sparse index ready.\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nMM49A5nTRTW"
      },
      "source": [
        "## Get ANN Index"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "HK2SugXwTVKI"
      },
      "outputs": [],
      "source": [
        "request = vectorsearch_v1beta.GetIndexRequest(\n",
        "    name=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}/indexes/plot_index\"\n",
        ")\n",
        "vector_search_service_client.get_index(request)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kd9Xc1TyTXZ3"
      },
      "source": [
        "## List ANN Indexes"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_ibj1MuQTYZo"
      },
      "outputs": [],
      "source": [
        "request = vectorsearch_v1beta.ListIndexesRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\"\n",
        ")\n",
        "vector_search_service_client.list_indexes(request)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YUvdsaasksj6"
      },
      "source": [
        "## Delete Indexes"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LvhXbWXDktmR"
      },
      "outputs": [],
      "source": [
        "request = vectorsearch_v1beta.DeleteIndexRequest(\n",
        "    name=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}/indexes/plot_index\"\n",
        ")\n",
        "delete_index_lro = vector_search_service_client.delete_index(request)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JDh8zTeNlNUT"
      },
      "outputs": [],
      "source": [
        "request = vectorsearch_v1beta.DeleteIndexRequest(\n",
        "    name=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}/indexes/sparse_index\"\n",
        ")\n",
        "delete_sparse_index_lro = vector_search_service_client.delete_index(request)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "N7WsQEwIlNgH"
      },
      "outputs": [],
      "source": [
        "print(f\"Waiting for dense index deletion LRO: {delete_index_lro.operation.name}\")\n",
        "delete_index_lro.result()\n",
        "print(\"Dense index deleted.\")\n",
        "\n",
        "print(\n",
        "    f\"Waiting for sparse index deletion LRO: {delete_sparse_index_lro.operation.name}\"\n",
        ")\n",
        "delete_sparse_index_lro.result()\n",
        "print(\"Sparse index deleted.\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0MFmsip8N91c"
      },
      "source": [
        "## Delete data objects"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mnVQI1gHOANw"
      },
      "outputs": [],
      "source": [
        "request = vectorsearch_v1beta.QueryDataObjectsRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    filter={\"$or\": [{\"title\": {\"$eq\": \"Seven\"}}, {\"title\": {\"$eq\": \"Se7en\"}}]},\n",
        "    output_fields=vectorsearch_v1beta.OutputFields(data_fields=[\"*\"]),\n",
        ")\n",
        "data_object_search_service_client.query_data_objects(request)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fEqfffm1OC3L"
      },
      "outputs": [],
      "source": [
        "delete_request = vectorsearch_v1beta.DeleteDataObjectRequest(\n",
        "    name=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}/dataObjects/seven\"\n",
        ")\n",
        "data_object_service_client.delete_data_object(delete_request)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2MSDfzgVOFWF"
      },
      "outputs": [],
      "source": [
        "request = vectorsearch_v1beta.SearchDataObjectsRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    vector_search=vectorsearch_v1beta.VectorSearch(\n",
        "        search_field=\"plot_embedding\",\n",
        "        vector=vectorsearch_v1beta.DenseVector(values=normalize([0.3, 0.4, 0.5])),\n",
        "        filter={\n",
        "            \"$and\": [\n",
        "                {\"genre\": {\"$eq\": \"Thriller\"}},\n",
        "                {\"director\": {\"$eq\": \"David Fincher\"}},\n",
        "            ]\n",
        "        },\n",
        "        output_fields=vectorsearch_v1beta.OutputFields(data_fields=[\"*\"]),\n",
        "    ),\n",
        ")\n",
        "data_object_search_service_client.search_data_objects(request)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-sO3OV4COZEg"
      },
      "source": [
        "## Batch Delete data objects"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NVZhkhTuOdFZ"
      },
      "outputs": [],
      "source": [
        "request = vectorsearch_v1beta.QueryDataObjectsRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    filter={\"$or\": [{\"genre\": {\"$eq\": \"Sci-Fi\"}}]},\n",
        "    output_fields=vectorsearch_v1beta.OutputFields(data_fields=[\"*\"]),\n",
        ")\n",
        "data_object_search_service_client.query_data_objects(request)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NSy_ZmtBOhhZ"
      },
      "outputs": [],
      "source": [
        "batch_delete_request = vectorsearch_v1beta.BatchDeleteDataObjectsRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    requests=[\n",
        "        vectorsearch_v1beta.DeleteDataObjectRequest(\n",
        "            name=f\"{collection_name}/dataObjects/the-matrix\"\n",
        "        ),\n",
        "        vectorsearch_v1beta.DeleteDataObjectRequest(\n",
        "            name=f\"{collection_name}/dataObjects/inception\"\n",
        "        ),\n",
        "    ],\n",
        ")\n",
        "data_object_service_client.batch_delete_data_objects(batch_delete_request)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "69_O9WbDOnKs"
      },
      "outputs": [],
      "source": [
        "request = vectorsearch_v1beta.QueryDataObjectsRequest(\n",
        "    parent=f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\",\n",
        "    filter={\"$or\": [{\"genre\": {\"$eq\": \"Sci-Fi\"}}]},\n",
        "    output_fields=vectorsearch_v1beta.OutputFields(data_fields=[\"*\"]),\n",
        ")\n",
        "data_object_search_service_client.query_data_objects(request)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-aE0ZTxOOruh"
      },
      "source": [
        "## Clean up Collection\n",
        "\n",
        "Please also delete any ANN indexes, as described in the `Delete Indexes` section above."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ezl_ZrdOOuaD"
      },
      "outputs": [],
      "source": [
        "collection_name = (\n",
        "    f\"projects/{PROJECT_ID}/locations/{LOCATION}/collections/{collection_id}\"\n",
        ")\n",
        "for movie in movies:\n",
        "    request = vectorsearch_v1beta.DeleteDataObjectRequest(\n",
        "        name=f\"{collection_name}/dataObjects/{movie['id']}\"\n",
        "    )\n",
        "    try:\n",
        "        data_object_service_client.delete_data_object(request)\n",
        "    except:\n",
        "        pass\n",
        "request = vectorsearch_v1beta.DeleteCollectionRequest(name=collection_name)\n",
        "vector_search_service_client.delete_collection(request).result()"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "vector-search-2-quickstart.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
