{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "sWRmbbv-L2-P"
      },
      "outputs": [],
      "source": [
        "# Copyright 2025 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "En_XtJCvMM9p"
      },
      "source": [
        "# Lyria 2 Music Generation\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/audio/music/getting-started/lyria2_music_generation.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Faudio%2Fmusic%2Fgetting-started%2Flyria2_music_generation.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Run in Colab Enterprise\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/audio/music/getting-started/lyria2_music_generation.ipynb\">\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br> Open in Vertex AI Workbench\n",
        "    </a>\n",
        "  </td>    \n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/audio/music/getting-started/lyria2_music_generation.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "</table>\n",
        "\n",
        "<div style=\"clear: both;\"></div>\n",
        "\n",
        "<b>Share to:</b>\n",
        "\n",
        "<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/audio/music/getting-started/lyria2_music_generation.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/audio/music/getting-started/lyria2_music_generation.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/audio/music/getting-started/lyria2_music_generation.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/audio/music/getting-started/lyria2_music_generation.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/audio/music/getting-started/lyria2_music_generation.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
        "</a>            \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hhcj6hs9NZwK"
      },
      "source": [
        "| Author |\n",
        "| --- |\n",
        "| [Katie Nguyen](https://github.com/katiemn) |"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7E3uyb0nNeWK"
      },
      "source": [
        "## Overview\n",
        "\n",
        "### Lyria 2\n",
        "\n",
        "Lyria 2 on Vertex AI is Google's latest music generation model. It is capable of generating high-fidelity audio tracks across a range of genres, developed with input from musicians and producers. Learn more about [Lyria on Vertex AI](https://cloud.google.com/vertex-ai/generative-ai/docs/music/generate-music)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FEkefaBDNi8z"
      },
      "source": [
        "In this tutorial, you will learn how to use the Vertex AI API to interact with Lyria 2 to generate music from text prompts that showcase:\n",
        "\n",
        "- Various styles and genres\n",
        "- How to play with moods and emotions in audio\n",
        "- Different tempos and instrumentation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DCCTrgzhNnC8"
      },
      "source": [
        "## Get started"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L4KHiioVN4oT"
      },
      "source": [
        "### Authenticate your notebook environment (Colab only)\n",
        "\n",
        "If you are running this notebook on Google Colab, run the following cell to authenticate your environment."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "eoWlrJrcN7xV"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "    from google.colab import auth\n",
        "\n",
        "    auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ks3WK7PcN-u3"
      },
      "source": [
        "### Set Google Cloud project information\n",
        "\n",
        "To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
        "\n",
        "Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "YIWKb9qIOCO7"
      },
      "outputs": [],
      "source": [
        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "On90JdAwOOvF"
      },
      "source": [
        "### Import libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "qoWFxpNYOUQB"
      },
      "outputs": [],
      "source": [
        "import base64\n",
        "\n",
        "from IPython.display import Audio\n",
        "import google.auth\n",
        "import google.auth.transport.requests\n",
        "import requests"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OAcSUy1sOVGg"
      },
      "source": [
        "### Define helper functions"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "g_Tlg194nWxr"
      },
      "outputs": [],
      "source": [
        "def send_request_to_google_api(api_endpoint, data=None):\n",
        "    \"\"\"\n",
        "    Sends an HTTP request to a Google API endpoint.\n",
        "\n",
        "    Args:\n",
        "        api_endpoint: The URL of the Google API endpoint.\n",
        "        data: (Optional) Dictionary of data to send in the request body (for POST, PUT, etc.).\n",
        "\n",
        "    Returns:\n",
        "        The response from the Google API.\n",
        "    \"\"\"\n",
        "\n",
        "    # Get access token calling API\n",
        "    creds, project = google.auth.default()\n",
        "    auth_req = google.auth.transport.requests.Request()\n",
        "    creds.refresh(auth_req)\n",
        "    access_token = creds.token\n",
        "\n",
        "    headers = {\n",
        "        \"Authorization\": f\"Bearer {access_token}\",\n",
        "        \"Content-Type\": \"application/json\",\n",
        "    }\n",
        "\n",
        "    response = requests.post(api_endpoint, headers=headers, json=data)\n",
        "    response.raise_for_status()\n",
        "    return response.json()\n",
        "\n",
        "\n",
        "def generate_music(request: dict):\n",
        "    req = {\"instances\": [request], \"parameters\": {}}\n",
        "    print(req)\n",
        "    resp = send_request_to_google_api(music_model, req)\n",
        "    return resp[\"predictions\"]\n",
        "\n",
        "\n",
        "def play_audio(preds):\n",
        "    for pred in preds:\n",
        "        bytes_b64 = dict(pred)[\"bytesBase64Encoded\"]\n",
        "        decoded_audio_data = base64.b64decode(bytes_b64)\n",
        "\n",
        "        audio = Audio(decoded_audio_data, rate=48000, autoplay=False)\n",
        "        display(audio)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JLXmJDmyIa0V"
      },
      "source": [
        "### Load the audio model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "mX7zMNnYIeXG"
      },
      "outputs": [],
      "source": [
        "music_model = f\"https://us-central1-aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/us-central1/publishers/google/models/lyria-002:predict\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iRyJ_-I8D2RV"
      },
      "source": [
        "## Generate music from text prompts"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wsGInu4HVXz1"
      },
      "source": [
        "### Explore various genres\n",
        "\n",
        "When prompting Lyria 2 it's helpful to consider the overall **style** of music you want to generate. Consider options such as: classical, electronic, rock, jazz, hip hop, or pop. You can even describe more general styles that include cinematic, ambient, or lo-fi.\n",
        "\n",
        "\n",
        "With Lyria 2, you can generate a 30 second WAV audio at a 48kHz sample rate from a text prompt. In order to generate an audio clip in the following sample, specify the following info:\n",
        "- **Prompt:** A detailed description of the music you would like to generate.\n",
        "- **Negative prompt:** Optionally, you can specify a description of what to exclude from the generated audio.\n",
        "- **Sample count:** The number of audio samples to generate."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_AO0cKS_FuZZ"
      },
      "outputs": [],
      "source": [
        "prompt = \"Smooth, atmospheric jazz. Moderate tempo, rich harmonies. Featuring mellow brass\"  # @param {type:\"string\"}\n",
        "negative_prompt = \"fast\"  # @param {type:\"string\"}\n",
        "sample_count = 2  # @param {type:\"number\"}\n",
        "\n",
        "music = generate_music(\n",
        "    {\"prompt\": prompt, \"negative_prompt\": negative_prompt, \"sample_count\": sample_count}\n",
        ")\n",
        "play_audio(music)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vo2odCKUSlWw"
      },
      "source": [
        "### Describe mood and emotion\n",
        "\n",
        "Another attribute to consider is the mood or **emotion** you want in your generated music. Describe the desired feeling or atmosphere with words like happy, melancholy, energetic, calm, tense, or dreamy.\n",
        "\n",
        "When generating music with Lyria 2 you can also set a seed value for deterministic generation. If provided, the model will attempt to produce the same audio with the same prompt and other parameters.\n",
        "\n",
        "**Note:** `seed` and `sample_count` cannot be set in the same request."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OlqK32nNuvof"
      },
      "outputs": [],
      "source": [
        "prompt = \"Dramatic dance symphony\"  # @param {type:\"string\"}\n",
        "negative_prompt = \"\"  # @param {type:\"string\"}\n",
        "seed = 111  # @param {type:\"number\"}\n",
        "\n",
        "music = generate_music(\n",
        "    {\"prompt\": prompt, \"negative_prompt\": negative_prompt, \"seed\": seed}\n",
        ")\n",
        "play_audio(music)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W57GFxtcS87E"
      },
      "source": [
        "### Play with tempo and instrumentation\n",
        "\n",
        "It can also help to include descriptions of the tempo or **rhythm**, such as fast, slow, or syncopated. You might also include specific **instruments** such as a piano, synthesizer, acoustic guitar, drums, strings, or flute.\n",
        "\n",
        "By default, all music generated with Lyria utilizes [SynthID](https://deepmind.google/technologies/synthid/), a technology that embeds an inaudible watermark directly into its waveform.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8PUfbdkWTL_-"
      },
      "outputs": [],
      "source": [
        "prompt = \"Acoustic guitar melody with a fast tempo\"  # @param {type:\"string\"}\n",
        "\n",
        "music = generate_music({\"prompt\": prompt})\n",
        "play_audio(music)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "lyria2_music_generation.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
