{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Interactive Visualization\n", "\n", "This tutorial demonstrates the interactive visualization tools in `opensoundscape.visualization`.\n", "These tools are designed for use in Jupyter or VS Code notebooks and provide:\n", "\n", "- **`inspect`**: Display a grid of spectrograms with click-to-play audio\n", "- **`annotate`**: Like `inspect`, but with toggle buttons to label clips in-place\n", "- **`explore_features`**: Interactive scatter plot that shows spectrograms for selected points\n", "- **`explore_histogram`**: Interactive histogram with per-label toggles and audio inspection\n", "\n", "### Requirements\n", "\n", "These functions require the optional `viz` dependencies:\n", "```bash\n", "pip install opensoundscape[viz]\n", "```\n", "This installs `ipywidgets` and `plotly`." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "# if this is a Google Colab notebook, install opensoundscape in the runtime environment\n", "if 'google.colab' in str(get_ipython()):\n", " %pip install \"opensoundscape[viz]\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup\n", "\n", "Download a sample audio file and create a DataFrame of clips to work with." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from opensoundscape import Audio\n", "from opensoundscape.visualization import (\n", " inspect,\n", " annotate,\n", " explore_features,\n", " explore_histogram,\n", ")\n", "import opensoundscape as opso\n", "\n", "import numpy as np\n", "import pandas as pd\n", "from pathlib import Path" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Audio duration: 60.0s, sample rate: 32000 Hz\n" ] } ], "source": [ "# Download a 60-second birdsong recording\n", "url = \"https://tinyurl.com/birds60s\"\n", "audio = Audio.from_url(url)\n", "\n", "# Save locally so visualization functions can load clips by file path\n", "audio_path = Path(\"demo_audio.wav\")\n", "audio.save(audio_path)\n", "\n", "print(f\"Audio duration: {audio.duration:.1f}s, sample rate: {audio.sample_rate} Hz\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
filestart_timeend_time
demo_audio.wav0.03.0
1.04.0
2.05.0
3.06.0
4.07.0
\n", "
" ], "text/plain": [ "Empty DataFrame\n", "Columns: []\n", "Index: [(demo_audio.wav, 0.0, 3.0), (demo_audio.wav, 1.0, 4.0), (demo_audio.wav, 2.0, 5.0), (demo_audio.wav, 3.0, 6.0), (demo_audio.wav, 4.0, 7.0)]" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Create a DataFrame of 3-second clips spanning the recording in 1-second steps\n", "clip_df = opso.utils.make_clip_df(audio_path, clip_duration=3.0, clip_overlap=2.0)\n", "\n", "clip_df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## `inspect`: View and listen to clips\n", "\n", "`inspect` displays a grid of spectrograms. Click any spectrogram to play its audio." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", "\n", "
\n", " \n", "
\n", " \n", " \n", "
\n", " \n", "
\n", " \n", " \n", "
\n", " \n", "
\n", " \n", " \n", "
\n", " \n", "
\n", " \n", " \n", "
\n", " \n", "
\n", " \n", " \n", "
\n", " \n", "
\n", " \n", " \n", "
\n", " \n", "
\n", " \n", " \n", "
\n", " \n", "
\n", " \n", " \n", "
\n", " \n", "
\n", " \n", " \n", "
\n", " \n", "
\n", " \n", " \n", "
\n", " \n", "
\n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "widget = inspect(\n", " clip_df,\n", " N=10,\n", " bandpass_range=(500, 10000),\n", " cell_width=180,\n", " cell_height=150,\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## `annotate`: Label clips interactively\n", "\n", "`annotate` displays spectrograms with toggle buttons below each clip.\n", "Clicking a button sets `clip_df.at[row_index, button_name] = True`;\n", "clicking again sets it to `None`. The DataFrame is modified **in-place**." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "a9ea9cc61ee940079be23c5e9a9cd211", "version_major": 2, "version_minor": 0 }, "text/plain": [ "GridBox(children=(VBox(children=(HTML(value='\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
filestart_timeend_timeWood ThrushBlack-and-White Warbler
47demo_audio.wav47.050.0NoneNone
22demo_audio.wav22.025.0NoneNone
20demo_audio.wav20.023.0NoneNone
4demo_audio.wav4.07.0TrueNone
26demo_audio.wav26.029.0TrueNone
\n", "" ], "text/plain": [ " file start_time end_time Wood Thrush Black-and-White Warbler\n", "47 demo_audio.wav 47.0 50.0 None None\n", "22 demo_audio.wav 22.0 25.0 None None\n", "20 demo_audio.wav 20.0 23.0 None None\n", "4 demo_audio.wav 4.0 7.0 True None\n", "26 demo_audio.wav 26.0 29.0 True None" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# After toggling some buttons above, check the annotations\n", "clips_to_label.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Generate synthetic features for demo\n", "\n", "For `explore_features` and `explore_histogram`, we need a DataFrame with\n", "numeric feature columns, scores, and category labels. We generate random\n", "values here for demonstration." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
feature_xfeature_yscorecategory
filestart_timeend_time
demo_audio.wav0.03.00.496714-0.4791740.237638noise
1.04.0-0.138264-0.1856590.728216noise
2.05.00.647689-1.1063350.367783song
3.06.01.523030-1.1962070.632306song
4.07.0-0.2341530.8125260.633530noise
\n", "
" ], "text/plain": [ " feature_x feature_y score category\n", "file start_time end_time \n", "demo_audio.wav 0.0 3.0 0.496714 -0.479174 0.237638 noise\n", " 1.0 4.0 -0.138264 -0.185659 0.728216 noise\n", " 2.0 5.0 0.647689 -1.106335 0.367783 song\n", " 3.0 6.0 1.523030 -1.196207 0.632306 song\n", " 4.0 7.0 -0.234153 0.812526 0.633530 noise" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.random.seed(42)\n", "\n", "# Add random 2D features (e.g. from UMAP or PCA of audio embeddings)\n", "clip_df[\"feature_x\"] = np.random.randn(len(clip_df))\n", "clip_df[\"feature_y\"] = np.random.randn(len(clip_df))\n", "\n", "# Add a random score (e.g. classifier confidence)\n", "clip_df[\"score\"] = np.random.uniform(0, 1, len(clip_df))\n", "\n", "# Add a random category label\n", "clip_df[\"category\"] = np.random.choice([\"song\", \"call\", \"noise\"], size=len(clip_df))\n", "\n", "clip_df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## `explore_features`: Interactive scatter plot\n", "\n", "`explore_features` shows a Plotly scatter plot of your data. Use box select\n", "or lasso select to highlight points, and spectrograms for the selected clips\n", "will appear below the plot.\n", "\n", "Use the `color_col` argument to color points by a categorical column." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "194eb5986df84387a05758d3f2ea83bb", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Output()" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "4724ba2361084ed8861be4adf4db1d45", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Button(button_style='info', description='Inspect selected', icon='search', style=ButtonStyle())" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "cf25fbc3d1de41ada0374d137ad7fc21", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Output()" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fw = explore_features(\n", " clip_df,\n", " x_col=\"feature_x\",\n", " y_col=\"feature_y\",\n", " color_col=\"category\",\n", " # kwargs below are passed to inspect() for selected points\n", " N=6,\n", " bandpass_range=(500, 10000),\n", " cell_width=150,\n", " cell_height=120,\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## `explore_histogram`: Interactive histogram\n", "\n", "`explore_histogram` displays overlaid histograms of a numeric column, split by\n", "a label column. Each label gets a color-matched toggle button to show/hide its\n", "histogram. Click \"Inspect random selection\" to view spectrograms of clips in\n", "the current x-axis range." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "5dbdca94ef484b5c86d50c817d16e3f3", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(FigureWidget({\n", " 'data': [{'marker': {'color': 'rgb(127, 60, 141)'},\n", " 'name': 'n…" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fw = explore_histogram(\n", " clip_df,\n", " value_col=\"score\",\n", " label_col=\"category\",\n", " bins=15,\n", " # kwargs below are passed to inspect()\n", " N=6,\n", " bandpass_range=(500, 10000),\n", " cell_width=150,\n", " cell_height=120,\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Clean up\n", "\n", "Uncomment and run this cell to remove the temporary audio file created for this tutorial." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# audio_path.unlink()\n", "# print(\"Cleaned up temporary files.\")" ] } ], "metadata": { "kernelspec": { "display_name": "opso_dev", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.5" } }, "nbformat": 4, "nbformat_minor": 4 }