OpenSoundscape

OpenSoundscape is free and open source software for the analysis of bioacoustic recordings (GitHub). Its main goals are to allow users to train their own custom species classification models using a variety of frameworks (including convolutional neural networks) and to use trained models to predict whether species are present in field recordings. OpSo can be installed and run on a single computer or in a cluster or cloud environment.

OpenSoundcape is developed and maintained by the Kitzes Lab at the University of Pittsburgh.

The Installation section below provides guidance on installing OpSo. The Tutorials pages below are written as Jupyter Notebooks that can also be downloaded from the project repository on GitHub.

Installation

• Mac and Linux
  • Installation via Anaconda
  • Installation via venv
• Windows
  • Get Ubuntu shell
  • Download Anaconda
  • Install OpenSoundscape in virtual environment
• Contributors
  • Poetry installation
  • Contribution workflow
• Jupyter
  • Use virtual environment
  • Create independent kernel

Tutorials

• Audio and spectrograms
  • Quick start
  • Audio loading
  • Audio methods
  • Spectrogram creation
  • Spectrogram methods
• Manipulating audio annotations
  • Download example files
  • View a subset of annotations
  • Saving annotations to Raven-compatible file
  • 1. Split Audio object, then split annotations to match
  • Split annotations using splitting parameters
  • Match up audio files and Raven annotations
  • Create label dataframes
  • Load Raven annotations
  • Create label dataframes
  • Sanity check: look at spectrograms of clips labeled 0 and 1
• Prediction with pre-trained CNNs
  • Load required packages
  • generate predictions with the model
  • Overlapping prediction clips
  • Inspect samples generated during prediction
  • Options for prediction
  • Using models from older OpenSoundscape versions
• Beginner friendly training and prediction with CNNs
  • Prepare audio data
  • Create and train a model
  • Visualize gradient activation maps
  • Prediction
  • Multi-class models
  • Save and load models
  • Predict using saved (or pre-trained) model
  • Continue training from saved model
  • Next steps
• Preprocessing audio samples with OpenSoundscape
  • Modifying the preprocessor of the CNN class
  • Download labeled audio files
  • Load dataframe of files and labels
  • Initialize preprocessor
  • Generate a sample from a Dataset
  • Subset samples from a Dataset
  • About Pipelines
  • About actions
  • View default parameters for an Action
  • Modify Action parameters
  • Bypass actions
  • Example: return Spectrogram instead of Tensor
  • analyse the output at steps of interest
  • Modify the sample rate
  • Modify spectrogram window length and overlap
  • Bandpass spectrograms
  • Change the output shape
  • Turn all augmentation on or off
  • Modify augmentation parameters
  • remove an action by its name
  • add an action at a specific position
  • Overlay augmentation
• Advanced CNN training
  • Prepare audio data
  • Creating a model
  • Model training parameters
  • Selecting CNN architectures
  • Multi-target training with ResampleLoss
  • Training and predicting with custom preprocessors
• RIBBIT Pulse Rate model demonstration
  • Import packages
  • Download example audio
  • Select model parameters
  • Search for pulsing vocalizations with ribbit()
  • Analyzing a set of files
  • Run RIBBIT on multiple species simultaneously
  • Detail view of RIBBIT method
  • Time to experiment for yourself

API Documentation

• Spectrogram
• CNN
• Annotations
• ARU
• Machine Learning utils
• CNN Architectures
• Logging with WandB (Weights and Biases)
• Data Selection
• Datasets
• CAM (Class Activation Maps)
• Loss
• Safe Dataset
• Sample
• Sampling
• Metrics
• Image Augmentation
• Actions
• Preprocessors
• preprocessors.utils
• Tensor Augment
• RIBBIT
• Signal Processing
• Taxa
• Localization
• utils

Index

• Index
• Modules