Patrik Lauha: Muuttolintujen Kevät - Automatic Bird Sound Classifier

LIFEPLAN project: international bio monitoring project, with DNA samples, camera traps, audio recordings. This is a huge amount of data.

Muuttolintujen Kevät (“Spring of migratory birds”) is a mobile application based citizen science campaign where citizens collect bird observations with help of an automated bird sound classifier of Finnish birds. You can download the app on your phone. Recordings are analyzed with AI model, which is trained to recognize the vocalizations of Finnish birds. You need a Finnish phone operator for the app to work.

You can also scroll through your old observations, see a list of species you have recorded. You see the confidence percentage of the bird recognition. Also a small game: listen and choose the correct bird.

We have more than 300 thousand users. This is 5 percent of the Finnish population! 16 million recordings. Most common species: common chaffinch, eurasian blue tit, great tit, common blackbird, willow warbler. So the people help us to collect data about birds in nature, without us having to install lots and lots of audio equipment.

We transform audio to a spectogram image. Why? Image recognition is really good. And it is natural to represent audio as an image. With audio you have about 48,000 tiny data points per second. Represented as image this is much less, but enough for the pattern to be recognizable. We use a short-time Fourier transform. See also https://bsg.laji.fi

We use TensorFlow and Keras in Python. Convolutional neural netword processes input audio in 3-second segments. Training data from global bird sounds library xeno-canto, Finnish field recordings, and our phone app. Trained for 263 Finnish bird species. Output is then for each species the likelyhood that there is a match.

We use data augmentation to avoid data overfitting, where the model would be really good at recognizing only the training data based on irrelevant details. In Python: Scipy, colorednoise, noisereduce.

With passive acoustic monitoring we record the whole day and can see what birds are most active in the early morning. With the data from citizens, we see most data a bit later in the morning, because people are asleep before, and another peak near the end of the day.

Ongoing research: with the app data:

• Digital twinning and real-time bird forecasts. With the data we get really good predictions of when the migratory birds are coming and going. We can also show predictions of where and at which time you are most likely to hear your favorite bird, so you can be there.
• Complementing the breeding bird atlas: https://lintuatlas.fi
• Humanities / social sciences study of nature experience. In the app you can record your nature experience: tell how your surroundings look, how you feel, what you hear, smell, think. We don't know who is behind the recordings, so you should feel safe to use this part of the application.

For training we need a large computer. For classification we don't need much.