Investigating Local and Global Information for Automated Audio Captioning with Transfer Learning

Understand

Automated audio captioning (AAC) aims at generating summarizing descriptions for audio clips.

• Multitudinous concepts are described in an audio caption, ranging from local information such as sound events to global information like acoustic scenery.
• Currently, the mainstream paradigm for AAC is the end-to-end encoder-decoder architecture, expecting the encoder to learn all levels of concepts embedded in the audio automatically.
• This paper first proposes a topic model for audio descriptions, comprehensively analyzing the hierarchical audio topics that are commonly covered.