![Analysis framework
LibXtract.
Hard-coded, C library.
Around 400 combined
audio features*.
[Bullock, 2007]
Vamp.
Plug-in within a plug-in.
Hosts LibXtract features,
amongst others.
[Cannam et al., 2006]](https://image.slidesharecdn.com/bcusemanticmedia-130925035108-phpapp01/85/Large-Scale-Capture-of-Producer-Defined-Musical-Semantics-Ryan-Stables-Semantic-Media-The-British-Library-23-September-2013-13-320.jpg)
Large-Scale Capture of Producer-Defined Musical Semantics - Ryan Stables (Semantic Media @ The British Library, 23 September 2013)
- 1. Large-Scale Capture of Producer-Defined Musical Semantics Ryan Stables School of Digital Media Technology Birmingham City University
- 2. Problem...
- 3. Problem Definition Producer: Audio effects parameters usually refer to low-level attributes. Professionally produced audio often requires extensive training. Researcher: Lack of semantically annotated music production datasets. How can we map low-level descriptors to perceived muscial timbre?
- 4. Problem Definition Descriptors need to represent the views of music producers. These may change with genre, musical instruments, etc... Various terms may be used to define similar things (colour, texture etc...)
- 5. Project Aims 1. Gather large amounts of semantics data during the music creation/production process. Develop a series of DAW plug-ins. Extract information and anonymously upload it to a server. 2. Identify correlation and patterns in the semantics data. 3. Use the data to improve/aid music production tasks.
- 6. Model...
- 7. Project overview Server Descriptor name... Save...Load... Save... Semantic Descriptor Parameter Space Feature Set Pre/Post Gain Analysis... Natural Language Processing Dimensionality Reduction Etc... (1) (2)(3) Figure : Schematic Overview of the Semantic Audio Feature Extraction Project.
- 8. (1) Plug-in interface Parameters can be set experimentally. Semantic descriptors to be stored in text field. Descriptors can be loaded through same interface. Parameters are stored and/or set. Figure : Semantic Audio plug-in: Multi-band distortion
- 9. (2) Feature Extraction Features are extracted from the selected region. The parameter space is stored. Semantic descriptors are sent as targets. Additional metadata is sent, if available. Server Descriptor name... Save...Load... Save... Semantic Descriptor Parameter Space Feature Set Pre/Post Gain Analysis... Natural Language Processing Dimensionality Reduction Etc... Figure : Stored attributes.
- 10. (3) Mapping NLP Algorithms to identify semantic correlation. Dimensionality reduction to find correlation in features/parameters. Additional data partitions based on metadata (Genre, instrument, etc...) Results sent back to user plug-in. Server Descriptor name... Save...Load... Sav Semantic Paramete Feature Se Pre/ Analysis... Natural Language Processing Dimensionality Reduction Etc... Figure : Results processing
- 12. Architecture Requirements: Maximisation of user-base. Transparency: Access to the processing chain. Design decisions: Stand-alone plug-ins. MultiFX. Plug-ins within a plug-in. Analysis-only. Other: Free field vs. fixed word. Before and after. Metadata pane.
- 13. Analysis framework LibXtract. Hard-coded, C library. Around 400 combined audio features*. [Bullock, 2007] Vamp. Plug-in within a plug-in. Hosts LibXtract features, amongst others. [Cannam et al., 2006]
- 14. Mini-Project...
- 15. Mini-Project: Aims Analyse the production requirements of musicians. Birmingham Conservatoire The Music Producers Guild The Birmingham Music Network Build a series of prototype systems for the collection of musical semantics data. Use these systems to collect data from a small group of musicians during the production process. Evaluate the results in order to identify a suitable system for future research. Demonstrate the feasibility of a wider research project in this area.
- 16. Mini Project: Schematic Plug-in development Interface design Algorithm Development Server, network, data distribution User Testing Data Aquisition Results Analysis Figure : Schematic Overview of the Mini-Project.
- 17. Positions and Timescale 2 x PhD Students: 1 x C4DM (QMUL) & 1 x DMT (BCU). 3 x Advisory roles. Timescale: 6-months from September 2013. Future: collaborative grant application. Thanks! [email protected]
- 18. References Bullock, J. (2007). Libxtract: A lightweight library for audio feature extraction. In Proceedings of the International Computer Music Conference, volume 43. Cannam, C., Landone, C., Sandler, M. B., and Bello, J. P. (2006). The sonic visualiser: A visualisation platform for semantic descriptors from musical signals. In ISMIR, pages 324–327.