Ideas for getting involved

Beat tracking and time stretching for mixing

Difficulty: 5

The end goal of this task would be an application that could assist DJs to align beats and mix tracks especially for non-dance music without a steady tempo. The following steps would be required: 1) Improve existing Marsyas beat extraction code and/or port additional beat trackers from MATLAB 2) Implement alignment of the extracted beat to the onset signal to calculate the "phase" of the beats 3) Repeat the above for two different tracks calculating their tempo maps (the mapping of beat locations to actual times). 4) Calculate the time stretch factors required to align the two tempo maps. Stretch the audio in segments using phasevocoder or delay lines to achieve the desired stretching 5) Provide a GUI that shows the process and allows for input by the DJ

Plugin galore

Difficulty: 4

One of the most interesting features in Marsyas is that any audio processing network can be instantiated and configured at run-time. This means that it is possible to create a "meta"-plugin that could run any processing that can be implemented in Marsyas. Alternatively Marsyas can be used as a plugin host for most if not all existing audio plugin architectures. The goal of this task would be to explore how two plugin architectures (VST and VAMP) can be used together with Marsyas. Ideally the final outcome would be a meta-VST and meta-VAMP plugin that would run any Marsyas network and the ability to host multiple VST and VAMP plugins as part of a Marsyas network.

Python, Ruby, Lua etc bindings

Difficulty: 3

Marsyas exposes a lot of functionality at run-time while retaining computational efficiency of compiled code for the processing. More specifically dataflow networks can be created at run-time and their behavior can be changed while the process sound. Currently there are bindings (using SWIG) from various scripting languages to the Marsyas run-time. These bindings are basic and works needs to clean them up and use them. This task would also involve expressing various existing audio networks and applications in Marsyas using the scripting language bindings and documenting the process. The end goal is to really simplify and illustrate the process for end-users.

Documentation and code spring cleaning

Difficulty: 3

Although the documentation has improved dramatically the past few months there is always a lot to be done. This is a very straight-forward project especially for inexperienced programmers who want to learn more about Marsyas. It involves running applications, looking at source code and ensuring that the corresponding documentation and code is complete and correct.

Software synthesizer

Difficulty: 4

Marsyas already has much of the functionality required for implementing a MIDI-controlled polyphonic software synthesizer. For example reading MIDI data, wavetables, envelopes, some physical modeling, delay lines are all components that are part of Marsyas. However they need to all be integrated in order to create a polyphonic software synthesizer that is controllable by MIDI or Open Sound Control. In addition interested programmers can add new building and mappings based on many existing computer music publications. This is a great project for someone interested in learning more about sound synthesis and how it can be efficiently implemented with software instead of dedicated hardware.

GUI for bextract

Difficulty: 2

bextract is the flagship command-line executable of Marsyas. It encapsulates audio feature extraction using a variety of front-ends as well as supervised learning (both training and prediction) using a variety of classifiers. There are many command-line arguments that control it's behavior and learning to use them can be intimidating for beginners. This is an easy task that would have a lot of impact. It is a great way to start learning how to build user interfaces using Qt. The GUI is not particularly challenging and only requires standard widgets.

Porting the Marsyas dataflow architecture to Java

Difficulty: 4

Although Marsyas is written in C++ for efficiency reasons the underlying dataflow architecture and concepts could be implemented in any object-oriented language subject to adjustments and modifications. The goal of this project would be to port enough of the core classes and architecture to Java so that then the task of porting can be simply done one object at a time in a straightforward manner. The porting will probably not be very challenging but the main challenge will be understanding the insides of Marsyas and the corresponding semantics. This is a great task for the aspiring software engineer who wants to get experience tackling large complex code bases.

Orca Vocalization Browser

Difficulty: 3

One of the projects that uses Marsyas involves analyzing 20000 hours of recorded Orca vocalizations. Obviously finding relevant information in such a large amount of audio is very challenging. The goal of this task is to develop a user interface that would assist biologists working with Orca sounds find what they are looking for. We already have some existing tools for this purpose but they are not integrated . The tool should also enable biologist to annotate the audio and link different views of it (waveforms, spectrograms, etc).

Exploring music collections using clustering and dimenstionality reduction

Difficulty: 4

Marsyas has many ways that a collection of music can be analyzed and represented as a dataset of high-dimensional points where each point corresponds to a song and points that are "close" to each other correspond to music pieces that "sound" similar. Using that information it is possible to construct 2D and 3D interfaces for browsing music collections. The main challenge is how to map the high-dimensional feature vectors to the object locations and properties. A variety of clustering techniques and dimensionality reduction techniques can be used for this purpose either isolated or in combination. The goal of this task would be to create a general interface that would allow interactive exploration of these different choices. Some methods such as self-organized maps are already implemented and graphical user interfaces for them exist. This would be a great project for someone interested in visualization and/or implementing clustering and dimensionality reduction algorithms.

Porting MATLAB MIR code to Marsyas

Difficulty: 2

This is another good entry point for someone to get involved with Marsyas development. It is a rather open-ended and flexible tasks. Basically pick any code related to Music Information Retrieval tasks (for example a beat tracker or a pitch detection algorithm) implemented in MATLAB and port it to Marsyas. Marsyas has an excellent run-time bridge to MATLAB and many of the building blocks required are already implemented so in most cases this is a rather straightforward process. At the end of the day you will get a huge speedup in terms of computation time. So if you have some MIR-related MATLAB code that is slowing you down and you are interested in making it faster this is your chance.

Code Janitor

Difficulty: 3

There are two tasks. (1) Architecture changes in the past two years have broken a number of applications. The code janitor should fix these applications, ensure that all applications obey common options, and add regression tests to avoid future breakage. (2) similar work should be done on the individual MarSystems.

Preparing first release of MarsyasX

Difficulty: 3

MarsyasX stands for Marsyas “cross-modal” and borrows from Marsyas 0.2 most of the concepts, namely the hierarchical composition paradigm and the implicit patching of modules. Similarly, data is processed in defined chunks by calling a tick() function and each module also has a set of controls that are used to access their internal parameters. The main conceptual difference is in the mechanism used to exchanged data between processing modules. This data exchange mechanism is highly generic and flexible, supporting any type of data - e.g. images, audio frames, MIDI, XML, lists of points, etc. The goals defined for the first release are mostly achieved but there is still work to be done, namely further testing, fixing some functionalities and improving the support for Linux and Mac OSX. This project is especially interesting for anyone motivated to collaborate in the future with the team working on this new architecture for Marsyas.

Implementing distributed support for MarsyasX

Difficulty: 5

MarsyasX is the result of a new architectural refactoring of Marsyas which aims to provide a generic and flexible platform to process all kinds of data, while inheriting many core Marsyas 0.2 paradigms (see description of the first project related to MarsyasX). In the future, allowing a MarsyasX processing network to be distributed across different processing nodes can represent an important improvement in performance and scalability. The support for distributed processing was thought from the beginning in MarsyasX, but it still didn't leave the paper. The software engineer interested in distributed systems' architectures should find in this project an appealing challenge.

Implementing visual processing algorithms to MarsyasX

Difficulty: 2

MarsyasX is the result of a new architectural refactoring of Marsyas which aims to provide a generic and flexible platform to process all kinds of data, while inheriting many core Marsyas 0.2 paradigms (see description of the first project related to MarsyasX). Most of the work until now focused on the implementation of core functionalities, but as the first release approaches it would be interesting to include more visual processing modules, such as filters, edge detectors, motion estimators, face detectors, etc. Most of the implementations are already publicly available in C++ hence, for the most part, it should be a straightforward task. Also, if you have an interest in developing a particular application on top of MarsyasX, we are open to your suggestions. In this case, the ported algorithm implementations would be mainly related to this application.

Speaker Segmentation and Tracking

Difficulty: 3

Marsyas already includes some (untested) code for the analysis and automatic segmentation of speakers using LSP features, quasi-GMM modeling and Bayesian Information Criteria (BIC) for the task. In fact, this code was ported from the previous Marsyas0.1 version where such application was implemented and achieved some promising results (it was even used for 3 scientific publications). The challenge is now to test, debug, refactor and improve the existing code and implement any new routines that can add robustness or new features. Such a new future would be to, after segmentation, trying to track speakers in a conversation recording.

SDIFSource and SDIFSink

Difficulty: 3

SDIF (Sound Description Interchange Format) is a standard for exchange sound related data. In fact the data model of slices in Marsyas is inspired by SDIF so it is good fit to how things are done. The task would be to write a SDIF source and sink MarSystem that would also Marsyas data to be stored as SDIF streams and vice-versa.