tasty-musicgen-small

tasty-musicgen-small is a musicgen-small fine-tuned on a patched version of the Taste & Affect Music Database. It generates music that's supposed to induce gustatory synesthesia perceptions based on multimodal research. It generates mono audio in 32khz.

Code and Dataset

Code and the dataset used to train this model are available at: https://osf.io/xs5jy/.

How to use

Here is a showcase on how to use the model with the transformer library, it is also possible to make inference with the audiocraft library, for a detailed explanation we suggest to read the official MusicGEN guide by Hugging Face

from transformers import pipeline
import scipy

synthesiser = pipeline("text-to-audio", "csc-unipd/tasty-musicgen-small")

music = synthesiser("sweet music for fine restaurents", forward_params={"do_sample": True})

scipy.io.wavfile.write("musicgen_out.wav", rate=music["sampling_rate"], data=music["audio"])

Citation

If you use this model, code or the data in your research, please cite the following article:

@article{10.3389/fcomp.2025.1575741,
  author={Spanio, Matteo  and Zampini, Massimiliano  and Rodà, Antonio  and Pierucci, Franco }, 
  title={A multimodal symphony: integrating taste and sound through generative AI},    
  journal={Frontiers in Computer Science},   
  volume={Volume 7 - 2025},
  year={2025},
  url={https://www.frontiersin.org/journals/computer-science/articles/10.3389/fcomp.2025.1575741},
  doi={10.3389/fcomp.2025.1575741},
  issn={2624-9898},
  abstract={In recent decades, neuroscientific and psychological research has identified direct relationships between taste and auditory perception. This article explores multimodal generative models capable of converting taste information into music, building on this foundational research. We provide a brief review of the state of the art in this field, highlighting key findings and methodologies. We present an experiment in which a fine-tuned version of a generative music model (MusicGEN) is used to generate music based on detailed taste descriptions provided for each musical piece. The results are promising: according to the participants' evaluations (n = 111), the fine-tuned model produces music that more coherently reflects the input taste descriptions compared to the non-fine-tuned model. This study represents a significant step toward understanding and developing embodied interactions between AI, sound, and taste, opening new possibilities in the field of generative AI.}
}