Publicación:
MIDI-Conditional Text-to-Audio Synthesis Using ControlNet on AudioLDM

Vista sencilla de ítem
dc.contributor.authorIbáñez Martínez, Laura
dc.contributor.directorCuadra Troncoso, José Manuel
dc.date.accessioned2024-09-18T13:01:24Z
dc.date.available2024-09-18T13:01:24Z
dc.date.issued2023-09
dc.description.abstractText-to-audio systems have gained attention in recent months, achieving impressive results in general audio synthesis. However, they often lack fine-grained control over the musical output, as note-level adjustments cannot be determined by text. In this work, we present MIDI-AudioLDM, which implements MIDI conditioning into AudioLDM with the use of ControlNet. This enables MIDI-conditional text-to-audio synthesis, which adds up to AudioLDM’s previous capacities, including direct text-to-audio synthesis as well as audio style transfer and inpainting. Like AudioLDM, the model uses contrastive language-audio pretraining (CLAP) latents and is trained on audio embeddings, while using text embeddings for inference. In contrast to unconditional audio synthesis, MIDI-AudioLDM offers detailed control over various musical aspects such as notes, genre, mood, and timbre, which makes it a more valuable tool for the music production process. A demo is available at https://huggingface.co/spaces/lauraibnz/midi-audioldm.en
dc.description.provenanceMade available in DSpace on 2024-09-18T13:01:24Z (GMT). No. of bitstreams: 1 Ibanez_Martinez_Laura_TFM.pdf: 1971185 bytes, checksum: b14a46b1727df5b8024fddb97de2f45a (MD5) Previous issue date: 2023-09en
dc.identifier.citationIbáñez Martínez, Laura (2023) MIDI-Conditional Text-to-Audio Synthesis Using ControlNet on AudioLDM. Trabajo Fin de Máster. Universidad Nacional de Educación a Distancia (UNED)
dc.identifier.urihttps://hdl.handle.net/20.500.14468/23782
dc.language.isoen
dc.publisherUniversidad Nacional de Educación a Distancia (UNED). E.T.S. de Ingeniería Informática
dc.relation.centerE.T.S. de Ingeniería Informática
dc.relation.degreeMáster Universitario en Investigación en Inteligencia Artificial
dc.relation.departmentInteligencia Artificial
dc.rightsinfo:eu-repo/semantics/openAccess
dc.rights.urihttps://creativecommons.org/licenses/by-nc-nd/4.0/deed.es
dc.subject1203.17 Informática
dc.subject.keywordsaudio synthesisen
dc.subject.keywordsMIDI conditioningen
dc.subject.keywordstext-to-audio systemsen
dc.subject.keywordsAudioLDMen
dc.subject.keywordsControlNeten
dc.titleMIDI-Conditional Text-to-Audio Synthesis Using ControlNet on AudioLDMes
dc.typemaster thesisen
dspace.entity.typePublication
Archivos
Bloque original
Mostrando 1 - 1 de 1
Miniatura
Nombre:
Ibanez_Martinez_Laura_TFM.pdf
Tamaño:
1.88 MB
Formato:
Adobe Portable Document Format
Descargar
Bloque de licencias
Mostrando 1 - 1 de 1
No hay miniatura disponible
Nombre:
license.txt
Tamaño:
3.62 KB
Formato:
Item-specific license agreed to upon submission
Descripción:
Descargar
Colecciones
Trabajos de fin de máster (TFM)