My Research: Towards a more creative, expressive, interactive world via Music AI.

I am broadly interested in the design of interactive intelligent systems to extend human musical creation and expression. This research lies in the intersection of Machine Learning, HCI, Robotics, and Computer Music. My representative works include interactive composition via style transfer, human-computer interactive performances, autonomous dancing robots, large-scale content-based music retrieval, haptic guidance for flute tutoring, and bio-music computing using slime mold.

Particularly, I build music agents that compose and arrange music via style transfer and analogy, perform and improvise music expressively in concert with human musicians by learning from rehearsal experience, and tutor music beginners using haptic guidance. These efforts unify expressive performance rendering, automatic accompaniment, and algorithmic composition in a machine-learning framework, making music a more accessible and friendly tool for everyone.

About me: A computer scientist and professional musician.

I am an Assistant Professor in Machine Learning at MBZUAI. I received my Ph.D. in the Machine Learning Department at Carnegie Mellon University where I study Machine Learning and Computer Music under the advice of Prof. Roger Dannenberg. I was a Neukom Fellow at Dartmouth from 2016 to 2017. In 2010, I received my undergraduate degree in Information Science with a minor in Psychology at Peking University. I am also a professional DI and XIAO (Chinese flute and vertical flute) player. I was the prime soloist of the Chinese Music Institute (CMI) of Peking University, where I also served as the president and assistant conductor. I held my solo concert in 2010 and a Music AI concert in 2022. (See Music Events)

News:
Music X Lab has moved from NYU Shanghai to MBZUAI at Abu Dhabi. I still hold a courtesy appointment at NYU Shanghai.

Our "Self-supervised learning via Physical Symmetry (SPS) paper is accepted by NeurIPS 2023. It introduces a pioneering direction in (music) concept learning.

Our "whole-song hierarchical generation" paper is acceptd by ICLR2023 as a spotlight. Being trained on a small dataset, this novel deep generative model learns to compose full structural pop songs from scratch.

I am always looking for passionate, self-motivated students to join Music X Lab. (Note that MBZUAI has an extended application period.)

An Outline of Main Publication Classification (organized according to topic and story lines. See here for a complete list).

Deep Music Generation

Big Picture: [Blog#1, Intro to Music X Lab's research] [Blog#2, Analogy vs. Prediction: a Debate on the Philosophy of Music Automation][Music Style Transfer: A Position Paper] Representation Learning and Disentanglement for Symbolic Music—From Monophonic to Polyphonic: [EC2-VAE] [PianoTree VAE][Poly-dis][Blog: coming soon] Multi-modal Music Representation Learning—Bridging Analysis and Creation: [Chimera: A Unified Model for Separation, Transcription and Synthesis][A2S: Auido-to-symbolic Arrangement][BUTTER] Accompaniment Arrangement via Music Representation Learning: [Pop909 Dataset][Poly-dis][A2S][AccoMontage][AccoMontage2][Q&A][AccoMontage3][Blog: coming soon]

Hierarchical Representation Learning
Big Picture: [Blog, Key Issues in Hierarchical Modeling: coming soon] Long-term Music Modeling via Compostional Hierarchy [Hierarchical-EC2-VAE][Whole-song Hierarchical Generation][Blog: coming soon] Modeling Context-dependency—Content, Form, and Motif Development [Transformer VAE][Content-Form Disentanglement][Sequence Attention][Motif Learning][Blog: coming soon]

Large-scale Language Models (LLMs) for Music
Big Picture: [Seminar Discussion, Interpretability in LLMs and "Functional Alignment": video and note] Foundation Models and Benchmarks: [MERT][MARBLE] Controlling and Fine-tuning Music LLMs: [Loop Copilot][Coco-Mulla]

Human-AI Music Co-creation
Big Picture: [Blog, The Many Ways to Control a Generative Model: coming soon] Inpainting and Refinement—Note-by-note, Chunk-by-chunk and Factor-by-factor: [MuseBERT][Phrase Inpaiting][Mixed-level Inpaiting][Blog: coming soon] Unifying Internal Control and External Control via Diffusion Models: [Polyffusion][Whole-song Gen by Cascaded Diffusion]

Computer-aided Music Learning via Multimodal feedbacks

D. Chin, Y. and G. Xia	A Computer-aided Multimodal Music Learning System with Curriculum: A Pilot Study [PDF]	2021	Proc. 22nd The International Conference on New Interfaces for Musical Expression (NIME)
D. Chin, Y. Zhang, and G. Xia	Hyper-hybrid Flute: Simulating and Augmenting How Breath Affects Octave and Microtone [PDF]	2021	Proc. 21st The International Conference on New Interfaces for Musical Expression (NIME)
Y. Li, Z. Piao, and G. Xia	A Wearable Haptic Interface for Breath Guidance in Vocal Training [PDF]	2021	Proc. 21st The International Conference on New Interfaces for Musical Expression (NIME)
D. Chin, Y. Zhang, T. Zhang, J. Zhao, and G. Xia	Rainbow Score: A Visual-centered Multimodal Flute Tutoring System [PDF]	2020	Proc. 20th The International Conference on New Interfaces for Musical Expression (NIME)
Y. Zhang, Y. Li, D. Chin, and G. Xia	Adaptive Multimodal Music Learning via Interactive-haptic Instrument [PDF]	2019	Proc. 19th The International Conference on New Interfaces for Musical Expression (NIME)
G. Xia, C. Jacobsen, Q. Chen, X-D. Yang, and R. Dannenberg	ShIFT: A Semi-haptic Interface for Flute Tutoring [PDF]	2018	Proc. 18th The International Conference on New Interfaces for Musical Expression (NIME)

Expressive and Interactive Performance

M. Xu, Z. Wang, and G. Xia	Modeling Perceptual Loudness of Piano Tone:Theory and Pratice [PDF coming soon]	2022	Proc. 23th the International Conference on Music Information Retrieval (ISMIR)
M. Xu, Z. Wang, and G. Xia	Transferring Piano Performance Control Across Environments [PDF] [Poster]	2019	Proc. 20th the International Conference on Music Information Retrieval (ISMIR)
G. Xia, and R. Dannenberg	Duet Interaction: Learning Improvisation Techniques for Automatic Accompaniment [PDF]	2017	Proc. 17th The International Conference on New Interfaces for Musical Expression (NIME)
S. Dai, and G. Xia,	Computational Models for Common Pipa Techniques [PDF coming soon]	2017	Proc. 5th  National Conference on Sound and Music Technology (CSMT) Best student paper award
G. Xia, K. Mao, M. Fu, S. Cosentino, G.Trovato, S. Sessa, A. Takanishi, and R. Dannenberg	Expressive Humanoid Robot for Automatic Accompaniment. [PDF] [Slides]	2016	Proc. 13th the International Conference on Sound and Music Computing (SMC)
G. Xia, Y. Wang, R. Dannenberg, and G. Gordon	Spectral Learning for Expressive Interactive Ensemble Music Performance. [PDF] [Slides] [Audio Demo]	2015	Proc. 16th the International Conference on Music Information Retrieval (ISMIR)
G. Xia & R. Dannenberg	Duet Interaction: Learning Musicianship for Automatic Accompaniment. [PDF] [Slides] [Audio Demo] [Data]	2015	Proc. 15th the International Conference on New Interfaces for Musical Expression (NIME)
M. Fu, G. Xia, R. Dannenberg, and L. Wasserman	A Statistical View on the Expressive Timing of Piano Rolled Chords. [PDF] [Poster]	2015	Proc. 16th the International Conference on Music Information Retrieval (ISMIR)
G. Xia, J. Tay, R. Dannenberg, and M. Veloso	Autonomous Robot Dancing Driven by Beats and Emotions of Music. [PDF] [Slides] [Demo1] [Demo2]	2012	Proc. 12th International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS)
D. Liang, G. Xia, and R. Dannenberg	A Framework for Coordination and Synchronization of Media. [PDF] [Slides] [Talk + Demo]	2011	Proc. 11th the International Conference on New Interfaces for Musical Expression (NIME)

Music Information Retrieval

X. Liu, D. Chin, Y. Huang, and G. Xia	Learning Interpretable Low-dimensional Representation via Physical SymmetryMeasures [PDF]	2023	Proc. 37th Conference on Neural Information Processing Systems (NeurIPS)
J. Jiang, D. Chin, Y. Zhang, and G. Xia	Learning Hierarchical Metrical Structure Beyond Measures [PDF]	2022	Proc. 23nd the International Conference on Music Information Retrieval (ISMIR)
L. Lin, Q. Kong, J. Jiang, and G. Xia	A Unified Model for Zero-shot Music Source Separation, Transcription and Synthesis. [PDF]	2021	Proc. 22nd the International Conference on Music Information Retrieval (ISMIR)
J. Jiang, K. Chen, W. Li, and G. Xia	Large Vocabulary Chord Transcription via Chord Structure Decomposition. [PDF]	2019	Proc. 20th the International Conference on Music Information Retrieval (ISMIR)
G. Xia, T. Huang, Y. Ma, R. Dannenberg, and C. Faloutsos	MidiFind: Similarity Search and Popularity Mining in Large MIDI Databases. [PDF] [Website: online MIDI search]	2014	Sound, Music, and Motion, LNCS 2014, pp 259 - 276
G. Xia, D. Liang, R. Dannenberg, and M. Harvilla	Segmentation, Clustering, and Display in a Personal Audio Database for Musicians. [PDF] [Poster]	2011	Proc. 12th the International Conference on Music Information Retrieval (ISMIR)

Selected Music Events

 

 

Web templates from Byron Boots: http://www.cc.gatech.edu/~bboots3/