How can we make AI-generated music easier to control?
AI can generate music with impressive structure and complexity. But what happens when we want to make one very specific change? For example: keep the same musical idea, but move it to a higher register and make the notes longer. This kind of precise control remains difficult for many generative AI systems. While models can learn complex musical patterns, their internal decision-making is often difficult to interpret, and changing one musical attribute can unintentionally affect another.
How can we make AI-generated music more precise and controllable, without retraining the model?
New research from the AIXPERT project explores this question in the paper Latent Space Disentanglement via Activation Steering for Interpretable Attribute Control in Symbolic Music Generation, accepted at EUSIPCO 2026 (31 August – 4 September, Bruges, Belgium). The research presents a way to control specific musical attributes directly within a model’s internal representations, opening new possibilities for more interpretable and precise music generation.
Why is precise control difficult?
Music-generation models learn complex relationships between elements such as pitch, duration and rhythm. These relationships help them create coherent music, but they also mean that changing one characteristic can affect another. Many existing approaches to controllable generation require retraining the model or changing its architecture. The AIXPERT research takes a different approach: can we intervene while an already-trained model is generating music?
How does activation steering work?
The team from Orfium investigated the Multitrack Music Transformer (MMT), focusing on two musical attributes: pitch and duration.
By analysing how the model represents examples with different pitch and duration characteristics, the researchers identified directions within its internal representation that correspond to these attributes. These become steering vectors. During generation, the vectors can be introduced into the model to encourage higher or lower pitch, or longer or shorter notes. Crucially, this happens at inference time. There is no retraining, fine-tuning or additional model parameters.
In simple terms, it is like adding a pitch knob or a duration knob to an already-trained music-generation model.
Can different musical attributes be controlled independently?
Not automatically. The research found that pitch and duration are partly entangled within the model. This reflects patterns in the training data: for example, higher-register passages can also tend to contain shorter, faster notes. Simply combining the two steering vectors can therefore produce unwanted changes. Asking for higher and longer notes may affect pitch and duration in ways that are difficult to separate. To address this, the team introduced a Dual Steering approach using Gram-Schmidt orthogonalisation. In simple terms, this separates overlapping directions so that one control has less influence on the other. Among the approaches tested, Gram-Schmidt with pitch priority performed best, achieving an overall dual-steering success rate of 88.5%.
Can the model override its existing musical context?
The researchers also tested whether steering could overcome a strong musical context. They gave the model a 16-beat sequence designed to push the continuation in one direction, and then used steering to move it in the opposite direction. The approach successfully changed the intended attribute in approximately 89% of trials, demonstrating that a relatively small intervention can influence generation even when the model’s existing context suggests otherwise. For pitch, the intervention could produce changes of around two to three octaves, while maintaining musical coherence within the tested operating range.
What does this mean for AI music generation?
The findings provide evidence that musical attributes such as pitch and duration can be represented as meaningful directions within a neural network. This offers a way to make an otherwise difficult-to-interpret model more transparent and controllable, without changing how it was trained. For creative applications, this could eventually mean moving beyond broad instructions such as “make it more energetic” towards much more precise interventions: “keep the composition, raise the pitch and lengthen the notes.”
What are the limitations?
The research focuses on pitch and duration, one model and one orchestral dataset. It measures whether an attribute moves in the intended direction rather than whether it reaches an exact target. The evaluation also uses objective measures rather than human listening tests, so further work is needed to understand how the results relate to human perception. It remains an open question whether the approach can be extended to other musical characteristics, such as harmony, instrumentation or dynamics, and to other generative models.
What comes next?
The team is now exploring ways to make the control even more precise. This includes using sparse autoencoders to identify more specific and interpretable features within the model, as well as feedback-based control that could allow the system to reach a particular target rather than simply move an attribute higher or lower. Ultimately, the research contributes to a broader AIXPERT goal: developing AI systems that are not only capable of generating content, but are also more interpretable, controllable and useful to humans.