Applying the mixture density recurrent neural network



A simple MDN RNN architecture diagram


The MDRNN is an exciting sequence model that can generate multiple continuous valued signals from a Gaussian mixture model at each step in time. This project will involve applying and extending my Keras MDRNN models into new applications in the creative arts and beyond. We’ve tried using the MDRNN for voice synthesis, motion capture data synthesis, and musical control data synthesis, but there are lots of other potential applications waiting for you to discover, e.g.: predicting future sensor values, generating robot movements, generating world models for video games or real life situations etc.


  • gain an understanding of mixture density networks and artificial neural network designs
  • apply the MDN to a new application area, this could be a creative arts application, sequence learning from real data, or some other application that you dream up!
  • develop your system and train with real-world data


  • Python programming
  • completed coursework in machine learning or artificial intelligence
  • motivation to work on a new problem and obtain a dataset for training and testing

Background Literature

  1. Christopher M. Bishop. 1994. Mixture Density Networks. Technical Report NCRG/94/004. Neural Computing Research Group, Aston University.
  2. Axel Brando. 2017. Mixture Density Networks (MDN) for distribution and uncertainty estimation. Master’s thesis. Universitat Politècnica de Catalunya.
  3. A. Graves. 2013. Generating Sequences With Recurrent Neural Networks. ArXiv e-prints (Aug. 2013).
  4. David Ha and Douglas Eck. 2017. A Neural Representation of Sketch Drawings. ArXiv e-prints (April 2017).
  5. Charles P. Martin and Jim Torresen. 2018. RoboJam: A Musical Mixture Density Network for Collaborative Touchscreen Interaction. In Evolutionary and Biologically Inspired Music, Sound, Art and Design: EvoMUSART ’18, A. Liapis et al. (Ed.). Lecture Notes in Computer Science, Vol. 10783. Springer International Publishing. DOI:10.1007/9778-3-319-77583-8_11


mixture density network, machine learning, applications

Updated:  10 August 2021/Responsible Officer:  Dean, CECS/Page Contact:  CECS Marketing