In this presentation, we discuss our experiences with the application of machine learning techniques and particularly neural networks in order to assess nasal breathing, identify possible health issues and in this way support decision making of a medical professional. As part of this task, we show how computer vision methods can augment available data and incorporate geometry of a nasal space into the learning process. In addition, we explore the problem of automatic retrieval of semantic annotations from medical images and reports. Finally, we compare a number of machine learning approaches and make suggestions on their applicability in the medical context.