Congratulations to Modestas Motiejauskas, the newly conferred Doctor of Natural Sciences in the field of Informatics, who successfully defended his dissertation on 25 May entitled "Convolutional Neural Network Architectures and Training Improvements: Visual Emotion Recognition Case".
(from right to left) Dr Modestas Motiejauskas and Prof. habil. Dr Gintautas Dzemyda, photo VU Faculty of Medicine
The research was carried out between 2021 and 2025 at the Institute of Data Science and Digital Technologies, Faculty of Mathematics and Informatics, Vilnius University (VU MIF DMSTI), under the supervision of Prof. habil. dr. Gintautas Dzemyda, and makes a significant contribution to affective computing – a branch of artificial intelligence concerned with the recognition and interpretation of human emotions.
How to Recognise Emotions
Although artificial intelligence is already capable of recognising facial expressions with reasonable accuracy, interpreting the emotional content of landscape photographs, street imagery, or abstract compositions is considerably more challenging.
The central difficulty is what researchers call the affective gap. The relationship between the primary visual features of an image – colours, textures, composition – and the emotional response it evokes in a viewer is unstable and highly subjective. A red flower against a bright background may evoke joy, whilst the same flower in darkness might provoke anxiety. The same object can carry different, even opposing, emotional meanings.
Owing to this subjectivity, visual emotion recognition in general-purpose images remains one of the most demanding unsolved problems in the field. The emotion an image elicits may depend not only on individual features, but on the image as a whole.
Dr Motiejauskas's dissertation addresses this problem using an eight-category emotion labelling scheme and several large-scale datasets.
A Style-Aware Architecture and Improved Machine Learning
The dissertation proposes a new model based on the EfficientNetV2S convolutional neural network architecture, augmented with two principal innovations designed to directly tackle the affective gap.
Dissertation defence, photo by Ugnius Bagdonavičius
The first is the integration of Gram matrix modules into the network. Conventional image classifiers focus primarily on object and scene recognition – that is, on what is depicted in a photograph. Gram matrix modules capture stylistic information from intermediate feature maps, including texture and colour-related patterns. These deeper-level features are precisely those that most strongly shape emotional perception, yet are frequently overlooked by traditional models. By embedding this capability into the network architecture, the model gains a richer and more comprehensive representation of each image.
The second innovation is a specialised training strategy employing a contrastive-centre loss function. This method alters the way in which the model organises the internal representation of emotion categories. During training, images belonging to the same emotion class are drawn closer together in the model's learned feature space, whilst different emotion categories – such as fear and contentment – are pushed further apart. As a result, the model's predictions become more consistent, more clearly defined, and less prone to confusing emotionally similar categories.
A New Metric for Prediction Consistency
Beyond the model itself, the dissertation introduces a novel evaluation tool called the top-2 cross-sentiment measure. It assesses the internal consistency of a model's predictions by examining whether the two highest-confidence predicted emotion classes fall into opposing sentiment groups – positive or negative. Crucially, this measure does not require ground-truth labels, making it applicable in settings where annotated data is unavailable.
This complements conventional accuracy metrics and provides a methodology that future research could apply independently of the proposed model.
"Solutions that may at first glance appear unrelated to the problem posed in the dissertation translate very well to my topic," says Dr Motiejauskas. "I am referring to technical solutions from computer vision — the analysis of images."
Dr Modestas Motiejauskas, photo by Ugnius Bagdonavičius (VU)
Applications
Dr Motiejauskas works at the VU MIF Institute of Data Science and Digital Technologies, and his dissertation opens a broad range of direct application possibilities.
- Mental health monitoring. Detecting emotional trends in social media images could help identify early signs of depression and other indicators of wellbeing.
- Adaptive learning environments. Educational systems capable of recognising emotional engagement from images could adjust the pace and content of instruction according to pupils' emotional states.
- Human–computer interaction. Interfaces responsive to the emotional tone of user-generated content could offer more personalised and context-sensitive experiences.
- Art and multimedia analysis. Computational tools for assessing the emotional impact of works of art could open new forms of cultural analysis and content recommendation.
Art and multimedia analysis strikes Dr Motiejauskas as one of the most relevant areas of application for his work. "It involves the fewest ethical complications. Even so, it remains essential to inform the user that automated models are being used," he notes.
A Historic Moment for Lithuania's Educational and Disability Communities
Representatives of Lithuanian disability organisations, the Parliamentary Commission on Persons with Disabilities, and the Ministry of Health attended the dissertation defence.
"This event is historic for Lithuania's educational and disability communities, as it marks the first time a person with such a severe disability has defended a doctoral degree," reads the statement from the Commission on the Rights of Persons with Disabilities. "Modestas's work and perseverance reaffirm that there are no limits to human will and ambition."
Commission Chair Indrė Kižienė underlined the broader significance of this achievement. "Modestas is an inspiration to us all – to the academic community, to persons with disabilities, and to every individual – demonstrating that one can and must pursue one's goals. This case proves once again that there are no insurmountable obstacles, and that great determination and hard work can lead to the highest of achievements."
"I had the will and resolve to complete my doctoral studies. I am not one to abandon work once begun," says Dr Motiejauskas. "I felt the support of my supervisor and my loved ones." He wishes society greater openness, goodwill, and tolerance towards others.
In the view of the Commission Chair, Modestas's story of success is not only a personal achievement, but also an important reminder to the educational system that higher education must remain open and accessible to all, regardless of individual needs or disability.
The statement also acknowledges the substantial contribution of his supervisor, Prof. habil. dr. G. Dzemyda of VU MIF DMSTI – his professionalism, feedback, and genuine care, all of which contributed to Dr Motiejauskas's success on his academic journey.
Dissertation Defence Committee:
- Academician Prof. Dr Olga Kurasova – Chair (Vilnius University)
- Prof. Dr Romas Baronas (Vilnius University)
- Prof. Dr Diana Kalibatienė (Vilnius Gediminas Technical University)
- Dr Gerda Ana Melnik-Leroy (Vilnius University)
- Prof. Dr Audris Mockus (University of Tennessee, USA)
5 June 2026