Abstract:

Generative AI has revolutionised visual content editing, empowering users to effortlessly modify images and videos. However, not all edits are equal. To perform realistic edits in domains such as natural image or medical imaging, modifications must respect causal relationships inherent to the data generation process. Such image editing falls into the counterfactual image generation regime. Evaluating counterfactual image generation is substantially complex: not only it lacks observable ground truths, but also requires adherence to causal constraints. Although several counterfactual image generation methods and evaluation metrics exist, a comprehensive comparison within a unified setting is lacking. In this talk we will present our work on building a comparison framework to thoroughly benchmark counterfactual image generation methods. We integrate all models that have been used for the task at hand and expand them to novel datasets and causal graphs, demonstrating the superiority of Hierarchical VAEs across most datasets and metrics. Our published framework is implemented in a user-friendly Python package that can be extended to incorporate additional SCMs, causal methods, generative models, and datasets for the community to build on.

Biography:

Thomas Melistas is currently a PhD student at the Archimedes Research Unit (Athena RC) supervised by Prof. Sotirios Tsaftaris, Prof. Yannis Panagakis (National and Kapodistrian University of Athens) and Dr. Giorgos Papanastasiou (Pfizer). His main research interests are causal machine learning and generative modeling. He is excited about the application of the above in the domains of medical imaging and computer vision. He received an Integrated MEng in Electrical and Computer Engineering from the National Technical University of Athens. His master’s thesis was on symbolic music generation. Prior to his PhD, he has worked as a Machine Learning Engineer and did research on the fields of speech emotion recognition and multimodal learning.

Nikos Spyrou is a PhD student at Archimedes RU / Athena RC and at the National and Kapodistrian University of Athens, coadvised by Prof. Yannis Panagakis and Prof. Sotirios Tsaftaris. His research interests mainly involve computer vision, representation learning, causality, generative models and applications in the medical domain. He obtained a MEng in Electrical and Computer Engineering from the National Technical University of Athens with specialization in computer science.

Nefeli Gkouti is a PhD student at Archimedes RU / Athena RC and at the National and Kapodistrian University of Athens, supervised by Prof. Yannis Panagakis and Prof. Sotirios Tsaftaris. Her main research interests include causality and interpretability in machine learning, representation learning, generative models and their application in healthcare. She holds a MSc degree in Computer Science from the Department of Informatics of the Athens University of Economics and Business, with specialization in machine learning, and a Bachelor degree from the Department of Mathematics of the National and Kapodistrian University of Athens.