Vulnerabilities and robustness in computer vision

Master Thesis
Author
Konstantakopoulos, Dimitris
Κωνσταντακόπουλος, Δημήτρης - Σίμος
Date
2024-06View/ Open
Keywords
Computer vision ; Robustness ; Vulnerabilities ; FGSM ; Genetic algorithms ; ObjectnetAbstract
Thιs thesis examines the robustness and vulnerabilities of model architectures used in computer vision. The approach to modeling this problem includes collecting a range of trained architectures widely used in computer vision, specifically in the field of image classification, highlighting the sector's shift from CNNs to the use of Transformers. These models include ResNet, Cvt, Swin Transformer, Vision Transformer, and ConvNeXt, where we verify their performance on the dataset they were trained on, ImageNet-1k, and apply them to a different dataset containing similar classes, where ObjectNet was used.After observing their inability to maintain their initial metrics by significant deviation, we investigate possible causes that may have influenced these results. Then, after categorising the main vulnerabilities in the field of computer vision, we perform an untargeted white-box and a targeted black-box attack on the our models using FGSM and a genetic algorithm, respectively .We analyze the results based on the architecture of these models. We conclude that robustness is a multifaced field with much work ahead to ensure its reliability and we present our findings on which architectures were more resilient in terms of accuracy during the respective experiments.