Deepfake detection with deep learning : leveraging remote heart rate estimation to discriminate real from fake videos of faces
View/
Keywords
Deepfake ; Deep learning ; CNN ; PyVHR ; FaceForensics++ dataset ; SincNetAbstract
Deepfakes are swiftly becoming a reality. These media are generated by very powerful tools
which allow fast and cheap media manipulation. These videos can be used for unethical
and malicious applications, such as spread of fake news or fake porn. Given the upcoming
problems these media pose, researchers have started working on different ways to try
and detect these types of media. This master’s thesis investigates the temporal aspect of
deepfake detection, with a specific emphasis on exploring the physiological artifacts asso ciated with the corruption of physiological signals, such as heart rate variability, obtained
through the pyVHR framework. The study employs several state-of-the-art Convolutional
Neural Network (CNN) models, including AlexNet, ResNet, and SincNet, along with vari ous modifications and different data preprocessing techniques. These models are trained
and evaluated on the FaceForensics+ dataset, providing a comprehensive evaluation of
their effectiveness in deepfake detection. The results of this investigation demonstrate
that the SincNet model outperforms other models across different datasets within the
FaceForensics++ dataset. Notably, it achieves an impressive accuracy of 95.8% and an
F1-score of 95.9% when trained on the FaceShifter dataset. However, the performance of the
model varies depending on the specific dataset used for training, indicating the importance
of dataset selection in deepfake detection. The results highlight the potential of using
physiological signals, extracted from the pyVHR framework, in detecting deepfake videos.
Remarkably, the SincNet model demonstrated the ability to effectively distinguish between
real and fake videos using only the BVP signals extracted from the pyVHR framework.
Postgraduate Studies Programme
Πληροφοριακά Συστήματα και ΥπηρεσίεςDepartment
Σχολή Τεχνολογιών Πληροφορικής και Επικοινωνιών. Τμήμα Ψηφιακών ΣυστημάτωνCorporate Department
University of MilanNumber of pages
65Language
EnglishCollections
Except where otherwise noted, this item's license is described as
Αναφορά Δημιουργού-Μη Εμπορική Χρήση-Όχι Παράγωγα Έργα 3.0 Ελλάδα
Αναφορά Δημιουργού-Μη Εμπορική Χρήση-Όχι Παράγωγα Έργα 3.0 Ελλάδα