Μοντελοποίηση ακολουθιών με βαθιά νευρωνικά δίκτυα
Sequence modelling with deep neural networks
The purpose of the present postgraduate dissertation is to present modern bibliography on the field of sequence modeling with deep neural networks. The main body of the analysis is about the Recurrent Neural Network architectures (RNN) which are at the heart of modern scientific research and seem to present the best results in the field. Attempts to model sequences have been made with different models of neural networks, such as the Convolutional Neural Networks (CNN), which, however, are not considered in this thesis. The dissertation presents the limitations and disadvantages of classical RNN structures for sequence modeling as well as the LSTM architecture, which extends the capabilities of RNN networks. Finally, the dissertation mentions the reasons that led to the development of Transformers models and their architecture is briefly presented.