Machine learning pipelines in serverless environments
Ανάπτυξη σύνθετων υπηρεσιών μηχανικής μάθησης σε περιβάλλοντα serverless
Master Thesis
Author
Παρασκευουλάκου, Ευτέρπη
Paraskevoulakou, Efterpi
Date
2020-02View/ Open
Keywords
Machine learning ; Artificial intelligence ; Serverless ; Function as a Service ; ContainersAbstract
The fast-growing rate of data has created many challenges, and manifold frameworks and approaches are made available towards facilitating analysis of the heterogeneous datasets to extract useful information, discover trends and insights in various domains. While data scientists employ algorithms to facilitate the actual transformation of datasets into valuable information, a key challenge refers to raising the level of abstraction from the underlying layers: the provision of frameworks and components as services and the provision of infrastructure resources. In this context, emerging cloud-related technologies focus on the serverless approach, which enables the deployment of applications and services. Serverless brings to light the concept of Function as a Service (FaaS), as a novel paradigm that provides the means to realize serverless offerings. The developers are able to develop their applications as Nanoservices with the required scalability without dealing with the deployment and management of the infrastructure resources. This thesis presents an approach for facilitating the provision of Machine Learning (ML) Functions as a Service (i.e., ML-FaaS). The proposed approach goes beyond atomic services to composite services, i.e., workflows of ML tasks, thus enables the realization of the complete data path functions performed by data scientists, including for example aggregation, cleaning, feature extraction, and analytics. We also demonstrate the operation of the aforementioned approach and evaluate its performance and effectiveness exploiting an anti-fraud detection machine learning pipeline.