https://towardsdatascience.com/using-gitlabs-ci-for-periodic-data-mining-b3cc314ecd85

Photo by Patrick Lindenberg on UnsplashOne of the most time-consuming and difficult stages in a standard data science development pipeline is creating a dataset. In the case where you have already been provided with a dataset kudos for you! You have just saved yourself a good amount of time and effort. Still though, on many occasions that would not be the case. As a matter of fact, the data mining stage can be one of the most demotivating periods in your project timeline. Thus it is always a plus when there are simple and easy techniques to mine the data required.

That being said, in this post I will be giving describing how GitLab’s CI pipelines can be used for periodic data mining jobs without the need of storage buckets, VPSes, external servers and so forth. So without further ado let’s dive in the tutorial.