Traditionally, we need to process the raw data before we feed them into our machine learning models (e.g. encode the label column, which is a string column, into an ordinal column). In carefree-learn, however, we can train neural networks directly on files without worrying about the rest:
m = cflearn.api.fit_ml("iris.data", carefree=True)
What's going under the hood is that carefree-learn will try to parse the iris.data automatically (with the help of carefree-data), split the data into training set and validation set, with which we'll train a fully connected neural network (fcnn).
We can further inspect the processed data if we want to know how carefree-learn actually parsed the input data:
print(m.cf_data.raw.x[0])
print(m.cf_data.raw.y[0])
print(m.cf_data.processed.x[0])
print(m.cf_data.processed.y[0])
Which yields
[5.1 3.5 1.4 0.2]
['Iris-setosa']
[-0.9006812 1.0320569 -1.3412726 -1.3129768]
[2]
It shows that the raw data is carefully normalized into numerical data that neural networks can accept. What's more, by saying normalized, it means that the input features will be automatically normalized to mean=0.0 and std=1.0:
The results shown above means we first normalized the data before we actually split it into train & validation set.
After training on files, carefree-learn can predict & evaluate on files directly as well. We'll handle the data parsing and normalization for you automatically:
# instantiate an `MLInferenceData` instance
idata = cflearn.MLInferenceData("iris.data")
# `contains_labels` is set to True because `iris.data` itself contains labels
As we know, neural networks are trained with stochastic gradient descent (and its variants), which will introduce some randomness to the final result, even if we are training on the same dataset. In this case, we need to repeat the same task several times in order to obtain the bias & variance of our neural networks.
Fortunately, carefree-learn introduced repeat_ml API, which can achieve this goal easily with only a few lines of code:
# With num_repeat=3 specified, we'll train 3 models on `iris.data`.
result = cflearn.api.repeat_ml("iris.data", carefree=True, num_repeat=3)
It is worth mentioning that carefree-learn supports Distributed Training, which means when we need to perform large scale benchmarking (e.g. train 100 models), we could accelerate the process through multiprocessing:
info
In carefree-learn, Distributed Training in Machine Learning tasks sometimes doesn't mean training your model on multiple GPUs or multiple machines. Instead, it may mean training multiple models at the same time.
# With num_jobs=2, we will launch 2 processes to run the tasks in a distributed way.
result = cflearn.api.repeat_ml("iris.data", carefree=True, num_repeat=10, num_jobs=2)
caution
On iris dataset, however, launching distributed training will actually hurt the speed because iris dataset only contains 150 samples, so the relative overhead brought by distributed training will be too large.
But this is not enough, because we want to know whether other models (e.g. scikit-learn models) could achieve a better performance than carefree-learn models. In this case, we can perform an advanced benchmarking with the Experiment helper class.
Notice that we specified run_command="python run_sklearn.py" for scikit-learn tasks, which means Experiment will try to execute this command in the current working directory for training scikit-learn models. The good news is that we do not need to speciy any command line arguments, because Experiment will handle those for us.
Here is basically what a run_sklearn.py should look like (source code):
import os
import pickle
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
With run_sklearn.py defined, we could run those tasks with one line of code:
results = experiment.run_tasks()
After finished running with this, we should be able to see the following file structure in the current working directory:
|--- __experiment__
|--- __data__
|-- x.npy
|-- y.npy
|-- x_cv.npy
|-- y_cv.npy
|--- fcnn/0
|-- _logs
|-- __meta__.json
|-- cflearn^_^fcnn^_^0000.zip
|--- linear/0
|-- ...
|--- decision_tree/0
|-- __meta__.json
|-- sk_model.pkl
|--- random_forest/0
|-- ...
As we expected, carefree-learn models are saved into .zip files, while scikit-learn models are saved into sk_model.pkl files. Since these models are not yet loaded, we should manually load them into our environment:
pipelines ={}
sk_patterns ={}
for workplace, workplace_key inzip(results.workplaces, results.workplace_keys):
Contained in this article is just a subset of the features that carefree-learn offers, but we've already walked through many basic & common steps we'll encounter in real life machine learning tasks.