carefree-learn was designed to support most commonly used methods with carefree APIs. Moreover,
carefree-learn was also designed with interface which is general enough, so that more sophisticated functionality can also be easily integrated in the future. This brings a tension in how to create abstractions in code, which is a challenge for us:
- On the one hand, it requires a reasonably high-level abstraction so that users can easily work around with it in a standard way, without having to worry too much about the details.
- On the other hand, it also needs to have a very thin abstraction to allow users to do (many) other things in new ways. Breaking existing abstractions and replacing them with new ones should be fairly easy.
carefree-learn, there are five main design principles that address this tension together:
carefree-learninto three parts:
- Build some
Common Blockswhich shall be leveraged across different
- Manage models / blocks with
registermechanism, so they can be accessed via their names (see Register Mechanism).
We will introduce the details in the following subsections.
Source code: blocks.py.
carefree-learn implements many basic blocks which can directly form famous models, such as VAE, AdaIN, CycleGAN, BERT, ViT, FNet, StyleGAN, U^2 Net etc. The best of
carefree-learn is that, it not only reproduces the official implementations, but also reuses everything it could. For example:
CycleGANis the same (with different args / kwargs).
Vitis the same (with different args / kwargs).
FNetshares most of the codes, except that
FNetuses fourier transform.
- And much more...
In general, there are three kinds of configurations in
- Model configurations.
- Trainer configurations.
- Pipeline configurations, which is basically constructed by the above two configurations.
See specifying configurations for more details.
carefree-learn, it is likely to see
@xxx.register(...) all around. This is very useful when we want to provide many useful defaults for users.
Here's a code snippet that well demonstrates how to use
Model should implement the core algorithms. It's basically an
nn.Module, with some extra useful functions:
As shown above, there are two special
forward methods defined in a
Model, which allows us to customize
onnx export procedure and
summary procedure respectively.
See Customize Models section for more details.
Trainer should implement the training loop, which includes updating trainable parameters with an optimizer (or, some optimizers), verbosing metrics / losses, checkpointing, early stopping, logging, etc.
Although we can construct a
Trainer from scratch,
make_trainer function, which contains many useful default
Pipeline should implement the high-level parts (e.g.
load, etc.), and will be the (internal) user interface. It's basically a 'wrapper' which can use a
Trainer to train a
Model properly, and can serialize the necessary information to disk.
carefree-learn focuses on Deep Learning tasks, the most general abstraction (
PipelineProtocol) can actually utilize traditional Machine Learning models, such as
See Specify Configurations section for more details.