Practical Deep Learning
All models need numbers as their inputs.
We should learn as we learn a new sport: we learn the rules, how fun it is and only later we start paying attention to details, techniques, special training regime etc.
Deep learning neural networks: they find features automatically.
Image-based algorithms may be used for classification of sounds (fft), mouse movements, …
Less code in deep learning is better.
Working with smaller images (thumbnails up to 400px) makes training faster. It may be that opening a large image takes much more time than processing it in a network.
The AI community now found a small amount of models which work very well for many types of tasks.
When predicting a range, it’s sometimes useful to train the prediction on a slightly wider range (e.g. 0.5–5.5 for predicting 1–5 stars)
Run the notebooks. Try prediction of more than one categories for pictures maybe?
Book, 1st chapter
Cat detector the default (resize to 224):
from fastai.vision.all import * path = untar_data(URLs.PETS)/'images' def is_cat(x): return x.isupper() dls = ImageDataLoaders.from_name_func( path, get_image_files(path), valid_pct=0.2, seed=42, label_func=is_cat, item_tfms=Resize(224) ) learn = vision_learner(dls, resnet34, metrics=error_rate) learn.fine_tune(1)
epoch train_loss valid_loss error_rate time 0 0.171127 0.011786 0.002706 00:25 epoch train_loss valid_loss error_rate time 0 0.063262 0.024726 0.006089 00:28
When resize was set to 32 px, the results were:
epoch train_loss valid_loss error_rate time 0 1.047951 0.762592 0.320704 00:12 epoch train_loss valid_loss error_rate time 0 0.725621 0.567899 0.266576 00:12
So not much faster but much worse results.
When resize was set to 512, the results were:
epoch train_loss valid_loss error_rate time 0 0.184101 0.028688 0.008119 01:25 epoch train_loss valid_loss error_rate time 0 0.038081 0.031769 0.008796 01:52
Overfitting is the single most important and challenging issue when training for all machine learning practitioners, and all algorithms.
Models using architectures with more layers take longer to train, and are more prone to overfitting (i.e. you can’t train them for as many epochs before the accuracy on the validation set starts getting worse).
If you’re creating a cat detector, for instance, you generally want at least 30 cats in your validation set.
A key property of the validation and test sets is that they must be representative of the new data you will see in the future.
Fastai library notes
DataBlock: creating training datasets from raw data,
TabularDataLoaders: loading CSV-type datasets,
show_batch then shows an example/sample of the data.
learner.show_results() will sample the model
fine_tune is not available
(there is no pretrained model, e.g. in the case of tabular data)
fit_one_cycle is the most commonly used method
for training fastai models from scratch
Jupyter notebooks tips
!pip install -Uqq fastai
to ensure the newest version of the library (in cloud)
RISE: a library to turn cells into slides and fragments (presentation)
Presentation about notebooks 101.
In command mode, pressing 0 twice will restart the kernel
Ideas for ML
- classification of lines in lyrics+chords
last modified: 2023-11-07