Fast.ai Course v3 Lesson 1 Notes

Lesson 1 notes

(41:48) In practice, use fit_one_cycle for “1 cycle policy learning” instead of fit. Some discussions of the cyclical learning rate can be found here, here and here.

(56:20) Advice on how to get the most out of the course: “Pick one project, do it really well, make it fantastic."

(1:06:19) Deep neural network architectures for image classification: ResNet-50 is good enough, take a look at the DAWNBench.

(1:09:31) Use plot_top_losses() to have a check on the model and how good is the prediction, by telling the “wrongest” predictions.

(1:14:00) Also use most_confused() to exam the “wrongest” content suggested by confusion matrix.

(1:15:53) To train the whole model instead of just fine tune the pre-trained model by adding several extra layers at the end, use unfreeze().

(1:21:58) The reason of why just finetuning the original model doesn’t work: Different layers of neural network represent different levels of kind of semantic complexity, and all the layers, for example, updating the representations of diagonal lines, gradients, and the representations of eyeballs, are all trained in the same speed (learning rate actually).

(1:23:00) To improve the accuracy of the model, try to take a look the learning rate plot first.

(1:28:26) Use smaller batch size if the training model doesn’t fit into GPU memory.

Notes on implementation

As I run the codes and projects of the fasi.ai courses on Google Colab, below are some notes about proper Jupyter notebook setup.

Put the below 2 code blocks at the very beginning of the notebook.

# Permit Colaboratory instance to read and write files to Google Drive

# Update packages and the course repo

• The easiest way is to pass get_transforms() to ds_tfms.
• We should also remember to include parameter size in those *“from_"* methods if the sizes of images are not the same, to ensure that all the images are cropped or padded to the same size that they can be collated into batches.