Tutorial objectives needed expanding, introduce concept of base architecture as "backbone" and be consistent with upcoming terminology
Many concepts referring to the same thing used interchangeably, can be VERY confusing for students (add in alternatives once but then stick to one set of terms!)
best example of that is (1) objective function, (2) cost function, (3) loss function and another one, I believe. (Fixed & standardised [to cost function]
Why are we importing the memory profiler in Python?
LATENT_DIM ? - Ah, it's defined just above ConvNeuralNet and then passed in. Probably makes more sense that way, actually.
Discussion of the "spatial hierarchy" in CNNs is not explained. This might be clear to all but it's a good chance to reinforce generalisable feature learning in images, how basic shapes, colours and blocks can combine into more complex shapes and that this in early CNN layers can support good generalisation to enable transfer learning
I extended this explanation to focus on that point and make it clearer
Small conflation about "temporal" dependencies that was slightly disconnected from the explanation that this was for video processing, needed correcting (done)
Added code clarifications to the ClassificationConvNet class, about the input params it takes
The change from the classification to the regression task needed a bit more info on the modelling problem and emphasis of what the "ON" pixels meant (fixed)
There was a comment about requiring the normalised images but this wasn't connected to code or referenced anywhere and was potentially confusing for the end goal (removed that bit)
Is it necessary to explain the PyTorch function of view to the students? Or just leave implementation as it is? Variable names kind of make it obvious, but I will also add a comment to make it extra clear (done)
Need specific definition of patch as it's used in masked image modelling, inpainting etc. (added)
Section 2 (Generalisation of reps) - needed a bit of intro reworking for an optimised explanation, linking back to the ideas of transfer learning I added into the earlier explanation at the start of the notebook
"Source / Destination Network/Task" used here without any clear definition or explanation (added in explanation)
Added "Big Picture" summary and re-emphasised some points and extended it
Tasks are specified by an appropriate cost function
Why might generalisable reps occur
Richer / poorer tasks / local / global contexts
Left them with a thought question about humans having the same network and forcing reuse and whether this is a key point to consider for brain-inspired approaches to AI