- [M] An autoencoder is a neural network that learns to copy its input to its output. When would this be useful?
- Self-attention. 15. [E] What’s the motivation for self-attention? 16. [E] Why would you choose a self-attention architecture over RNNs or CNNs? 17. [M] Why would you need multi-headed attention instead of just one head for attention? 18. [M] How would changing the number of heads in multi-headed attention affect the model’s performance?
- Transfer learning 19. [E] You want to build a classifier to predict sentiment in tweets but you have very little labeled data (say 1000). What do you do? 20. [M] What’s gradual unfreezing? How might it help with transfer learning?
- Bayesian methods. 21. [M] How do Bayesian methods differ from the mainstream deep learning approach? 22. [M] How are the pros and cons of Bayesian neural networks compared to the mainstream neural networks? 23. [M] Why do we say that Bayesian neural networks are natural ensembles?
- GANs. 24. [E] What do GANs converge to? 25. [M] Why are GANs so hard to train?