1. Shift and Scale Parameters in Deep Learning
Do you know what the shift and scale parameters in deep learning are? Please discuss their role and how they are used in models.
2. Batch Normalization: Training vs. Inference
What are the differences between batch normalization during training and inference? Please explain any changes that occur and why they are necessary.
3. Batch Normalization
What is batch normalization and how is it computed? Please explain the concept and the calculations involved, including how to compute mean and variance for the input matrix X.
4. Activation Functions in Neural Networks
What are the different activation functions used in neural networks? Please list them and discuss when and why you would use each one.
5. Optimization Methods in Deep Learning Training
What optimization methods are used in deep learning training? Please discuss the methods you are familiar with and how they contribute to the training process.
