دانلود کتاب Normalization Techniques in Deep Learning

فهرست مطالب :

Preface Reference Acknowledgements References Contents About the Author 1 Introduction [DELETE] 1.1 Denotations and Definitions 1.1.1 Optimization Objective 1.1.2 Neural Networks 1.1.3 Training DNNs 1.1.4 Normalization 2 Motivation and Overview of Normalization in DNNs [DELETE] 2.1 Theory of Normalizing Input 2.2 Towards Normalizing Activations 2.2.1 Proximal Back-Propagation Framework 2.2.2 K-FAC Approximation 2.2.3 Highlights of Motivation 3 A General View of Normalizing Activations [DELETE] 3.1 Normalizing Activations by Population Statistics 3.2 Local Statistics in a Sample 3.3 Batch Normalization 4 A Framework for Normalizing Activations as Functions [DELETE] 4.1 Normalization Area Partitioning 4.2 Normalization Operation 4.2.1 Beyond Standardization Towards Whitening 4.2.2 Variations of Standardization 4.2.3 Reduced Standardization 4.3 Normalization Representation Recovery 5 Multi-mode and Combinational Normalization [DELETE] 5.1 Multiple Modes 5.2 Combination 6 BN for More Robust Estimation [DELETE] 6.1 Normalization as Functions Combining Population Statistics 6.2 Robust Inference Methods for BN 7 Normalizing Weights [DELETE] 7.1 Constraints on Weights 7.2 Training with Constraints 8 Normalizing Gradients [DELETE] 9 Analysis of Normalization [DELETE] 9.1 Scale Invariance in Stabilizing Training 9.1.1 Auto-Tuning on Learning Rate 9.2 Improved Conditioning in Optimization 9.3 Stochasticity for Generalization 9.3.1 Theoretical Model for Stochasticity 9.3.2 Empirical Analyses for Stochasticity 9.4 Effects on Representation 9.4.1 Constraint on Feature Representation 9.4.2 Effect on Representational Capacity of Model 10 Normalization in Task-Specific Applications [DELETE] 10.1 Domain Adaptation 10.1.1 Domain Generalization 10.1.2 Robust Deep Learning Under Covariate Shift 10.1.3 Learning Universal Representations 10.2 Image Style Transfer 10.2.1 Image Translation 10.3 Training GANs 10.4 Efficient Deep Models 11 Summary and Discussion A Appendix A.1 Back-Propagation Through Eigenvalue Decomposition A.2 Derivation of Constraint Number of Normalization Methods A.3 Proofs of Theorems