Applied deep learning : a case-based approach to understanding deep neural networks / Umberto Michelucci.
Work with advanced topics in deep learning, such as optimization algorithms, hyper-parameter tuning, dropout, and error analysis as well as strategies to address typical problems encountered when training deep neural networks. You’ll begin by studying the activation functions mostly with a single ne...
Saved in:
| Online Access: |
Full Text (via Springer) |
|---|---|
| Main Author: | |
| Format: | eBook |
| Language: | English |
| Published: |
[United States] :
Apress,
2018.
|
| Subjects: |
Table of Contents:
- Intro; Table of Contents; About the Author; About the Technical Reviewer; Acknowledgments; Introduction; Chapter 1: Computational Graphs and TensorFlow; How to Set Up Your Python Environment; Creating an Environment; Installing TensorFlow; Jupyter Notebooks; Basic Introduction to TensorFlow; Computational Graphs; Tensors; Creating and Running a Computational Graph; Computational Graph with tf.constant; Computational Graph with tf.Variable; Computational Graph with tf.placeholder; Differences Between run and eval; Dependencies Between Nodes; Tips on How to Create and Close a Session.
- Chapter 2: Single NeuronThe Structure of a Neuron; Matrix Notation; Python Implementation Tip: Loops and NumPy; Activation Functions; Identity Function; Sigmoid Function; Tanh (Hyperbolic Tangent Activation) Function; ReLU (Rectified Linear Unit) Activation Function; Leaky ReLU; Swish Activation Function; Other Activation Functions; Cost Function and Gradient Descent: The Quirks of the Learning Rate; Learning Rate in a Practical Example; Example of Linear Regression in tensorflow; Dataset for Our Linear Regression Model; Neuron and Cost Function for Linear Regression.
- Satisficing and Optimizing a MetricExample of Logistic Regression; Cost Function; Activation Function; The Dataset; tensorflow Implementation; References; Chapter 3: Feedforward Neural Networks; Network Architecture; Output of Neurons; Summary of Matrix Dimensions; Example: Equations for a Network with Three Layers; Hyperparameters in Fully Connected Networks; sof tmax Function for Multiclass Classification; A Brief Digression: Overfitting; A Practical Example of Overfitting; Basic Error Analysis; The Zalando Dataset; Building a Model with tensorflow; Network Architecture.
- Modifying Labels for the softmax Function-One-Hot EncodingThe tensor flow Model; Gradient Descent Variations; Batch Gradient Descent; Stochastic Gradient Descent; Mini-Batch Gradient Descent; Comparison of the Variations; Examples of Wrong Predictions; Weight Initialization; Adding Many Layers Efficiently; Advantages of Additional Hidden Layers; Comparing Different Networks; Tips for Choosing the Right Network; Chapter 4: Training Neural Networks; Dynamic Learning Rate Decay; Iterations or Epochs?; Staircase Decay; Step Decay; Inverse Time Decay; Exponential Decay; Natural Exponential Decay.
- Tensorflow ImplementationApplying the Methods to the Zalando Dataset; Common Optimizers; Exponentially Weighted Averages; Momentum; RMSProp; Adam; Which Optimizer Should I Use?; Example of Self-Developed Optimizer; Chapter 5: Regularization; Complex Networks and Overfitting; What Is Regularization?; About Network Complexity; ℓp Norm; ℓ2 Regularization; Theory of ℓ2 Regularization; tensorflow Implementation; ℓ1 Regularization; Theory of ℓ1 Regularization and tensorflow Implementation; Are Weights Really Going to Zero?; Dropout; Early Stopping; Additional Methods; Chapter 6: Metric Analysis.