---
title: "DSPA2: Data Science and Predictive Analytics (UMich HS650)"
subtitle: "<h2><u>Deep Learning, Neural Networks</u></h2>"
author: "<h3>SOCR/MIDAS (Ivo Dinov)</h3>"
date: "`r format(Sys.time(), '%B %Y')`"
tags: [DSPA, SOCR, MIDAS, Big Data, Predictive Analytics] 
output:
  html_document:
    theme: spacelab
    highlight: tango
    includes:
      before_body: SOCR_header.html
    pandoc_args: [
      "--verbose",
      "+RTS", "-K4000m",
      "-M8G", "-RTS"
    ]
    keep_md: true
    toc: true
    number_sections: true
    toc_depth: 4
    toc_float:
      collapsed: false
      smooth_scroll: true
    code_folding: show
    self_contained: yes
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
# Set environment variable for Pandoc to increase the memory allocation
Sys.setenv(PANDOC_OPTS = "+RTS -K2048m -RTS")

options(knitr.knit.hook = function() {
  Sys.setenv(PANDOC_OPTS = "+RTS -K2048m -RTS")
})
```

This [DSPA2 module](https://socr.umich.edu/DSPA2/DSPA2_notes/14_DeepLearning.html) is intentionally split into three complementary parts to facilitate easy compilation of the enormous amount of content that was
in the [expanded (old) DSPA2 Chapter 14 Chapter 14 (Deep Learning, Neural Networks)](https://socr.umich.edu/DSPA2/DSPA2_notes/14_DeepLearning_old.html).

# Deep Learning Neural Networks

[Deep learning](https://en.wikipedia.org/wiki/Deep_learning#Deep_neural_%0Anetwork_architectures) is a special branch of machine learning using a collage of algorithms to model high-level motifs in data. Deep learning resembles the biological communications between brain neurons in the central nervous system (CNS), where synthetic graphs represent the CNS network as nodes/states and connections/edges between them. For instance, in a simple synthetic network consisting of a pair of connected nodes, an output sent by one node is received by the other as an input signal. When more nodes are present in the network, they may be arranged in multiple levels (like a multiscale object) where the $i^{th}$ layer output serves as the input of the next $(i+1)^{st}$ layer. The signal is manipulated at each layer and sent as a layer output downstream and interpreted as an input to the next, $(i+1)^{st}$ layer, and so forth. Deep learning relies on multiple layers of nodes and many edges linking the nodes forming input/output (I/O) layered grids representing a multiscale processing network. At each layer, linear and non-linear transformations are converting inputs into outputs.

In this chapter, we explore the R-based deep neural network learning and demonstrate state-of-the-art deep learning models utilizing CPU and GPU for fast training (learning) and testing (validation). Other powerful deep learning frameworks include *TensorFlow*, *Theano*, *Caffe*, *Torch*, *CNTK* and *Keras*.

*Neural Networks vs. Deep Learning*: Deep Learning is a machine learning strategy that learns a deep multi-level hierarchical representation of the affinities and motifs in the dataset. Machine learning Neural Nets tend to use shallower network models. Although there are no formal restrictions on the depth of the layers in a Neural Net, few layers are commonly utilized. Recent methodological, algorithmic, computational, infrastructure and service advances overcome previous limitations. In addition, the rise of *Big Data* accelerated the evolution of *classical Neural Nets* to *Deep Neural Nets*, which can now handle lots of layers and many hidden nodes per layer. The former is a precursor to the latter, however, there are also *non-neural* deep learning techniques. For example, *syntactic pattern recognition methods* and *grammar induction discover hierarchies*.

**Background**: Review [Chapter 6 (Black Box Machine-Learning Methods: Neural Networks, Support Vector Machines, and Random Forests)](https://socr.umich.edu/DSPA2/DSPA2_notes/06_ML_NN_SVM_RF_Class.html) prior to proceeding.


This [DSPA2 module](https://socr.umich.edu/DSPA2/DSPA2_notes/14_DeepLearning.html) is split into three complementary parts to facilitate easy compilation of the enormous amount of content that was
in the [expanded (old) DSPA2 Chapter 14 Chapter 14 (Deep Learning, Neural Networks)](https://socr.umich.edu/DSPA2/DSPA2_notes/14_DeepLearning_old.html). 

 - [Part 1 of the DSPA2 Chapter 14 (Deep Learning, Neural Networks)](https://socr.umich.edu/DSPA2/DSPA2_notes/14_DeepLearning_Part1.html),
 - [Part 2 of the DSPA2 Chapter 14 (Deep Learning, Neural Networks)](https://socr.umich.edu/DSPA2/DSPA2_notes/14_DeepLearning_Part2.html),
 - [Part 3 of the DSPA2 Chapter 14 (Deep Learning, Neural Networks)](https://socr.umich.edu/DSPA2/DSPA2_notes/14_DeepLearning_Part3.html), and
 - [Part 4 of the DSPA2 Chapter 14 (Deep Learning, Neural Networks)](https://socr.umich.edu/DSPA2/DSPA2_notes/14_DeepLearning_Part4.html).


<footer> <center> <a href="https://socr.umich.edu/">SOCR Resource</a> Visitor number 
<img class="statcounter" src="https://c.statcounter.com/5714596/0/038e9ac4/0/" alt="Web Analytics" align="middle" border="0">
<script type="text/javascript"> var d = new Date(); document.write(" | " + d.getFullYear() + " | "); </script> <a href="https://socr.umich.edu/img/SOCR_Email.png"> <img alt="SOCR Email" title="SOCR Email" src="https://socr.umich.edu/img/SOCR_Email.png" style="border: 0px solid ;"> </a> 
</center> </footer>
<!-- Google Analytics (classic) -->
<script src="https://www.google-analytics.com/urchin.js" type="text/javascript"></script><script type="text/javascript"> _uacct = "UA-676559-1"; urchinTracker(); </script>
