Artificial-Neural-Networks-and-Deep-Architectures

DD2437 at KTH

Github repository

• Labs

  • Lab 1
  • Lab 2
  • Lab 3
  • Lab 4

• Lectures:
  Contains the lecture slides and the lecture notes.

• Notes:
  Contains the notes taken during the lectures.

Course Content:

The course is concerned with computational problems in massively parallel artificial neural network (ANN) architectures, which rely on distributed simple computational nodes and robust learning algorithms that iteratively adjust the connections between the nodes by making extensive use of available data. The learning rule and network architecture determine specific computational properties of the ANN. The course offers a possibility to develop the conceptual and theoretical understanding of the computability of ANNs starting from simpler systems and then gradually study more advanced architectures. A wide range of learning types are thus studied – from strictly supervised to purely exploratory unsupervised situations. The course content therefore includes among others multi-layer perceptrons (MLPs), self-organising maps (SOMs), Boltzmann machines, Hopfield networks and state-of-the-art deep neural networks (DNNs) along with the corresponding learning algorithms. An important objective of the course is for the students to gain practical experience of selecting, developing, applying and validating suitable networks and algorithms to effectively address a broad class of regression, classification, temporal prediction, data modelling, explorative data analytics or clustering problems. Finally, the course provides revealing insights into the principles of generalisation capabilities of ANNs, which underlie their predictive power.

Intended learning outcomes:

After completing the course, the students shall be able to:

• describe the structure and the function of the most common artificial neural network types (ANN), e.g. (feedforward) multi layer perceptron, recurrent network, self organising maps, Boltzmann machine, deep belief network, autoencoder, and give examples of their applications

• Explain mechanisms of supervised/unsupervised learning from data- and information processing in different ANN architectures, and give an account for derivations of the basic ANN algorithms discussed in the course

• Demonstrate when and how deep architectures lead to increased performance in pattern recognition and data mining problems

• Quantitatively analyse the process and outcomes of learning in ANNs, and account for their shortcomings, limitations

• Apply, validate and evaluate suggested types of ANNs in typical small problems in the realm of regression, prediction, pattern recognition, scheduling and optimisation

• Design and implement ANN approaches to selected problems in pattern recognition, system identification or predictive analytics using commonly available development tools, and critically examine their effectiveness

in order to:

• Obtain an understanding of the technical potential as well as advantages and limitations of today's learning, adaptive and self-organizing systems,
• Acquire the ANN practitioner’s competence to apply and develop ANN based solutions to data analytics problems.