(Project during the semester)
 
(22 intermediate revisions by the same user not shown)
Line 4: Line 4:
 
     | otherprograms = 2-IKV
 
     | otherprograms = 2-IKV
 
}}
 
}}
{{Infolist|2-AIN-132|Course information sheet >}}
+
{{Infolist|2-AIN-132|Course information sheet}}
  
 
__TOC__
 
__TOC__
Line 33: Line 33:
 
!Day
 
!Day
 
!Time
 
!Time
!Place
+
!Location
 
!Teacher
 
!Teacher
 
|-
 
|-
Line 46: Line 46:
 
|16:30 - 18:00
 
|16:30 - 18:00
 
|online
 
|online
|[[Endre Hamerlik|Endre Hamerlik]],[[Stefan Pocos|Štefan Pócoš]],
+
|[[Endre Hamerlik|Endre Hamerlik]], [[Stefan Pocos|Štefan Pócoš]]
 
|}
 
|}
  
== Sylabus ==
+
== Syllabus ==
  
 
{| class="alternative table-responsive"
 
{| class="alternative table-responsive"
!Dátum
+
!No.
!Názov prednášky
+
!Date
!Literatúra
+
!Topic
 +
!References
 
|-
 
|-
 +
|01
 
|16.2.
 
|16.2.
|Conditions for passing the course. Introduction, inspiration from neurobiology, brief history of NN, basic concepts. NN with logical neurons.  <!--[http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.intro.L01.4x.pdf slajdy-L01]-->
+
|Conditions for passing the course. Introduction, inspiration from neurobiology, brief history of NN, basic concepts. NN with logical neurons.  [http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.intro.L01.4x.pdf slides-L01]
| [U1/1][U3/1][U5/1]
+
| [U1/1][U3/1][U4/1][U5/1]
 
|-
 
|-
 +
|02
 
|23.2.
 
|23.2.
|Binary and continuous perceptron: supevised learning, error functions, binary classification and regression, linear separability. Relation to the Bayesian classifier.   <!--[http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.perceptron.L02.4x.pdf slajdy-L02]-->
+
|Binary and continuous perceptron: supervised learning, error functions, binary classification and regression, linear separability. Relation to the Bayesian classifier. [http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.perceptron.L02.4x.pdf slides-L02]
|[U1/1-3]
+
|[U1/1-3][U4/2]
 
|-
 
|-
 +
|03
 
|02.3.
 
|02.3.
|Single-layer NS: Linear autoassociation: General Inverse model. Classification into n-classes. Error functions, relation to information theory . <!--[http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.single-layer-models.L03.4x.pdf slajdy-L03]-->
+
|Single-layer NS: Linear autoassociation: General Inverse model. Classification into n-classes. Error functions, relation to information theory. [http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.single-layer-models.L03.4x.pdf slides-L03]  
  
 
|[U4/3][U5/4]
 
|[U4/3][U5/4]
 
|-
 
|-
 +
|04
 
|09.3.
 
|09.3.
|Multilayer perceptron: error back-propagation algorithm. Training, validation, testing. Model selection. <!--[http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.mlp.L04.4x.pdf slajdy-L04]-->
+
|Multilayer perceptron: error back-propagation algorithm. Training, validation, testing. Model selection.   [http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.mlp.L04.4x.pdf slides-L04]  
 
|[U1/4][U4/4]
 
|[U1/4][U4/4]
 
|-
 
|-
 +
|05
 
|16.3.
 
|16.3.
|Modifications of gradient methods, second order optimization, regularization. Optimization problems. <!--[http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.optimization.L05.4x.pdf slajdy-L05]-->
+
|Modifications of gradient methods, second-order optimization, regularization. Optimization problems. [http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.optimization.L05.4x.pdf slides-L05]
 
|[U1/15][U4/11]
 
|[U1/15][U4/11]
 
|-
 
|-
 +
|06
 
|23.3.
 
|23.3.
|Unsupervised learning, feature extraction, neural PCA model. Data visualization: self-organizing map (SOM) model. <!--[http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.unsup.L06.4x.pdf slajdy-L06]-->
+
|Unsupervised learning, feature extraction, neural PCA model. Data visualization: self-organizing map (SOM) model. [http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.unsup.L06.4x.pdf slides-L06]  
 
|[U1/8-9][U5/7]
 
|[U1/8-9][U5/7]
 
|-
 
|-
 +
|07
 
|30.3.
 
|30.3.
|Sequential data modeling: forward NS, relation to n-grams, partially and fully recurrent models, SRN model, BPTT, RTRL algorithm. <!--[http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.seq-models.L07.4x.pdf slajdy-L07]-->
+
|Sequential data modeling: forward NS, relation to n-grams, partially and fully recurrent models, SRN model, BPTT, RTRL algorithm. [http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.seq-models.L07.4x.pdf slides-L07]
 
|[U4/8][U5/6]
 
|[U4/8][U5/6]
 
|-
 
|-
 +
|08
 
|06.4.
 
|06.4.
|Expansion of hidden representation: NS with radial basis functions (RBF), echo state network (ESN). <!--[http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.rbf-esn.L08.4x.pdf slajdy-L08]-->
+
|Expansion of hidden representation: NS with radial basis functions (RBF), echo state network (ESN). [http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.rbf-esn.L08.4x.pdf slides-L08]
 
|[U1/5][U2]
 
|[U1/5][U2]
 
|-
 
|-
 +
|09
 
|13.4.
 
|13.4.
|Deep learning. Convolutional neural networks: introduction. <!--[http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.deep-convol.L09.4x.pdf slajdy-L09]-->
+
|Deep learning. Convolutional neural networks: introduction. [http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.deep-convol.L09.4x.pdf slides-L09]
 
|[U3/6,9, U4/6]
 
|[U3/6,9, U4/6]
 
|-
 
|-
 +
|10
 
|20.4.
 
|20.4.
|Newer models: autoencoders, GRU, LSTM. <!--[http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.autoenc-gated.L10.4x.pdf slajdy-L10]-->
+
|More recent models: autoencoders, GRU, LSTM. [http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.autoenc-gated.L10.4x.pdf slides-L10]
 
|[U3/14,U4/9.1-2]
 
|[U3/14,U4/9.1-2]
 
|-
 
|-
 +
|11
 
|27.4.
 
|27.4.
|Hopfield model: deterministic dynamics, attractors, autoassociative memory, sketch of the stochastic model . <!--[http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.hopfield-aam.L11.4x.pdf slajdy-L11]-->
+
|Hopfield model: deterministic dynamics, attractors, autoassociative memory, sketch of the stochastic model, modern versions. [http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.hopfield-aam.L11.4x.pdf slides-L11]
 
|[U1/13][U5/9]
 
|[U1/13][U5/9]
 
|-
 
|-
 +
|12
 
|04.5.
 
|04.5.
|Stochastic recurrent models: basics of probability theory and statistical mechanics, Boltzmann machine, RBM model, Deep Belief Network.  <!--[http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.stochastic.L12.4x.pdf slajdy]-->
+
|Stochastic recurrent models: basics of probability theory and statistical mechanics, Boltzmann machine, RBM model, Deep Belief Network.  [http://dai.fmph.uniba.sk/courses/NN/Lectures/nn.stochastic.L12.4x.pdf slides-L12]
 
|[U1/11][U3/16]
 
|[U1/11][U3/16]
 
|-
 
|-
 +
|13
 
|11.5.
 
|11.5.
|Recent advances in NN field.
+
|Recent advances in the field. - cancelled.
 
|
 
|
 
|}
 
|}
  
== Literatúra ==
+
== References ==
  
* Farkaš I. (2016). [http://dai.fmph.uniba.sk/courses/NN/neural-networks.2016.pdf Neural networks]. Knižničné a edičné centrum FMFI UK v Bratislave. Slajdy k prednáškam.
+
* Farkaš I. (2016). [http://dai.fmph.uniba.sk/courses/NN/neural-networks.2016.pdf Neural networks]. Knižničné a edičné centrum FMFI UK v Bratislave. Slajdes to the lectures (not updated).
 
* Haykin S. (2009). Neural Networks and Learning Machines (3rd ed.). Upper Saddle River, Pearson Education (k dispozícii na štúdium v knižnici FMFI, ale aj stiahnuteľné z webu). [U1]
 
* Haykin S. (2009). Neural Networks and Learning Machines (3rd ed.). Upper Saddle River, Pearson Education (k dispozícii na štúdium v knižnici FMFI, ale aj stiahnuteľné z webu). [U1]
 
* Jaeger H. (2007). [http://www.scholarpedia.org/article/Echo_state_network Echo-state network]. Scholarpedia, 2(9):2330. [U2]
 
* Jaeger H. (2007). [http://www.scholarpedia.org/article/Echo_state_network Echo-state network]. Scholarpedia, 2(9):2330. [U2]
Line 119: Line 133:
 
* Kvasnička V., Beňušková., Pospíchal J., Farkaš I., Tiňo P. a Kráľ A. (1997). [http://dai.fmph.uniba.sk/courses/NN/UvodDoTeorieNS.pdf.zip Úvod do teórie neurónových sietí]. Iris: Bratislava. [U5]
 
* Kvasnička V., Beňušková., Pospíchal J., Farkaš I., Tiňo P. a Kráľ A. (1997). [http://dai.fmph.uniba.sk/courses/NN/UvodDoTeorieNS.pdf.zip Úvod do teórie neurónových sietí]. Iris: Bratislava. [U5]
  
== Podmienky absolvovania a spôsob hodnotenia ==
+
== Conditions and grading ==
  
* Odovzdanie aspoň dvoch (z troch) projektov počas semestra (max. 3x5 = 15 bodov). Termíny na odovzdanie projektov budú včas uvedené na stránke. Projekty budú ponúkať aj bonusy (max. 2 body).
+
* Submission of at least two (out of three) functioning projects during the semester (max. 3x5 = 15 points). The deadlines will be announced on the webpage. The projects will offer bonuses (max. 2 points).
* Študent môže byť požiadaný, aby prezentoval svoj funkčný kód na cvičení ako súčasť odovzdania projektu. Na cvičeniach sa budú robiť malé úlohy, za ktoré tiež bude možné získať body (max. 17 za semester).
+
* The exercises will consist of small tasks to be completed, and will be graded (max. 17 points during the semester). You have to acquire at least 7 points from exercises.
* Absolvovanie záverečnej písomno-ústnej skúšky (3 otázky po 5 bodov, pseudonáhodný výber). Na skúšku sa môže prihlasiť len ten, kto má oznámkované aspoň dva projekty s fungujúcim kódom. Skúška je povinná, ústna, je nutné získať aspoň 6 bodov.
+
* Passing the final oral exam (3 questions,  5 points each, pseudorandom choice). To register for the exam, you have to have at least two functioning projects graded. The exam is compulsory, you have to get at least 6 points.
* Prednášky sú dobrovoľné, za účasť však možno získať max. 3 body.
+
* The lectures are not compulsory, but you can get up to 3 points for participation.
* <b>Celkové hodnotenie:</b> A (50-46), B (45-41), C (40-36), D (35-31), E (30-26), Fx (25-0).
+
* <b>Overall grading:</b> A (50-46), B (45-41), C (40-36), D (35-31), E (30-26), Fx (25-0).
  
== Projekty počas semestra ==
+
== Projects during the semester ==
  
* Projekt, spolu so zdrojovým kódom, sa odovzdáva k stanoveného termínu. Za každý oneskorený kalendárny deň (max. 5 dní) odovzdania vzniká malus 1 bod (až do nuly). Úspešný projekt (fungujúci model) odovzdaný viac ako týždeň po termíne sa započítava bez bodov.
+
* The project, together with the source code, is to be submitted before the deadline. Late submissions are penalized by -1 point each day. The successful project (i.e. with a well functioning model) submitted more than 5 days after the deadline counts, without points.
* Pri projekte sa hodnotí najmä obsah, ale záleží aj na forme (čitateľnosť, úprava). Obsah projektu musí byť zrozumiteľný, aby bolo jasné, čo a ako ste robili, t.j. grafické výstupy kombinované so sprievodným textom. Za výraznejšie formálne a gramatické nedostatky môže byť strhnutý bod.
+
* The projects are graded mainly based on content, but the form is considered, too (readability). The content should be comprehensible, i.e. graphical outputs combined with text.  
* Model je naprogramovaný v jazyku Python a projekt sa odovzdáva vo formáte PDF.
+
* The model is to be implemented in Python and the project must be submitted as a PDF (no title page is required, the title and your name is enough).
* Pri zistení plagiátorstva študent(ka) získava automaticky nula bodov z projektu a nebude pripustený(-á) ku skúške.
+
* In case of plagiarism detection, the student automatically receives zero points from the project and will not be admitted to the exam.

Latest revision as of 13:10, 18 May 2021

Neural Networks 2-AIN-132

Course information sheet

The aim of the course is to get acquainted with the basic concepts and algorithms of learning artificial neural networks and their use in solving various problems. Theoretical lectures are combined with practical modeling in Python exercises

News

Partial changes were made in lectures and exercises last year. Some older parts have been shortened, newer topics have been added. The syllabus is updated, as well as the evaluation of course activities.


Schedule

Type Day Time Location Teacher
Lecture Tuesday 09:50 - 11:20 online Igor Farkaš
Exercise Thursday 16:30 - 18:00 online Endre Hamerlik, Štefan Pócoš

Syllabus

No. Date Topic References
01 16.2. Conditions for passing the course. Introduction, inspiration from neurobiology, brief history of NN, basic concepts. NN with logical neurons. slides-L01 [U1/1][U3/1][U4/1][U5/1]
02 23.2. Binary and continuous perceptron: supervised learning, error functions, binary classification and regression, linear separability. Relation to the Bayesian classifier. slides-L02 [U1/1-3][U4/2]
03 02.3. Single-layer NS: Linear autoassociation: General Inverse model. Classification into n-classes. Error functions, relation to information theory. slides-L03 [U4/3][U5/4]
04 09.3. Multilayer perceptron: error back-propagation algorithm. Training, validation, testing. Model selection. slides-L04 [U1/4][U4/4]
05 16.3. Modifications of gradient methods, second-order optimization, regularization. Optimization problems. slides-L05 [U1/15][U4/11]
06 23.3. Unsupervised learning, feature extraction, neural PCA model. Data visualization: self-organizing map (SOM) model. slides-L06 [U1/8-9][U5/7]
07 30.3. Sequential data modeling: forward NS, relation to n-grams, partially and fully recurrent models, SRN model, BPTT, RTRL algorithm. slides-L07 [U4/8][U5/6]
08 06.4. Expansion of hidden representation: NS with radial basis functions (RBF), echo state network (ESN). slides-L08 [U1/5][U2]
09 13.4. Deep learning. Convolutional neural networks: introduction. slides-L09 [U3/6,9, U4/6]
10 20.4. More recent models: autoencoders, GRU, LSTM. slides-L10 [U3/14,U4/9.1-2]
11 27.4. Hopfield model: deterministic dynamics, attractors, autoassociative memory, sketch of the stochastic model, modern versions. slides-L11 [U1/13][U5/9]
12 04.5. Stochastic recurrent models: basics of probability theory and statistical mechanics, Boltzmann machine, RBM model, Deep Belief Network. slides-L12 [U1/11][U3/16]
13 11.5. Recent advances in the field. - cancelled.

References

  • Farkaš I. (2016). Neural networks. Knižničné a edičné centrum FMFI UK v Bratislave. Slajdes to the lectures (not updated).
  • Haykin S. (2009). Neural Networks and Learning Machines (3rd ed.). Upper Saddle River, Pearson Education (k dispozícii na štúdium v knižnici FMFI, ale aj stiahnuteľné z webu). [U1]
  • Jaeger H. (2007). Echo-state network. Scholarpedia, 2(9):2330. [U2]
  • Goodfellow I., Bengio Y., Courville A. (2016). Deep Learning. MIT Press. [U3]
  • Zhang A. et al. (2020). Dive into Deep Learning. An interactive deep learning book with code, math, and discussions, based on the NumPy interface. [U4]
  • Kvasnička V., Beňušková., Pospíchal J., Farkaš I., Tiňo P. a Kráľ A. (1997). Úvod do teórie neurónových sietí. Iris: Bratislava. [U5]

Conditions and grading

  • Submission of at least two (out of three) functioning projects during the semester (max. 3x5 = 15 points). The deadlines will be announced on the webpage. The projects will offer bonuses (max. 2 points).
  • The exercises will consist of small tasks to be completed, and will be graded (max. 17 points during the semester). You have to acquire at least 7 points from exercises.
  • Passing the final oral exam (3 questions, 5 points each, pseudorandom choice). To register for the exam, you have to have at least two functioning projects graded. The exam is compulsory, you have to get at least 6 points.
  • The lectures are not compulsory, but you can get up to 3 points for participation.
  • Overall grading: A (50-46), B (45-41), C (40-36), D (35-31), E (30-26), Fx (25-0).

Projects during the semester

  • The project, together with the source code, is to be submitted before the deadline. Late submissions are penalized by -1 point each day. The successful project (i.e. with a well functioning model) submitted more than 5 days after the deadline counts, without points.
  • The projects are graded mainly based on content, but the form is considered, too (readability). The content should be comprehensible, i.e. graphical outputs combined with text.
  • The model is to be implemented in Python and the project must be submitted as a PDF (no title page is required, the title and your name is enough).
  • In case of plagiarism detection, the student automatically receives zero points from the project and will not be admitted to the exam.