• to introduce the connectionist approach to Artificial Intelligence and Cognitive Science;
• to introduce the connectionist type of computation which is quite different from the classic von Neuman style of computation;
• to learn the basic concepts and methods of connectionism, its pros, cons and limitations;
• to get some hands-on experience with connectionist models.

• demonstrate familiarity with the central concepts and ideas of connectionism;
• present arguments and evidence coherently in verbal and written form;
• debate and discuss theoretical issues and concepts;
• demonstrate a detailed and critical understanding of connectionist papers - both theoretical and experimental, identify and explain significant developments and contradictions;
• select an appropriate network architecture and build a particular network as a solution of a particular (not very complex) task using the PDP package or a con-nectionist environment;
• critically evaluate particular connectionist cognitive models;
• argue about the advantages and limits of the connectionist approach.

Parallel Distributed Processing: Exploration in the Microstructure of Cognition, vol. 1&2, MIT Press.

[PDP3] McClelland & Rumelhart (1988),

Explorations in Parallel Distributed Processing, MIT Press.

[TNC] Hertz, Krogh & Palmer (1991),

Introduction to the Theory of Neural Computation, (Lecture Notes of the Santa Fe Institute), Addison-Wesley.

[NC1] Anderson & Rosenfeld (1988),

Neurocomputing: Foundation of Research, MIT Press.

[NC2] Anderson, Pellionisz & Rosenfeld (1990),

Neurocomputing 2: Directions for Research, MIT Press.

[CM] Waltz & Feldman (1988),

Connectionist Models and their Implications: Readings from Cognitive Science,, Ablex.

[NNAI] Zeidenberg (1990),

Neural Network Models in Artificial Intelligence, Ellis Horwood.

[NN] Mueller & Reinhardt (1990),

Neural Networks: An Introduction, Springer.

[NCM] Levine, D. (1991),

Introduction to Neural and Cognitive Modeling, Lawrence Erlbaum, Hillsdale, NJ.

[CP] Quinlan, P. (1991),

Connectionism and Psychology: A Psychological Perspective on New Connectionist Research, Harvester, New York.

[ANN] Patterson, D.,

Artificial Neural Networks, Prentice Hall, 1996.

• introduction Ð basic concepts, biological basis, history;
• relaxation search Ð methods for searching for the best solution to a problem;
• representation Ð how concepts, visual images, facts, and complex data struc-tures are embodied in a neural network;
• learning Ð algorithms that allow connectionist networks to learn from experi-ence, either on their own or with supervision from a teacher or critic;
• cognitive models Ð how connectionist techniques can be used to explain human cognitive behaviors such as perception, memory, and language;
• concluding discussions Ð pros and cons, open problems, relations with the symbolic approach to AI.

Introduction

Topic 1: Biological basis of neural networks (neural networks: biological and artificial). The connectionist approach to AI and Cognitive Science. Historical remarks.
Seminar: Demonstration of a tape recording of a NN reading (NETtalk) and composing music (Mozer). Discussion.

Required readings:

• [PDP1] Chapters 1 and 4.
• [TNC] Chapter 1
• [ANN] Chapter 1

Topic 2: General Connectionist Architecture. Basic concepts. Connectionist Model usage (relaxation): heteroassociator (pattern associator) and autoassociator (associative memory). Learning paradigms: supervised, unsupervised, reinforcement learning.
Seminar: Discussing different particular architectures.

Required readings:

• [PDP1] Chapter 2 and 4.
• [ANN] Chapter 2.

Relaxation Search

Topic 3: Associative Memory: Interactive Activation and Competition Model. Properties: content addressability, graceful degradation, default values, spontaneous generalization.
Lab: Experiments with the computer simulation: Sharks and Jets.

Required reading:

• [PDP3] Chapter 2.

Topic 4: Associative Memory: Hopfield Networks. Potential Functions. The constraint satisfaction problem: Global extremum. Physics Analogy. Bolzmann Machines.
Lab: Experiments with the computer simulation: Necker cube.

Required readings:

• [PDP1] Chapter 7.
• [PDP3] Chapter 3.
• [TNC] Chapters 2 and 4.
• [ANN] Chapter 10.

Representation

Topic 5: Local vs. distributed representation. Schema representation. The Schema Model. Harmony Theory.
Lab: Experiments with the computer simulation: Room schema.

Required readings:

• [PDP1] Chapter 3.
• [PDP2] Chapter 14.
• [PDP3] Chapter 3.

Topic 6: Complex structure representation. Relation representation. Coarse coding. Tensor Product Formalism.
Seminar: Building examples of representations of complex structures.

Learning

Topic 7: Learning in Single-layered Networks. Hebbian rule. Associative memo-ries. Least Mean Squares Learning Rule. Perceptron Learning rule.
Lab: Experiments with the computer simulation: Pattern association.

Required readings:

• [PDP3] Chapter 4.
• [TNC] Chapter 5.

Topic 8: Supervised Learning in Multi-layered Networks. Limitations of Perceptrons and LMS. Back-propagation learning rule.
Lab: Experiments with the computer simulation: XOR problem.

Required readings:

• [PDP1] Chapter 8.
• [PDP3] Chapter 5.
• [TNC] Chapter 6.

Topic 9: Unsupervised learning: Competitive Learning. Kohonen topographic maps.
Lab: Experiments with the computer simulation: Clustering the Jets and Sharks.

Required readings:

• [PDP1] Chapter 5.
• [PDP3] Chapter 6
• [TNC] Chapter 9.
• [ANN] Chapter 14

Topic 10: Unsupervised learning: Brain-State-in-a-Box Model. Hebbian learning. Adaptive Resonance Theory.
Lab: Experiments with the computer simulation: BSB model.

Required readings:

• [TNC] Chapter 9.
• [ANN] Chapter 15.

Topic 11: Reinforcement Learning: The credit assignment problem. Adaptive search elements. Associative reward-penalty algorithm.
Seminar: Examples and discussion on learning methods.

Required reading:

• [TNC] Chapter 7.

Cognitive Models

Topic 12: Perception Models.
Lab: Experiments with the computer simulation: Interactive Activation Model of Reading. Exploring the role of context.

Required readings:

• [PDP3] Chapter 7.
• [PDP2] Chapter 16.

Topic 13: Natural Language Understanding. Learning the past tenses of English verbs. Parsing and role assignment.
Seminar: Discussing the adequacy of the models.

Required reading:

• [PDP2] Chapter 18, 19.

Topic 14: Memory and Reasoning models.
Lab: Experiments with the computer simulation: DMA model. Discussing the adequacy of the models.

Required reading:

• [PDP2] Chapter 17.

Topic 15: Pros and cons of neural networks. Comparing connectionist and sym-bolic approaches. The symbol vs. subsymbol debate. Historical lessons.
Seminar: Final discussion about the advantages and disadvantages of the connec-tionist and symbolic approaches and their relations.

• Homework assignment 1: To read and comment a connectionist paper (classic or modern) writing a 3000 words paper and defending it before the group.
• Homework assignment 2: Implementation project Ð to develop a particular model using an appropriate environment and performing simulation experiments.
• Multiple choice test.

Grading procedure:

25% written paper

• 25% public defense
• 25% implementation project
• 25% multiple choice test