G89.3403 - An Introduction to Learning - Todd M. Gureckis


G89.3403


Meeting time/place

Class meets weekly on Tuesday 12:30-2:30pm in 469.

Syllabus

A PDF of the syllabus is available here.

Blurb

Learning is a critical component of adaptive behavior in animals and humans. This course will expose students to key concepts, theories, and experimental paradigms for studying human learning. The goal is to provide an integrative view of the area that crosses both classic approaches (e.g., classical conditioning, instrumental learning) as well as modern issues (e.g., cognitive neuroscience of learning, language learning, social learning, computational approaches). Special attention will be given to exploring what is known about the neural substrates of learning and memory, as well as computational and mathematical theories. Students will leave the course as sophisticated consumers of learning research and be able to apply learning concepts directly to their own research. This course fulfills part of the introductory “core” cognition requirements for the NYU psychology program. As such there will be a series of take-home exams throughout the semester that assess mastery of the key concepts.



Instructor

Todd M. Gureckis
email: todd.gureckis@nyu.edu
website: here, lab
office: 859
office hours: after class Tuesday or by appt.



Textbook

Learning and Memory: From Brain to Behavior by Gluck, Mercado, and Myers [website]
Readings from the book will be supplemented with additional research articles distributed from this website.



Course Announcements

1.24.2011 - Dear site visitor/websurfer. Due to copyright restrictions, I have removed all the links to the PDFs, but the PDFs of my lecture notes are available.

1.24.2011 - Dear class, the final will be sent back (graded soon). I only recently received all of them.

12.11.2010 - Live-blogging event! Dear class, i wanted to keep you in the loop on grading to the midterm. Despite spending the last three days almost entirely devoted to this, I've still only read 60% of the 132 essay responses (fortunately the multiple choice was easier to grade)!! I will try to finish and send back this weekend (it is my priority so you have a chance to get feedback). However, I apologize I wasn't able to get this finished by Friday. -T

12.11.2010 - The final has been posted! (here and below). Previous exam grades and comments are coming ASAP. It is less than 1/2 as long as the previous exam for just about 1/2 as much time to complete. The earlier you can turn in the final the better for me (giving me more time to grade).

10.19.2010 - Note that in class on Tuesday we agreed my a straw poll to reschedule the class canceled on the 9th for the 23rd. This way we can avoid marathon 2.5 hour lectures to make up lost time.

11.3.2010 - dear class, I just realized that I have to attend a grant panel meeting next week in d.c. that conflicts with our class meeting time. one option is to reschedule for sometime on Friday next week instead of Tuesday. However, given the scheduling issues this would incur in an enrolled class of almost 30 people, it might be better to just do like this week and combine two lectures into one longer session. So, plan on lectures 10 and 11 for Tuesday the 16th. Readings will be updated soon. To close, a link from Dylan: superhell. (btw, superhell is closely related to where you go to work on your midterm, which, by the way, is till due on the 9th).

10.26.2010 - Note that the midterm exam will be due Nov. 9th before class. Please email your responses including the [nyulearn] in the subject line.

10.18.2010 - an APS article of interest by Roddy Roediger.

10.16.2010 - It wasn't easy, but the set of papers for Instrumental Conditioning (week 1) have been decided. Consider the papers for this week shown below as "set in stone". Week 2 of instrumental condition is still evolving.

10.15.2010 - Dear class, a bit behind on choosing instrumental learning papers (some of which I have to scan in). There are a couple to get you started. Will update this weekend.

10.14.2010 - Website colors fixed for Firefox!

10.5.2010 - Ok, fixed some bugs in site and syllabus. Also added references to all slides indicating what other materials I am borrowing from (other books, lectures, etc..). Might be useful to you all.

9.28.2010 - Some minor updates to readings. We will try to move a little faster so I have moved some of the theory papers for the next section (Classical Conditioning) forward by a day. It may be we blend this a bit across weeks. Trust the webpage and not the PDF syllabus as things are adjusting as we go. Thanks.

9.21.2010 - Hi, I have posted a short survey. If you could please fill it out and email your response it would be greatly appreciated. I'm looking to know more who is taking the course (I know many of you but not all), and what you are hoping to get out of it. Also, I'm interested in some early feedback about the course structure. If you prefer not to respond to any question because you feel you can't do it anonymously, please skip it and you will have another chance to give feedback later in the semester via NYU's standard mid-semester course survey.

9.15.2010 - A couple minor updates to syllabus and reading schedule for weeks 2&3. Please press and hold the shift key while pressing reload in your web-browser to ensure you are seeing the most recent versions of everything.

9.14.2010 - Welcome to class! The slides for week 1 are posted. Also, note that the readings for week 1 are longer, but then level off for week 2. As a result, try to catch up on week 1 immediately and glance at week 2 stuff before class. This will help promote some discussion as you will have been exposed to the ideas in the lectures a bit. Eventually, we want to switch over so that people are doing the reading before class rather than after. Hopefully we can get there by week 3. Also, the readings will be posted as we go (don't take the syllabus too literally more than 3-4 weeks out. Things may change a bit as we go.)

9.4.2010 - **IMPORTANT** I had received word from Academic Affairs that our class meeting time is moving to Tuesday (for the reasons mentioned below). The time will be moved slightly (12:30-2pm). The room is Meyer 469. For the first class meeting (Sept. 10) we will still meet at the originally scheduled time (Friday 12-2pm, room 469). At this meeting we can iron our any details of the class switch.

8.24.2010 - For various reasons (religious holidays, miniconvention, my travel schedule this semester) we are looking into moving the class meeting time. Nothing is definite yet, but will post if we find out anything.

8.19.2010 - This is where course announcements will appear. Please check back often.



Schedule (with links to slides)
coming soon

Date Description Slides
Sept. 14

1. Introduction/Overview - What is learning? Historical ideas and the birth of the modern science of learning. Additional topics include learning/performance, innate behaviors versus adaptation (nature/nurture), critical periods, models and mechanisms, and levels of analysis


Textbook reading: Gluck, Ch. 1 - The psychology of learning and memory

Watson, J.B. (1994) “Psychology as the Behaviorist Views It” Psychological Review, 101(2), 248-253. [PDF]

Hull, C.L. (1937) “Mind, Mechanism, and Adaptive Behavior” Psychological Review, 44 (1), 1-32.[PDF]

Pinker, S. (2004) “Why nature & nurture won’t go away” Daedalus, 133(4), 5-17. [PDF]

Optional Readings (discussed in lecture):

Phattanasri, P., Chiel, H.J., and Beer, R.D. (2007) “The Dynamics of Associative Learning in Evolved Model Circuits” Adaptive Behavior, 15(4), 377-396.[PDF]

Skinner, B.R. (1950) “Are Theories of Learning Necessary?” Psychological Review, 57 (4), 193-216.[PDF]

[slides]
Sept. 21

2. Basic concepts in the neuroscience of learning and memory - In the following weeks we will explore a number of basic phenomena of learning. However, it is helpful to begin by casting these ideas against the backdrop of contemporary neuroscience. Today’s lecture will be a basic whirl-wind tour of the neural processes thought to underly learning and memory. We’ll talk about the function of neurons, the specialization of function in the brain, basic learning mechanisms (hebbian learning, LTP), and modern techniques for studying learning and memory (fMRI, EEG, etc...)

Since this is primarily a introduction/review for students who have no prior exposure to neuroscience or psychology, we will default primarily to the book for neuroscience background, then turn to the paper readings for the evolution of a view of the organization of learning and memory in the brain. If you already know all this, great, but it's always important to look back and realize exactly WHY you know this.


Textbook reading: Gluck, Ch. 2 - The neuroscience of learning and memory

Lashley, K.S. (1950) “In search of the Engram” Society of Experimental Biology Symposium, 4, 454-482. [PDF]

Scoville, W.B. and Milner, B. (1957) “Loss of Recent Memory After Bilateral Hippocampal Lesions” Journal of Neurology, Neurosurgery and Psychiatry, 20, 11-21.[PDF]

Posner, M.I., Peteresen, S.E., Fox, P.T., Raichle, M.E. (1988) “Localization of Cognitive Operations in the Human Brain” Science, 240, 1627-1631.[PDF]

Squire, L.R. (1992) “Declarative and Nondeclarative Memory: Multiple Brain Systems Supporting Learning and Memory” Journal of Cognitive Neuroscience, 4 (3), 232-243.[PDF]

[slides]
Sept. 28

3. Non-associative/perceptual forms of learning - This lecture will cover basic, non-associative forms of learning including perceptual learning, habituation/sensitization (incl. habituation as a empirical technique for studying learning in non-linguistic animals), latent learning, feature learning, imprinting, priming, repetition suppression, etc....


Textbook reading: Gluck, Ch. 6 - Non-Associative Learning

Foundational Work:
Tolman, E.C. (1948) “Cognitive Maps in Rats and Men” Psychological Review, 55(4), 189-208. [PDF]

Contemporary Work:
Goldstone, R.L. (1998) "Perceptual Learning" Annual Review of Psychology, 49, 585-612. [PDF]

Grill-Spector, K., Henson, R. and Martin, A. (2006) "Repetition and the brain: neural models of stimulus-specific effects" Trends in Cognitive Sciences, 10(1), 14-23. [PDF]

Some theory:
Barlow, H.B. (1989) “Unsupervised Learning” Neural Computation, 1, 295-311.[PDF]

Optional (might be useful if you have an interest in study non-verbal primates):
Jeffrey, W. E., & Cohen, L. B. (1971). "Habituation in the human infant." In H. Reese (Ed.), Advances in child development and behavior, Vol.6 (pp. 63-97). New York: Academic Press. [PDF]

[slides]
Oct. 5

4. Classical Conditioning I - Pavlov, basic procedure, phenomena and terms (CS/US, etc...), basic findings, blocking and overshadowing, etc..., Resorla-Wagner model, Pearce-Hall model and the role of attention/associability in classical conditioning, basic neural substrates of classical conditioning, interactions with other learning systems (e.g., role of hippocampus in trace conditioning)


Textbook reading: Gluck, Ch. 7 - Classical Conditioning

Foundational Work:
Rescorla, R.A. (1998) “Pavlovian Conditioning: It's not what you think it is” American Psychologist, 43(4), 151-160. [PDF]

A theory:
Rescorla, R.A. and Wagner, A.R.(1971) “A Theory of Pavlovian Conditioning: Variations in the Effectiveness of Reinforcement and Non-reinforcement” in Black, A.H. & Prokasy, W.F. (eds.), Classical conditioning II: Current research and theory (pp. 64-99). New York: Appleton-Century-Crofts. [PDF]

Example empirical result:
Clark, R.E. and Squire, L.R. (1998) “Classical Conditioning and Brain System: The Role of Awareness” Science, 280, 77-81. [PDF]

More theories:
Dayan, P., Kakade, S. and Montague, P.R. (2000) “Learning and selective attention” Nature Neuroscience, 3, 1218-1223. [PDF]

Optional Theory Paper:
Pearce, J.M. and Hall, G. (1980) “A Model for Pavlovian Learning: Variations in the Effectiveness of Conditioned by Not of Unconditioned Stimuli” Psychological Review, 87, 532-552. [PDF]

[slides]
[RW XLS]
[RW Numbers]
Oct. 12

5. Classical Conditioning II - modern theories including causal interpretations of classical conditioning, context-dependent learning, second-order condition (temporal-difference model and relationship to Rescorla-Wagner), neural basis of prediction errors


Learning and prediction errors:
Niv, Y. and Schoenbaum, G. (2008) “Dialogues on prediction errors” Trends in Cognitive Science, 12(7), 265-72. [PDF]

Schultz, W., Dayan, P. & Montague, P.R. (1997) “A neural substrate of prediction and reward” Science, 275, 1593. [PDF]

"Latent Cause" models of classical conditioning:
Courville, A.C., Daw, N.D., Gordon, G.J., and Touretzky, D.S. (2003) “Model Uncertainty in Classical Conditioning” Neural Information Processing Systems, 16, 977-984. [PDF]

Context dependence of conditioning:
Gershman, S.J., and Blei, D. and Niv, Y. (2009) “Context, learning, and extinction” Psychological Review, 117(1), 197-209. [PDF]

[slides]
Oct. 19

6. Instrumental Conditioning I - law of effect, role of reinforcement, stimulus control, choice behavior, matching law, melioration, concurrent schedules, self control/impulsivity, habits and planning, superstitious responding (special thanks to nathaniel daw for sharing slides and thoughts on the instrumental section)


Textbook reading: Gluck, Ch. 8 - Instrumental Conditioning

Dickinson, A. (1994) “Instrumental Conditioning” Animal Learning and Cognition, Chapter 3, pg 45-79. [PDF]

Herrnstein, R.J. (1970) “On the law of effect” Journal of the Experimental Analysis of Behavior, 13, 243-266. [PDF]

Skinner, B.F. (1948) “Superstition in the Pigeon” Journal of Experimental Psychology, 38, 168-172. [PDF]

Dickinson, A. (1985) “Actions and Habits: The Development of Behavioral Autonomy” Philosophical Transactions of the Royal Society of London. Series B, Biological, 38, 168-172. [PDF]

[slides]
Oct. 26

7. Instrumental Conditioning II - habits and planning (model-based and model-free RL), motivation for behavior,instrumental condition and decision theory, matching behavior and the representation of value in the brain (special thanks to nathaniel daw for sharing slides and thoughts on the instrumental section)


Balleine, B.W., Daw, N.D., and O'Doherty, J.P. (2008) “Multiple Forms of Value Learning and the Function of Dopamine” Neuroeconomics: Decision Making and the Brain (Ed. Glimcher, P.W., Camerer, C.F., Fehr, E., and Poldrack, R.A., Chapter 24, 367-387. [PDF]

Niv, Y., Daw, N.D., Joel, D., Dayan, P. (2006) “Tonic dopamine: Opportunity costs and the control of response vigor” Psychopharmachology, 191(1), 507-520. [PDF]

Sugrue, L.P. (2004) “Matching behavior and the representation of value in the parietal cortex” Science, 304, 1782. [PDF]

Herrnstein, R.J. and Prelec, D. (1991) “Melioration: A Theory of Distributed Choice" The Journal of Economic Perspectives, 5(3), pg 137-156. [PDF]

[slides]
[midterm]
Nov. 2

8. Generalization and Discrimination - Pearce (configural) vs. R-W (elemental), stimulus generalization, attention learning, context dependent learning

Textbook reading: Gluck, Ch. 9 - Generalization, Discrimination, and the Representation of Similarity

Mitchell, T.M. (1980). The need for biases in learning generalizations (Report CBM- TR-5-110). New Brunswick, NJ: Rutgers University, Department of Computer Science. [PDF]

Shepard, R.N. (1987) “Toward a universal law of generalization for psychological science” Science, 237(4820), 1317-1323. [PDF]

Tenenbaum, J.B. and Griffiths, T.L. (2001) "Generalization, similarity, and Bayesian inference.", Behavioral and Brain Sciences, 24, 629-641. [PDF]

[slides]
Nov. 16

9. Cognitive Forms of Learning - category and concept learning, hypothesis testing behavior, learning with rules or associations

Textbook reading: Gluck, Ch. 4 - Skill Memory

Sloman, S.A. (1996) “The empirical case for two systems of reasoning” Psychological Bulletin,119(1), 3-22. [PDF]

Maddox, W.T. and Ashby, F.G. (2004). Dissociating explicit and procedural-learning based systems of perceptual category learning. Behavioral Processes, 66, 309-332. [PDF]

Johansen, M.K. and Palmeri, T.J. (2002) "Are there representational shifts during category learning", Cognitive Psychology, 45, 482-553. [PDF]

[slides]
Nov. 23

10. Cognitive Forms of Learning (continued) - relational processing, causal learning, learning by analogy, prior knowledge and basic learning processes,

Erickson, M.A. and Kruschke, J.K. (1998) "Rules and Exemplars in Category Learning", Journal of Experimental Psychology: General, 127 (2), 107-140. [PDF]

Gopnik, A. et al. (2004). A Theory of Causal Learning in Children: Causal Maps and Bayes Networks. Psychological Review, 111(1), 3-32. [PDF]

Gentner, D. (2003) "Why We’re So Smart" in Gentner, D. and Goldin-Meadow, S. (Eds.) Language in mind: Advances in the study of language and thought (pp. 195-235) Cambridge, MA: MIT Press. [PDF]

[slides]
Nov. 30

11. Learning and Language - Are languages learned? Statistical learning, grammar learning, word learning

Textbook reading: Gluck, Ch. 13 - Language Learning

Two short empirical findings:
Saffran, J.R., Aslin, R.N. and Newport, E.L. (1996) “Statistical Learning by 8-month-old Infants” Science, 274, 1926-1928. [PDF]

Marcus, G.F., Vijayan, S., Rao, S.B., Vishton, P.M. (1996) “Rule Learning by Seven-Month-Old Infants” Science, 283, 77-80. [PDF]

Some theory:
Marcus, G.F. (1993) “Negative evidence in language acquisition” Cognition, 46, 53-85. [PDF]

Perfors, A., Tenenbaum, J.B., and Regier, T. (in press) “The learnability of abstract syntactic principles” Cognition [PDF]

[slides]
Dec. 7

12. Observational Learning - Learning from other people, copying, imitation, mirror neuron systems, observational learning, learning in communicative settings (e.g., pedagological reasoning)

Textbook reading: Gluck, Ch. 11 - Observational Learning

Bandura, A. (1965) “Vicarious Processes: A Case of No-Trial Learning” Advances in Experimental Social Psychology, 2, 1-55. [PDF]

Laland, K. (2004) “Social Learning Strategies” Learning & Behavior, 32(1), 4-14. [PDF]

Dinstein, I., Thomas, C., Behrmann, M., Heeger, D.J. (2008) “A Mirror Up to Nature” Current Biology, 18(1), R13-R18. [PDF]

Burke, C.J., Tobler, P.N., Baddeley, M., and Schultz, W. (2010) “Neural mechanisms of observational learning” Proceedings of National Academy of Science, 107(32), 14431-14436. (suggestion from candance) [PDF][Supp]

[slides]
Dec. 14

13. Final day/various topics - Developmental, aging, and effects of brain damage on learning and memory, educational implications of research in learning and memory

[final]
[slides]