Successful investors seeking returns, animals foraging for food, and pilots controlling aircraft all must take into account how their current decisions will impact their future standing. One challenge facing decision makers is that options that appear attractive in the short-term may not turn out best in the long run. In this paper, we explore human learning in a dynamic control task which places short- and long-term rewards in con?ict. Our goal in these studies was to evaluate how people?s mental representation of a task a?ects their ability to discover an optimal decision strategy. We ?nd that perceptual cues that readily align with the underlying state of the task environment help people overcome the impulsive appeal of short term rewards. Our experimental manipulations, predictions, and analyses are motivated by current work in reinforcement learning which details how learners discount future rewards, the importance of ?state? representations, and the role that exploration and exploitation play in e?ective learning.

Effective learning often involves actively querying the environment for information that disambiguates potential hypotheses. However, the space of observations available in any situation can vary greatly in potential "informativeness." In this report, we study participants' ability to gauge the information value of potential observations in a cognitive search task based on the children's game Battleship. Participants selected observations to disambiguate between a large number of potential game configurations subject to information-collection costs and penalties for making errors in a test phase. An "ideal-learner" model is developed to quantify the utility of possible observations in terms of the expected gain in points from knowing the outcome of that observation. The model was used as a tool for measuring search efficiency, and for classifying various types of information collection decisions. We find that participants are generally effective at maximizing gain relative to their current state of knowledge and the constraints of the task. In addition, search behavior shifts between an slower, but more efficient "exploitive" mode of local search and a faster, less efficient pattern of "exploration."

In engineering systems, noise is a curse, obscuring important signals and increasing the uncertainty associated with measurement. However, the negative effects of noise are not universal. In this paper, we examine how people learn sequential control strategies given different sources and amounts of feedback variability. In particular, we consider people's behavior in a task where short- and long-term rewards are placed in conflict (i.e., the best option in the short-term is worst in the long-term). Consistent with a model based on reinforcement learning principles (Gureckis & Love, 2009), we find that learners differentially weight information predictive of the current task state. In particular, when cues that signal state are noisy, we find that participants' ability to identify an optimal strategy is strongly impaired relative to equivalent amounts of variability that obscure the rewards/valuations of those states. In other situations, we find that noise and variability in reward signals may paradoxically improve performance by encouraging exploration. Our results demonstrate how experimentally-manipulated task variability can be used to test predictions about the mechanisms that learners engage in dynamic decision making tasks.

We evaluate two broad classes of cognitive mechanisms that might support the learning of sequential patterns. According to the ?rst, learning is based on the gradual accumulation of direct associations between events based on simple conditioning principles. The other view describes learning as the process of inducing the transformational structure that de?nes the material. Each of these learning mechanisms predict di?erences in the rate of acquisition for di?erently organized sequences. Across a set of empirical studies, we compare the predictions of each class of model with the behavior of human sub jects. We ?nd that learning mechanisms based on transformations of an internal state, such as recurrent network architectures (e.g., Elman, 1990), have di?culty accounting for the pattern of human results relative to a simpler (but more limited) learning mechanism based on learning direct associations. Our results suggest new constraints on the cognitive mechanisms supporting sequential learning behavior.

A novel study is presented that explores the effect that learning internally organized categories has on the ability to subsequently discriminate category members. The results demonstrate the classic categorical perception effect whereby discrimination of stimuli that belong to different categories is improved following training, while the ability to discriminate stimuli belonging to the same category is reduced. We further report a new within-category perceptual effect whereby category members that share the same category label but fall into different sub-clusters within that category are better discriminated than items that share the same category and cluster. The results show that learners are sensitive to multiple sources structure beyond simply the labels provided during supervised training. A computational model is presented to account for the results whereby multiple levels of encoding (i.e., at the item-, cluster-, and category- level) may simultaneously contribute to perception.

We examine the interdependence between individual and group behavior surrounding a somewhat arbitrary, real world decision: selecting a name for one's child. Using a historical database of the names given to children over the last century in the United States, we find that naming choices are influuenced by both the frequency of a name in the general population, and by its "momentum" in the recent past in the sense that names which are growing in popularity are preferentially chosen. This bias toward rising names is a recent phenomena: in the early part of the 20th century, increasing popularity of a name from one time period to the next was correlated with a decrease in future popularity. However, more recently this trend has reversed. We evaluate a number of formal models that detail how individual decision making strategies, played out in a large population of interacting agents, can explain these empirical observations. We argue that cognitive capacities for change detection, the encoding of frequency in memory, and biases towards novel or incongruous stimuli may interact with the behavior of other decision makers to determine the distribution and dynamics of cultural tokens such as names.

The ability to adjust the specificity of memory following encoding was investigated in three experiments. Participants studied random dot patterns under instructions emphasizing either categorization or recognition and were then asked to make either categorization or recognition judgements about novel transfer items that varied in similarity to the studied items. Overall, participants' generalization behavior depended primarily on the test instructions, independent of the study instructions, suggesting flexible changes in the specificity of memory retrieval at test. However, this ability varied as a function of both the number of training patterns and the amount of experience participants were given with each item during study. Model-based analysis were mostly consistent with an exemplar-based account whereby the specificity of encoding was strategically adjusted at test. However, an alternative model assuming imperfect storage and a changing response criterion also provided a close account of the data. Implications for the flexible nature of category representations are discussed.

Recent functional magnetic resonance imaging (fMRI) studies have found that distinct neural systems may mediate perceptual category learning under incidental and intentional learning conditions. The present study was designed to replicate and extend previous investigations of these effects using an event-related design. In addition, the design is aimed at decoupling the influence of stimulus-encoding processes from the contribution of implicit and explicit learning on the recruitment of alternative neural systems. Consistent with previous reports, following incidental learning in a dot-pattern classification task, participants show decreased neural activity in occipital visual cortex (extrastriate region V3, BA 19) in response to novel exemplars of a studied category compared to members of a foil category, but do not show this decreased neural activity following explicit learning. Crucially, however, our results show that this pattern can be modulated by aspects of the stimulus-encoding instructions provided at the time of study. In particular, when participants in an implicit learning condition were encouraged to evaluate the overall shape and configuration of the stimuli during study, we failed to find the pattern of brain activity that has been taken to be a signature of implicit learning, suggesting that activity in this area does not uniquely reflect implicit memory for perceptual categories.

A series of experiments are presented that explore the effect that learning internally structured categories has on the ability to visually discriminate category members. The results demonstrate the classic categorical perception (CP) effect whereby discrimination of stimuli that belong to different categories is improved following supervised category training, while the ability to discriminate stimuli belonging to the same category is reduced. However, we also find evidence of a within-category perceptual effect whereby category members that share the same category label but fall into different "sub-clusters" within the category are better discriminated than items that share the same category and cluster. The results show that learners are sensitive to multiple sources of structure when learning beyond simply the labels provided during supervised training including the distribution of items within categories. We compare these results to the predictions of a variety of models of CP including those based on error-driven learning processes and selective attention to diagnostic information. The results appear most consistent with an account whereby multiple levels of encoding (i.e., at the item-, cluster-, and category- level) may simultaneously, and competitively, contribute to perceptual processing.

This work explores the influence of motivation on choice behavior in a dynamic decision-making environment, where the payoffs from each choice depend on one?s recent choice history. Previous research reveals that participants in a regulatory fit exhibit increased levels of exploratory choice and flexible use of multiple strategies over the course of an experiment. The present study placed promotion and prevention-focused participants in a dynamic environment for which optimal performance is facilitated by systematic exploration of the decision space. These participants either gained or lost points with each choice. Our experiment revealed that participants in a regulatory fit were more likely to engage in systematic exploration of the task environment than participants in a regulatory mismatch and performed more optimally as a result. Implications for contemporary models of human reinforcement-learning are discussed.

Numerous proposal have been put forward concerning the nature of human category representations, ranging from rules to exemplars to prototypes. However, it is unlikely that a single, ?xed form of representation is suf?cient to account for the ?exibility of human categories. In this chapter, we describe an alternative to these ?xed-representation accounts based on the principle of adaptive clustering. The speci?c model we consider, SUSTAIN, represents categories in terms of feature bundles called clusters which are adaptively recruited in response to task demands. In some cases, SUSTAIN acts like an exemplar model, storing each category instance, while in others it appears more like a prototype model extracting only the central tendency of a number of items. In addition, selective attention in the model allows it to mimic many of the behaviors associated with rule-based systems. We review a variety of evidence in support of the clustering principle, including studies of the relationship between categorization and recognition memory, changes in unsupervised category learning abilities across development, and the in?uence of category learning on perceptual discrimination. In each case, we show how the nature of human category representations are best accounted for using an adaptive clustering scheme. SUSTAIN is just one example of a system that casts category learning in terms of adaptive clustering and future directions for the approach are discussed.

Successful investors seeking returns, animals foraging for food, and pilots controlling aircraft all must take into account how their current decisions will impact their future standing. One challenge facing decision makers is that options that appear attractive in the short-term may not turn out best in the long run. In this paper, we explore human learning in a dynamic control task which places short- and long-term rewards in con?ict. Our goal in these studies was to evaluate how people?s mental representation of a task a?ects their ability to discover an optimal decision strategy. We ?nd that perceptual cues that readily align with the underlying state of the task environment help people overcome the impulsive appeal of short term rewards. Our experimental manipulations, predictions, and analyses are motivated by current work in reinforcement learning which details how learners discount future rewards, the importance of ?state? representations, and the role that exploration and exploitation play in e?ective learning.

Effective learning often involves actively querying the environment for information that disambiguates potential hypotheses. However, the space of observations available in any situation can vary greatly in potential "informativeness." In this report, we study participants' ability to gauge the information value of potential observations in a cognitive search task based on the children's game Battleship. Participants selected observations to disambiguate between a large number of potential game configurations subject to information-collection costs and penalties for making errors in a test phase. An "ideal-learner" model is developed to quantify the utility of possible observations in terms of the expected gain in points from knowing the outcome of that observation. The model was used as a tool for measuring search efficiency, and for classifying various types of information collection decisions. We find that participants are generally effective at maximizing gain relative to their current state of knowledge and the constraints of the task. In addition, search behavior shifts between an slower, but more efficient "exploitive" mode of local search and a faster, less efficient pattern of "exploration."

In engineering systems, noise is a curse, obscuring important signals and increasing the uncertainty associated with measurement. However, the negative effects of noise are not universal. In this paper, we examine how people learn sequential control strategies given different sources and amounts of feedback variability. In particular, we consider people's behavior in a task where short- and long-term rewards are placed in conflict (i.e., the best option in the short-term is worst in the long-term). Consistent with a model based on reinforcement learning principles (Gureckis & Love, 2009), we find that learners differentially weight information predictive of the current task state. In particular, when cues that signal state are noisy, we find that participants' ability to identify an optimal strategy is strongly impaired relative to equivalent amounts of variability that obscure the rewards/valuations of those states. In other situations, we find that noise and variability in reward signals may paradoxically improve performance by encouraging exploration. Our results demonstrate how experimentally-manipulated task variability can be used to test predictions about the mechanisms that learners engage in dynamic decision making tasks.

We evaluate two broad classes of cognitive mechanisms that might support the learning of sequential patterns. According to the ?rst, learning is based on the gradual accumulation of direct associations between events based on simple conditioning principles. The other view describes learning as the process of inducing the transformational structure that de?nes the material. Each of these learning mechanisms predict di?erences in the rate of acquisition for di?erently organized sequences. Across a set of empirical studies, we compare the predictions of each class of model with the behavior of human sub jects. We ?nd that learning mechanisms based on transformations of an internal state, such as recurrent network architectures (e.g., Elman, 1990), have di?culty accounting for the pattern of human results relative to a simpler (but more limited) learning mechanism based on learning direct associations. Our results suggest new constraints on the cognitive mechanisms supporting sequential learning behavior.

We examine the interdependence between individual and group behavior surrounding a somewhat arbitrary, real world decision: selecting a name for one's child. Using a historical database of the names given to children over the last century in the United States, we find that naming choices are influuenced by both the frequency of a name in the general population, and by its "momentum" in the recent past in the sense that names which are growing in popularity are preferentially chosen. This bias toward rising names is a recent phenomena: in the early part of the 20th century, increasing popularity of a name from one time period to the next was correlated with a decrease in future popularity. However, more recently this trend has reversed. We evaluate a number of formal models that detail how individual decision making strategies, played out in a large population of interacting agents, can explain these empirical observations. We argue that cognitive capacities for change detection, the encoding of frequency in memory, and biases towards novel or incongruous stimuli may interact with the behavior of other decision makers to determine the distribution and dynamics of cultural tokens such as names.

Research on dynamic decision-making tasks, in which the payoffs associated with each choice vary with participants recent choice history, finds that humans have difficulty making long-term optimal choices in the presence of attractive immediate rewards. However, a number of recent studies have shown that simple cues providing information about the underlying state of the task environment may facilitate optimal responding. This study examines the mechanism by which this state knowledge influences choice behavior. We examine the possibility that participants use state information in conjunction with changing payoffs to extrapolate payoffs in future states. We find support for this hypothesis in a study where generalizations based on this state information work to the benefit or detriment of task performance, depending on the task's payoff structure.

see PDF

see PDF

This work explores the influence of motivation on choice behavior in a dynamic decision-making environment, where the payoff from each choice depend on one's recent choice history. Previous research reveals increased levels of exploratory choice among participants in a regulatory fit. The present study placed promotion and prevention-focused participants in a dynamic environment for which optimal performance requires that participants sustain a single choice strategy in the face of temporary payoff decreases. These participants either gained or lost points with each choice. Our behavioral results and a model-based analysis, using an average-reward reinforcement learning framework, revealed differential levels of reactivity to local changes in payoffs - specifically, participants in a regulatory fit were less reactive to local perturbations in payoffs than participants in a regulatory mismatch and performed more optimally as a result.

A novel study is presented that explores the effect that learning internally organized categories has on the ability to subsequently discriminate category members. The results demonstrate the classic categorical perception effect whereby discrimination of stimuli that belong to different categories is improved following training, while the ability to discriminate stimuli belonging to the same category is reduced. We further report a new within-category perceptual effect whereby category members that share the same category label but fall into different sub-clusters within that category are better discriminated than items that share the same category and cluster. The results show that learners are sensitive to multiple sources structure beyond simply the labels provided during supervised training. A computational model is presented to account for the results whereby multiple levels of encoding (i.e., at the item-, cluster-, and category- level) may simultaneously contribute to perception.

We live in a world consisting of concrete experiences, yet we appear to form abstractions that transcend the details of our experiences. In this contribution, we argue that the abstract nature of our thought is overstated and that our representations are inherently bound to the examples we experience during learning. We present three lines of related research to support this general point. The ?rst line of research suggests that there are no separate learning systems for acquiring mental rules and storing exceptions to these rules. Instead, both items types share a common representational substrate that is grounded in experienced training examples. The second line of research suggests that representations of abstract concepts, such as same and different that can range over an unbounded set of stimulus properties, are rooted in experienced examples coupled with analogical processes. Finally, we consider how people perform in dynamic decision tasks in which short? and long?term rewards are in opposition. Rather than invoking explicit reasoning processes and planning, people's performance is best explained by reinforcement learning procedures that update estimates of action values in a reactive, trial?by?trial fashion. All three lines of research implicate mechanisms of thought that are capable of broad generalization, yet inherently local in terms of the procedures used for updating mental representations and planning future actions. We end by considering the bene?ts of designing systems that operate according to these principles.

We asked 169 participants to spontaneously categorize nine sets of items. A category structure was assumed to be more intuitive if a large number of participants consistently produced the same classification. Our results provide a rich empirical framework for examining models of unsupervised categorization?and illustrate the corresponding profound modeling challenge. We provide a preliminary examination comparing two models of unsupervised categorization: SUSTAIN (Love, Medin, & Gureckis, 2004) and the simplicity model (Pothos & Chater, 2002).

Mental localization efforts tend to stress the where more than the what. We argue that the proper targets for localization are well-specified cognitive models. We make this case by relating an existing cognitive model of category learning to a learning circuit involving the hippocampus, perirhinal, and prefrontal cortices. Results from groups varying in function along this circuit (e.g., infants, amnesics, and older adults) are successfully simulated by reducing the model?s ability to form new clusters in response to surprising events, such as an error in supervised learning or an unfamiliar stimulus in unsupervised learning. Clusters in the model are akin to conjunctive codes that are rooted in an episodic experience (the surprising event) yet can develop to resemble abstract codes as they are updated by subsequent experiences. Thus, the model holds that the line separating episodic and semantic information can become blurred. Dissociations (categorization vs. recognition) are explained in terms of cluster recruitment demands.

In recent years, statistical learning (SL) has emerged as a compelling alternative to nativist theories of language acquisition (i.e., Chomsky, 1980). However, in many ways the framework of statistical learning echoes aspects of classic behaviorism by stressing the role of associative learning processes and the environment in shaping behavior. How far backwards has the needle swung? In this paper, we show how a subset of behaviors studied under the rubric of SL are in fact entirely consistent with a simple form of conditioned priming inspired by models from the behaviorist tradition (i.e. a variant of Rescola-Wagner which learns associative relationships through time).

Is cognition an exclusive property of the individual or can groups have a mind of their own? We explore this question from the perspective of complex adaptive systems. One of the principle insights from this line of work is that rules that govern behavior at one level of analysis (the individual) can cause qualitatively different behavior at higher levels (the group). We review a number of behavioral studies from our lab that demonstrate how groups of people interacting in real-time can self-organize into adaptive, problem-solving group structures. A number of principles are derived concerning the critical features of such ?distributed? information processing systems. We suggest that while cognitive science has traditionally focused on the individual, cognitive processes may manifest at many levels including the emergent group-level behavior that results from the interaction of multiple agents and their environment.

Mental localization e?orts tend to stress the where more than the what. We argue that the proper targets for localization are well-speci?ed cognitive models. We make this case by relating an existing cognitive model of category learning to a learning circuit involving the hippocampus, perirhinal, and prefrontal cortex. Results from groups varying in function along this circuit (e.g., infants, amnesics, older adults) are successfully simulated by reducing the model?s ability to form new clusters in response to surprising events, such as an error in supervised learning or an unfamiliar stimulus in unsupervised learning. Reported task dissociations (e.g., categorization vs. recognition) are explained in terms of cluster recruitment demands.

NO PDF

In this report, a model of human sequence learning is developed called the linear associative shift register (LASR). LASR uses a simple error-driven associative learning rule to incrementally acquire information about the structure of event sequences. In contrast to recent modeling approaches, LASR describes learning as a simple and limited process. We argue that this simplicity is a virtue in that the complexity of the model is better matched to the demonstrated complexity of human processing. The model is applied in a variety of situations including implicit learning via the serial reaction time (SRT) task and statistical word learning. The results of these simulations highlight commonalities between different tasks and learning modalities which suggest similar underlying learning mechanisms. riety of implicit learning situations including the serial reaction time (SRT) task and statistical word learning paradigms. LASR provides a similar account of the type of processing which underlies performance in both kinds of tasks, suggesting that they may rely on similar underlying mechanisms.

Human behavior is intrinsically linked to the temporal and sequential characteristics of the environment. In this report, a model of human sequence learning is developed called the linear associative shift register (LASR). LASR uses a simple error-driven associative learning rule to incrementally acquire information about the structure of event sequences. The model is applied in a variety of situations including implicit learning via the serial reaction time (SRT) task and statistical word learning. The results of these simulations highlight commonalities between di?erent tasks and learning modalities which suggests similar underlying learning mechanisms. Two novel experiments are reported which test unique predictions of the model and which di?erentiate between competing theories concerning the mechanisms underlying implicit sequence learning.

SUSTAIN (Supervised and Unsupervised STratified Adaptive Incremental Network) is a model of how humans learn categories from examples. SUSTAIN initially assumes a simple category structure. If simple solutions prove inadequate and SUSTAIN is confronted with a surprising event (e.g., it is told that a bat is a mammal instead of a bird), SUSTAIN recruits an additional cluster to represent the surprising event. Newly recruited clusters are available to explain future events and can themselves evolve into prototypes?attractors?rules. SUSTAIN?s discovery of category substructure is affected not only by the structure of the world but by the nature of the learning task and the learner?s goals. SUSTAIN successfully extends category learning models to studies of inference learning, unsupervised learning, category construction, and contexts in which identification learning is faster than classification learning.

Computational models of infant categorization often fail to elaborate the transitional mechanisms that allow infants to achieve adult performance. In this paper, we apply a successful connectionist model of adult category learning to developmental data. The Supervised and Unsupervised Strati?ed Adaptive Increment Network (SUSTAIN) model is able to account for the emergence of infants? sensitivity to correlated attributes (e.g., has wings and can ?y). SUSTAIN o?ers two complimentary explanations of the developmental trend. One explanation centers on memory storage limitations, whereas the other focuses on limitations in perceptual systems. Both explanations parallel published ?ndings concerning the cognitive and sensory limitations of infants. SUSTAIN?s simulations suggest that conceptual development follows a continuous and smooth tra jectory, despite qualitative changes in behavior, and that the mechanisms that underlie infant and adult categorization may not di?er signi?cantly.

no abstract

SUSTAIN (Supervised and Unsupervised STratified Adaptive Incremental Network) is a network model of human category learning. SUSTAIN initially assumes a simple category structure. If simple solutions prove inadequate and SUSTAIN is confronted with a surprising event (e.g., it is told that a bat is a mammal instead of a bird), SUSTAIN recruits an additional cluster to represent the suprising event. Newly recruited clusters are available to explain future events and can themselves evolve into prototypes/attractors/rules. SUSTAIN has expanded the scope of ?ndings that models of human category learning can address. This paper extends SUSTAIN so that it can be used to account for both supervised and unsupervised learning data through a common mechanism. A modi?ed recruitment mechanism is introduced that creates new conceptual clusters in response to surprising events during learning. The new formulation of the model is called uSUSTAIN for ?uni?ed SUSTAIN.? The implications of using a uni?ed recruitment method for both supervised and unsupervised learning is discussed.

SUSTAIN (Supervised and Unsupervised STrati?ed Adaptive Incremental Network) is a network model of human category learning. SUSTAIN initially assumes a simple category structure. If simple solutions prove inadequate and SUSTAIN is confronted with a surprising event (e.g., it is told that a bat is a mammal instead of a bird), SUSTAIN recruits an additional cluster to represent the surprising event. Newly recruited clusters are available to explain future events and can themselves evolve into prototypes/attractors/rules. SUSTAIN has expanded the scope of ?ndings that models of human category learning can address. This paper extends SUSTAIN to account for both supervised and unsupervised learning data through a common mechanism. The modi?ed model, uSUSTAIN (uni?ed SUSTAIN), is successfully applied to human learning data drawn from Love (2002) that compares unsu- pervised and supervised learning performance.

SUSTAIN (Supervised and Unsupervised STrati?ed Adaptive Incremental Network) is a network model of human category learning. This paper extends SUSTAIN so that it can be used to model unsupervised learning data. A modi?ed recruitment mechanism is introduced that creates new conceptual clusters in response to surprising events during learning. Two seemingly contradictory unsupervised learning data sets are modeled using this new recruitment method. In addition, the feasibility of using a uni?ed recruitment method for both supervised and unsupervised learning is discussed.

How do people learn and organize examples in the absence of a teacher? This paper explores this question through a examination of human data and computational modeling results. The SUSTAIN (Supervised and Unsupervised STrati?ed Incremental Network) model successfully ?ts human learning data drawn from two published studies. The first study examines how correlations between features can facilitate unsupervised learning. The second set of studies examines the role that similarity and attention play in unsupervised category construction (i.e., sorting) tasks. Importantly, SUSTAIN suggests two novel behavioral predictions that are confirmed.