Cookies on this website
We use cookies to ensure that we give you the best experience on our website. If you click 'Continue' we'll assume that you are happy to receive all cookies and you won't see this message again. Click 'Find out more' for information on how to change your cookie settings.
  • The neural mechanisms of learning from competitors.

    24 October 2018

    Learning from competitors poses a challenge for existing theories of reward-based learning, which assume that rewarded actions are more likely to be executed in the future. Such a learning mechanism would disadvantage a player in a competitive situation because, since the competitor's loss is the player's gain, reward might become associated with an action the player should themselves avoid. Using fMRI, we investigated the neural activity of humans competing with a computer in a foraging task. We observed neural activity that represented the variables required for learning from competitors: the actions of the competitor (in the player's motor and premotor cortex) and the reward prediction error arising from the competitor's feedback. In particular, regions positively correlated with the unexpected loss of the competitor (which was beneficial to the player) included the striatum and those regions previously implicated in response inhibition. Our results suggest that learning in such contexts may involve the competitor's unexpected losses activating regions of the player's brain that subserve response inhibition, as the player learns to avoid the actions that produced them.

  • Optimal decisions: From neural spikes, through stochastic differential equations, to behavior

    24 October 2018

    There is increasing evidence from in vivo recordings in monkeys trained to respond to stimuli by making left- or rightward eye movements, that firing rates in certain groups of neurons in oculo-motor areas mimic drift-diffusion processes, rising to a (fixed) threshold prior to movement initiation. This supplements earlier observations of psychologists, that human reaction-time and error-rate data can be fitted by random walk and diffusion models, and has renewed interest in optimal decision-making ideas from information theory and statistical decision theory as a clue to neural mechanisms. We review results from decision theory and stochastic ordinary differential equations, and show how they may be extended and applied to derive explicit parameter dependencies in optimal performance that may be tested on human and animal subjects. We then briefly describe a biophysically-based model of a pool of neurons in locus coeruleus, a brainstem nucleus implicated in widespread norepinephrine release. This neurotransmitter can effect transient gain changes in cortical circuits of the type that the abstract drift-diffusion analysis requires. We also describe how optimal gain schedules can be computed in the presence of time-varying noisy signals. We argue that a rational account of how neural spikes give rise to simple behaviors is beginning to emerge. Copyright © 2005 The Institute of Electronics, Information and Communication Engineers.

  • A comparison of bounded diffusion models for choice in time controlled tasks.

    24 October 2018

    The Wiener diffusion model (WDM) for 2-alternative tasks assumes that sensory information is integrated over time. Recent neurophysiological studies have found neural correlates of this integration process in certain neuronal populations. This paper analyses the properties of the WDM with two different boundary conditions in decision making tasks in which the time of response is indicated by a cue. A dual reflecting boundary mechanism is proposed and its performance is compared with a well-established absorbing boundary in the cases of the WDM, the WDM with extensions, and the WDM with prior probability. The two types of boundary influence the dynamics of the model and introduce differential weighting of evidence. Comparisons with Ornstein-Uhlenbeck models are also done, and it is shown that the WDM with both types of boundaries achieves similar performance and produce similar fits to existing behavioural data. Further studies are proposed to distinguish which boundary mechanism is more consistent with experimental data.

  • Bounded Ornstein-Uhlenbeck models for two-choice time controlled tasks

    24 October 2018

    The Ornstein-Uhlenbeck (O-U) model has been successfully applied to describe the response accuracy and response time in 2-alternative choice tasks. This paper analyses properties and performance of variants of the O-U model with absorbing and reflecting boundary conditions that limit the range of possible values of the integration variable. The paper focuses on choice tasks with pre-determined response time. The type of boundary and the growth/decay parameter of the O-U model jointly determine how the choice is influenced by the sensory input presented at different times throughout the trial. It is shown that the O-U models with two types of boundary are closely related and can achieve the same performance under certain parameter values. The value of the growth/decay parameter that maximizes the accuracy of the model has been identified. It is shown that when the boundaries are introduced, the O-U model may achieve higher accuracy than the diffusion model. This suggests that given the limited range of the firing rates of integrator neurons, the neural decision circuits could achieve higher accuracy employing leaky rather than linear integration in certain tasks. We also propose experiments that could distinguish between different models of choice in tasks with pre-determined response time. © 2010 Elsevier Inc.

  • An anti-Hebbian model of familiarity discrimination in the perirhinal cortex

    24 October 2018

    Much evidence indicates that the perirhinal cortex of the temporal lobe is involved in the familiarity discrimination aspect of recognition memory. All previously published models of familiarity discrimination in the perirhinal cortex are based on Hebbian learning. Here we present a biologically plausible model based on anti-Hebbian learning. When the responses of neurons providing input to the familiarity discrimination network are correlated (as is indicated by experimental data), then the anti-Hebbian model achieves a much higher capacity (up to thousands of times) and hence a crucially higher efficiency than models based on Hebbian learning. © 2002 Elsevier Science B.V. All rights reserved.

  • Model of co-operation between recency, familiarity and novelty neurons in the perirhinal cortex

    24 October 2018

    Much evidence indicates that discrimination of the familiarity of visual stimuli is dependent on the perirhinal cortex of the temporal lobe. Within the monkey's perirhinal cortex, ∼25% of neurons respond strongly to the sight of novel objects but respond only weakly or briefly when these objects are seen again. These neurons can be divided into three populations based on their patterns of responsiveness. Specific temporal dependencies exist among the activities of the three populations of neurons, suggesting the existence of specific connections between them. This report concerns computer modelling that indicates how such connections may be used to increase reliability in the determination of whether or not a stimulus is being seen for the first time. © 2001 Elsevier Science B.V. All rights reserved.

  • Effective connectivity of the subthalamic nucleus-globus pallidus network during Parkinsonian oscillations

    24 October 2018

    In Parkinsonism, subthalamic nucleus (STN) neurons and two types of external globus pallidus (GP) neuron inappropriately synchronise their firing in time with slow (∼1 Hz) or beta (13-30 Hz) oscillations in cortex. We recorded the activities of STN, Type-I GP (GP-TI) and Type-A GP (GP-TA) neurons in anaesthetised Parkinsonian rats during such oscillations to constrain a series of computational models that systematically explored the effective connections and physiological parameters underlying neuronal rhythmic firing and phase preferences in vivo. The best candidate model, identified with a genetic algorithm optimising accuracy/complexity measures, faithfully reproduced experimental data and predicted that the effective connections of GP-TI and GP-TA neurons are quantitatively different. Estimated inhibitory connections from striatum were much stronger to GP-TI neurons than to GP-TA neurons, whereas excitatory connections from thalamus were much stronger to GP-TA and STN neurons than to GP-TI neurons. Reciprocal connections between GP-TI and STN neurons were matched in weight, but those between GP-TA and STN neurons were not; only GP-TI neurons sent substantial connections back to STN. Different connection weights between and within the two types of GP neuron were also evident. Adding to connection differences, GP-TA and GP-TI neurons were predicted to have disparate intrinsic physiological properties, reflected in distinct autonomous firing rates. Our results elucidate potential substrates of GP functional dichotomy, and emphasise that rhythmic inputs from striatum, thalamus and cortex are important for setting activity in the STN-GP network during Parkinsonian beta oscillations, suggesting they arise from interactions between most nodes of basal ganglia-thalamocortical circuits. © 2013 The Authors. The Journal of Physiology published by John Wiley & Sons Ltd on behalf of The Physiological Society.

  • Robust versus optimal strategies for two-alternative forced choice tasks.

    24 October 2018

    It has been proposed that animals and humans might choose a speed-accuracy tradeoff that maximizes reward rate. For this utility function the simple drift-diffusion model of two-alternative forced-choice tasks predicts a parameter-free optimal performance curve that relates normalized decision times to error rates under varying task conditions. However, behavioral data indicate that only ≈ 30% of subjects achieve optimality, and here we investigate the possibility that, in allowing for uncertainties, subjects might exercise robust strategies instead of optimal ones. We consider two strategies in which robustness is achieved by relinquishing performance: maximin and robust-satisficing. The former supposes maximization of guaranteed performance under a presumed level of uncertainty; the latter assumes that subjects require a critical performance level and maximize the level of uncertainty under which it can be guaranteed. These strategies respectively yield performance curves parameterized by presumed uncertainty level and required performance. Maximin performance curves for uncertainties in response-to-stimulus interval match data for the lower-scoring 70% of subjects well, and are more likely to explain it than robust-satisficing or alternative optimal performance curves that emphasize accuracy. For uncertainties in signal-to-noise ratio, neither maximin nor robust-satisficing performance curves adequately describe the data. We discuss implications for decisions under uncertainties, and suggest further behavioral assays.

  • Extending a biologically inspired model of choice: Multi-alternatives, nonlinearity, and value-based multidimensional choice

    24 October 2018

    © Cambridge University Press 2012. The Leaky Competing Accumulator (LCA) is a biologically inspired model of choice. It describes the processes of leaky accumulation and competition observed in neuronal populations during choice tasks and it accounts for reaction time distributions observed in psychophysical experiments. This chapter discusses recent analyses and extensions of the LCA model. First, it reviews the dynamics and it examines the conditions that make the model achieve optimal performance. Second, it shows that nonlinearities of the type present in biological neurons improve performance when the number of choice-alternatives increases. Third, the model is extended to value-based choice, where it is shown that nonlinearities in the value function, explain risk-aversion in risky-choice and preference reversals in choice between alternatives characterised across multiple dimensions. Introduction: Making choices on the basis of visual perceptions is an ubiquitous and central element of human and animal life, which has been studied extensively in experimental psychology. Within the last half century, mathematical models of choice reaction times have been proposed which assume that, during the choice process, noisy evidence supporting the alternatives is accumulated (Laming, 1968; Ratcliff, 1978; Stone, 1960; Vickers, 1970). Within the last decade, data from neurobiological experiments have shed further light on the neural bases of such choice. For example, it has been reported that while a monkey decides which of two stimuli is presented, certain neuronal populations gradually increase their firing rate, thereby accumulating evidence supporting the alternatives (Gold and Shadlen, 2002; Schall, 2001; Shadlen and Newsome, 2001). Recently, a series of neurocomputational models have offered an explanation of the neural mechanism underlying both, psychological measures like reaction times and neurophysiological data of choice. One such model, is the Leaky Competing Accumulator (LCA; Usher and McClelland, 2001), which is sufficiently simple to allow a detailed mathematical analysis. Furthermore, as we will discuss, this model can, for certain values of its parameters, approximate the same computations carried out by a series of mathematical models of choice (Busemeyer and Townsend, 1993; Ratcliff, 1978; Shadlen and Newsome, 2001; Vickers, 1970; Wang, 2002).

  • On optimal decision making in brains and social insect colonies

    24 October 2018

    © Cambridge University Press 2012. The problem of how to compromise between speed and accuracy in decision making faces organisms at many levels of biological complexity. Striking parallels are evident between decision making in primate brains and collective decision making in social insect colonies: in both systems separate populations accumulate evidence for alternative choices, when one population reaches a threshold a decision is made for the corresponding alternative, and this threshold may be varied to compromise between the speed and accuracy of decision making. In primate decision making simple models of these processes have been shown, under certain parameterisations, to implement the statistically optimal procedure that minimises decision time for any given error rate. In this chapter, we adapt these same analysis techniques and apply them to new models of collective decision making in social insect colonies. We show that social insect colonies may also be able to achieve statistically optimal collective decision making in a very similar way to primate brains, via direct competition between evidence-accumulating populations. This optimality result makes testable predictions for how collective decision making in social insects should be organised. Our approach also represents the first attempt to identify a common theoretical framework for the study of decision making in diverse biological systems. Animals constantly make decisions. Habitat selection, mate selection, and foraging require investigation of, and choice between, alternatives that may determine an animal's reproductive success. For example, many animals invest considerable time and energy in finding a safe home (Franks et al., 2002; Hansell, 1984; Hazlett, 1981; Seeley, 1982). Similarly an animal may frequently have to deal with ambiguous sensory information in deciding whether a predator is present or not (Trimmer et al., 2008).