External and internal modes of sensory processing


Veith Weilnhammer, MD
Helen Wills Neuroscience Institute, UC Berkeley

2 Questions



  • How are unambiguous conscious experiences generated from ambiguous sensory information?

  • What causes psychotic experiences?

Feedforward processing

  • Individual frames are processed independently

Feedforward processing

  • Individual frames are processed independently

Natural environments

  • Individual frames are strongly autocorrelated

Natural environments

  • One frame predicts the next

Feedback

  • Predictive processing compresses the input and saves energy

Feedback

  • Predictive processing creates unambiguous experiences from ambiguous inputs

Circularity

  • How does perception balance stability with sensitivity to change?

Graded Ambiguity

Serial Dependencies

  • Stimulus- and history-congruence are autocorrelated.

Serial Dependencies

  • Internally-biased processing suppresses stimulus sensitivity.

Perceptual Modes

  • Are between-mode fluctuations a general phenomenon in perceptual decision-making?

Confidence Database

  • 4500 humans, 22 million choices

Confidence Database

  • Quadratic relationship to RTs and confidence

IBL Database

  • 165 mice, 1.5 million choices

Bimodal inference

  • Leaky accumulation of evidence + fluctuating weights
  • Alternatives: GLM-HMM (Ashwood et al. Nat. Neuroscience 2022), Psytrack (Roy et al. Neuron 2024), Semi-HMM (Bruins et al. BioRxiv 2023)

Are changes in bimodal inference related to psychotic experiences?

Summary



  • Perceptual decision-making fluctuates between external and internal modes of sensory processing
  • NMDAR antagonism and schizphrenia increase external mode processing
  • Psychotic experience may reflects transient episodes of external mode processing

What does it all mean?



  • Adaptive behavior requires inferring the state of the world from an ongoing stream of ambiguous sensory information.

  • Perception depends on two integrated processes: the rapid feed-forward encoding of sensory infomration, and the slower feed-back reconstruction of incoming signals.

  • Errors may result from two sources: errors of encoding (limited external bandwidth) + changes in the statistics of the environment (failure to update internal models).

  • What should be updated in response to errors?

Thanks a lot for your attention!

References


Weilnhammer, Stuke, Hesselmann, Sterzer, Schmack. A Predictive Coding Account of Bistable Perception. PLOS Computational Biology 2017.

Weilnhammer, Lukas, Eckert, Stuke, Heinz, Sterzer. Psychotic Experiences in Schizophrenia and Sensitivity to Sensory Evidence. Schizophrenia Bulletin 2020.

Weilnhammer, Fritsch, Chikermane, Eckert, Kathak, Stuke, Sterzer. An Active Role of Inferior Frontal Cortex in Conscious Experience. Current Biology 2021.

Weilnhammer, Chikermane, Sterzer. Bistable perception alternates between internal and external modes of sensory processing. iScience 2021.

Weilnhammer, Stuke, Standvoß, Sterzer. Sensory processing in humans and mice fluctuates between external and internal modes. PLOS Biology 2023.