Structural correspondence

From A conversation about the brain
Jump to: navigation, search
On the left, a plan view of the scene used by Zhu et al[1] The blue stars and arrows show the locations and views that were rewarded. On the right is a plan view of a scene and, in brown, the locations of the camera which in this case has a 360deg field of view.
On the left, a 'tSNE plot' of the representation generated through reinforcement learning by Zhu et al[1]. a tSNE plot preserves distances between pairs of high dimensional points when plotting them in 2D. On the right, a 'tSNE plot' of the representation suggested by Glennerster et al[2]. The colour code corresponds to that used in the map above. There is a close structural correspondence.
Shea[3] fig 5.6 illustrates the distance between two dimensional vectors defined by the firing rates in two neurons. This is problematic in a number of ways, especially when it refers to faces and inanimate faces. A tSNE plot of high dimensional vectors describing the responses of a large number of neurons to the stimuli S1 - S4 would be more relevant.

In Chapter 5, Shea[3]. talks about structural correspondence between elements of a representation and the thing that is represented. He uses place cells as an example. He also discusses Churchland's ideas[4] on using similarity in a high dimensional space to indicate similarity between things that are represented. In a recent paper[5], we explore the structural correspondence between navigated space and a high dimensional representation generated by an agent learning to navigate in that space.

The key take-home message from the paper[5] is that the structure of the representation is dominated by task (goal image), then camera orientation and much less so by information about spatial location. The other representation we explore (right hand side) has a much more direct structural correspondence to space in the real world (brown colours in the tSNE plot correspond to brown colours of camera locations in the scene). Poster version of Muryy et al: File:MuryyPoster.pdf.

Back to notes on Shea (2018).

References

  1. ↑ 1.0 1.1 Zhu, Y., Mottaghi, R., Kolve, E., Lim, J. J., Gupta, A., Fei-Fei, L., & Farhadi, A. (2017). Target-driven visual navigation in indoor scenes using deep reinforcement learning. In 2017 IEEE international conference on robotics and automation (ICRA) (pp. 3357-3364).
  2. ↑ Glennerster, A., Hansard, M. E., & Fitzgibbon, A. W. (2001). Fixation could simplify, not complicate, the interpretation of retinal flow. Vision Research, 41(6), 815-834
  3. ↑ 3.0 3.1 Shea, N. (2018). Representation in cognitive science. Oxford University Press.
  4. ↑ Churchland, P. M. (2012). Plato's camera: How the physical brain captures a landscape of abstract universals. MIT press.
  5. ↑ 5.0 5.1 Muryy, A., Siddharth, N., Nardelli, N., Glennerster, A., & Torr, P. H. (2019). Lessons from reinforcement learning for biological representations of space. arXiv preprint arXiv:1912.06615. (Vision Research, in press)
Retrieved from "http://wiki.glennersterlab.com/index.php?title=Structural_correspondence&oldid=1906"