If there is a candidate for a true, 3D representation in the brain, then V1 is probably the best. Receptive fields have an orderly structure in two dimensions of visual direction and many neurons also have a receptive field in the depth dimension. Hence, it is legitimate to describe V1 as encoding information about the 3D structure of the scene in a coordinate frame centred on the fixated object with cyclopean visual direction and disparity making up the three axes.
There is nowhere else in the brain that such a crystalline structure has been found instantiating an alternative coordinate frame. Although neurons have been identified with ‘world-centred’  or ‘head-centred’ receptive fields, these tend not to ordered in the same way as V1 cells, side by side in the cortex according to the receptive field location of the cell. Indeed, as discussed before, there is an important debate about whether these neurons really form part of a true 3D representation in the sense that neurons in V1 do. Clear evidence that ‘world-centred’ and ‘head-centred’ visual representations resulted from the wholesale 3D rotation and translation of the receptive fields in V1 (i.e. an appropriate coordinate transformation) would be startling and a knockout blow to the ideas on 3D vision presented here.
Back to Wrong directions.
- O'Keefe, J., & Burgess, N. (1996). Geometric determinants of the place fields of hippocampal neurons. Nature, 381, 425-428.
- Duhamel, J. R., Colby, C. L., & Goldberg, M. E. (1998). Ventral intraparietal area of the macaque: congruent visual and somatic response properties. Journal of Neurophysiology, 79(1), 126-136.