Tangentially related book recommendation:

Jeff Hawkins, A Thousand Brains: A New Theory of Intelligence. It describes the importance of reference frames for intelligence, for locating limbs relative to the body, locating the body in space, locating objects in space (even abstract objects in abstract spaces), and a possible "implementation" of that in the columns of the neocortex (by, basically, if I understood correctly, positing certain neurons that encode location, eg via intersection of coordinates, and certain neurons that encode features, and then learning a connection between those).

A short and interesting read (with quite some implications for progress in AI).

Podcast with Sam Harris: https://samharris.org/podcasts/255-future-intelligence/

Jeff Hawkins at Wikipedia: https://en.wikipedia.org/wiki/Jeff_Hawkins

The tech developed by his AI company: https://en.wikipedia.org/wiki/Hierarchical_temporal_memory

Book at Amazon: https://www.amazon.com/Thousand-Brains-New-Theory-Intelligen...

ETA: Oh, quite some previous discussion on HN:

https://news.ycombinator.com/item?id=20326396

https://news.ycombinator.com/item?id=19311279

https://news.ycombinator.com/item?id=26240901

https://news.ycombinator.com/item?id=26794286

An interesting theory as to why spatiality helps so much is outlined in Jeff Hawkins' "1000 brain theory" (https://www.amazon.com/Thousand-Brains-New-Theory-Intelligen...). His team's research hints that all neocortical processing happens with reference frames at the core.

Hawkins also has a new book coming. His first book (as said in other comments) is a fantastic read.

https://www.amazon.com/Thousand-Brains-New-Theory-Intelligen...