After taking a look at the paper, take a look at this post by the Google deep learning team. Take a look especially at the images.
I think most if not all of the author's arguments that the brain is not a computer, can also be applied to Google's deep learning (which obviously runs on computers).
I think the author is very naive about how broad a computer is and what a computer can do. Deep learning is obviously running on a computer, but like the author states for the brain, any single image or fact is probably not encoded in a single place.
In addition, we have probabilistic data structures, for example a bloom filter. Would a computer running a bloom filter to recognize data it had seen before not be a computer because it did not store an complete representation of any one item?
What we are seeing now, is that as computers go beyond just storing and retrieving data, to actually doing stuff like speech/image/pattern recognition, the representation is becoming more and more like the brain representation.
In addition, the author ignores that the brain did indeed develop a way encode information exactly. If the author would like more information about this, I suggest they look up "drawing" and "writing" in wikipedia. With drawing and especially with writing, the brain developed a way to encode information exactly that could later be retrieved. In addition, it could be retrieved by other brains (if they know the language).
Also, the author proposes straw man arguments for the other side. I am somewhat familiar with both computers and brains ( I trained in Neurosurgery, but work now as a programmer), and I never met a professional who claimed that memories are stored in individual neurons! I was always taught that memories are encoded in the connections between neurons - which is very similar to how neural networks encode information ( as the strength of connections between elements of the neural network ).
It is also interesting that he quotes Kandel. When I actually read Principles of Neural Science (by Kandel and Schwartz - http://www.amazon.com/Principles-Neural-Science-Eric-Kandel/...), I got the distinct impression that Kandel actually viewed the brain as an information processor, with lower level information being processed into higher level representations (see for example the chapter on the visual system).
So overall, the author takes a very limited view of both computers and brain and concludes (I think falsely) that the brain is not a computer.
If you're looking for something a bit more digestable, you could try "Biological Psychology: An Introduction to Behavioral, Cognitive, and Clinical Neuroscience", which is more appropriate for a survey level (100, 1000, depending on your school) course.
Get dozens of book recommendations delivered straight to your inbox every Thursday.