It's probably a bit out of date now, but my dog-eared copy is still a good read. Ah, nostalgia. There once were days when I dreamed that a CS degree would make me as a god; the silly thoughts of a child. Now I know that it is a _PhD_ which makes gods of men.
I'm not sure what good beginner resources for learning assembly programming are, but either way, you'll probably want a copy of the Intel instruction set reference manuals[2] (and maybe the other Intel/AMD manuals[3]).
[1] http://www.amazon.com/Reversing-Secrets-Engineering-Eldad-Ei...
[2] vol1: http://www.intel.com/Assets/PDF/manual/253666.pdf vol2: http://www.intel.com/Assets/PDF/manual/253667.pdf
[3] Intel: http://www.intel.com/products/processor/manuals/ AMD: http://developer.amd.com/documentation/pages/default.aspx
Smashing The Stack For Fun And Profit [1] Reversing: Secrets of Reverse Engineering [2] The IDA Pro Book [3]
The iOS Hacker's Handbook [4] was interesting as a sort of case study on exploiting and hacking embedded hardware.
Mostly what I've found, though, is just starting with a question and googling the answer yield the most results. For example, see mention of a stack overflow attack google how and why stack overflow attacks work (or don't) and once that side of things is understood the thought process behind finding them becomes easier to understand, although not really easier to do (for me, at least).
[1] http://insecure.org/stf/smashstack.html
[2] http://www.amazon.com/Reversing-Secrets-Engineering-Eldad-Ei...
[3] http://www.amazon.com/IDA-Pro-Book-Unofficial-Disassembler/d...
[4] http://www.amazon.com/iOS-Hackers-Handbook-Charlie-Miller/dp...