Smashing The Stack For Fun And Profit 
Reversing: Secrets of Reverse Engineering 
The IDA Pro Book 
The iOS Hacker's Handbook  was interesting as a sort of case study on exploiting and hacking embedded hardware.
Mostly what I've found, though, is just starting with a question and googling the answer yield the most results. For example, see mention of a stack overflow attack google how and why stack overflow attacks work (or don't) and once that side of things is understood the thought process behind finding them becomes easier to understand, although not really easier to do (for me, at least).
It's probably a bit out of date now, but my dog-eared copy is still a good read. Ah, nostalgia. There once were days when I dreamed that a CS degree would make me as a god; the silly thoughts of a child. Now I know that it is a _PhD_ which makes gods of men.
I'm not sure what good beginner resources for learning assembly programming are, but either way, you'll probably want a copy of the Intel instruction set reference manuals (and maybe the other Intel/AMD manuals).
 vol1: http://www.intel.com/Assets/PDF/manual/253666.pdf vol2: http://www.intel.com/Assets/PDF/manual/253667.pdf
 Intel: http://www.intel.com/products/processor/manuals/ AMD: http://developer.amd.com/documentation/pages/default.aspx
Fresh book recommendations delivered straight to your inbox every Thursday.