The Intel reference manual is incredibly bloated and dry reading. Yeah, it has literally everything you would want to know. But good luck trying to understand all of it in a reasonable amount of time.
I learned x86 while studying buffer overflows in college. We used Hacking: The Art of Exploitation which walked us through most of the core concepts really well.
https://www.amazon.com/Hacking-Art-Exploitation-Jon-Erickson...
Quick link to Amazon: http://www.amazon.com/Hacking-The-Art-Exploitation-Edition/d...
And the world was never the same again.
Hacking: The Art of Exploitation, 2nd Edition
http://www.amazon.com/Hacking-The-Art-Exploitation-Edition/d...
Hacking Exposed 7
http://www.amazon.com/Hacking-Exposed-Network-Security-Solut...
This next site is basically Youtube for security conferences. They also offer some online courses on writing exploits in assembly and python, but not all of them are free.
Computer security is just like programming, you can obtain a world class education for free, from the Internet. You just have to know where to look.
I haven't read the books that the other guys mentioned, but I've seen them recommended so often that I'd bet they're worth a read as well.
Incidentally, have you ever read Hacking: The Art of Exploitation (http://www.amazon.com/Hacking-The-Art-Exploitation-Edition/d...)? It's my favorite book on the subject.
https://www.amazon.ca/Hacking-Art-Exploitation-Jon-Erickson/...
Which I've dabbled in, and haven't gotten further than what I already know from my CS education, but the consensus seems to be it's a good book to learn from.