A bit older, but both worth a read, and most of this is still relevant today...

https://www.amazon.com/High-Performance-Web-Sites-Essential/...

https://www.amazon.com/Even-Faster-Web-Sites-Performance/dp/...

Get the two Souders (creator of YSlow, now works on web performance at Google) books, which are amazing. I'm not affiliated with Amazon.com, O'Reilly or Yahoo!; just like Patrick says, these guys know their shit:

High Performance Web Sites (O'Reilly, Sept. 2007): http://www.amazon.com/High-Performance-Web-Sites-Essential/d...

Even Faster Web Sites (O'Reilly, June 2009): http://www.amazon.com/Even-Faster-Web-Sites-Performance/dp/0...

Half of King's 'Website Optimization: Speed, Search Engine & Conversion Rate Secrets' (O'Reilly, July 2008) is devoted to performance, so it's also worth a look: http://www.amazon.com/Website-Optimization-Search-Conversion...

The book (O'Reilly's High Performance Web Sites: Essential Knowledge for Front-End Engineers): http://www.amazon.com/High-Performance-Web-Sites-Essential/d...

And his upcoming book (O'Reilly's Even Faster Web Sites: Performance Best Practices for Web Developers): http://www.amazon.com/Even-Faster-Web-Sites-Performance/dp/0...