I haven't read it, but "PGP: Source Code and Internals", by PGP's author Philip Zimmerman is worth a try. I've read his other book "Official PGP User's Guide" and learned quite a bit.

https://www.amazon.com/PGP-Internals-Philip-R-Zimmermann/dp/...