I'm not missing arbitrary frequency components; please reread the penultimate paragraph, where I mention them explicitly. As I said there, you can extend my argument about a single frequency to include all multiples of that base frequency, and since you can set the coefficient for each frequency arbitrarily based on your ordering, you can set the fourier coefficients arbitrarily and in so doing recover any waveform you want.

Also, your point about commutativity is more subtle than you think; it fails for an infinite sum because you have an infinite space in which to rearrange things. Sure, the terms cancel eventually, but you can keep sticking the negative terms farther and farther back in a pattern so that by the time they've cancelled earlier positive terms, there's already a bunch of new positive terms to take their place. The subtlety comes from the fact that you can keep doing this forever, and you can do it in a way where the sum eventually converges to a specific value.

But don't take my word for it. This is an extremely well-known and basic result in mathematical analysis (the fancy math term for calculus and related topics). Again, see links above, or go straight to a proof [0]. If you want a deeper understanding, check out Rudin's Principle's of Mathematical Analysis [1], which explains this and other fun math stuff very well.

[edit] Just to be crystal clear, the Riemann series theorem does not apply to partial sums, which is what you are saying; if you do an infinite sum on a conditionally convergent series (like the alternating harmonic sum, a variation on which I used in my example), then your final result can literally be any number you want based on how you order the terms in the series. You can set it up so that the infinite sum keeps getting closer an closer to an arbitrary value. If this sounds nonintuitive, it's because infinite phenomena are subtle and nonintuitive!! This is a very cool example of how weird things get once you start dealing with the infinite.

[0] https://en.wikipedia.org/wiki/Riemann_series_theorem#Proof

[1] https://www.amazon.com/Principles-Mathematical-Analysis-Inte...

I wouldn't recommend Spivak for self-study and a first exposure to analysis. It's known to be notoriously difficult even for good students.

I learned from this Dover book[1]; I think it's pretty good. From there you might move up to Baby Rudin[2], but it might have a lot of typos, or big gaps in the exposition that are taken to be obvious but require several steps to fill in, since that was certainly the case for Papa Rudin[3].

[1]: https://www.amazon.com/Introduction-Analysis-Dover-Books-Mat... [2]: https://www.amazon.com/Principles-Mathematical-Analysis-Inte... [3]: https://www.amazon.com/Real-Complex-Analysis-Higher-Mathemat...

I'm a big fan of SICP. I'd describe it as "hard-core": requiring hard work, but extremely rewarding for the people who take the time and effort to go through it.

I'd recommend the following three books to similarly cover the three main areas of higher mathematics (analysis, algebra, and topology):

1. Principles of Mathematical Analysis by Rudin (http://www.amazon.com/Principles-Mathematical-Analysis-Inter...)

2. Algebra by Artin (http://www.amazon.com/Algebra-2nd-Featured-Titles-Abstract/d...)

3. Topology by Munkres (http://www.amazon.com/Topology-2nd-James-Munkres/dp/01318162...)

Working through all three (including the exercises!) will give you a solid understanding of the basis of modern mathematics. If you don't have experience with proofwriting, you might find them difficult at first - the activity is very different from performing calculations or solving equations. It's also best to have someone trained in mathematics talk to you about the proofs, until you develop a feel for the needed level of logical rigor.

Rudin, in particular, leaves a lot of work to the reader; going through that book is the most intellectually difficult work I've ever done. If you find it hard-going (which is completely natural), you might want to try Artin first, especially since you have some background in algebra.

(Incidentally, these are the three books used to teach analysis, algebra, and topology to MIT mathematics majors. You can look up the assignments and exams for the three courses - 18.100B, 18.701, and 18.901 - for a good list of exercises to work through.)