I'm not missing arbitrary frequency components; please reread the penultimate paragraph, where I mention them explicitly. As I said there, you can extend my argument about a single frequency to include all multiples of that base frequency, and since you can set the coefficient for each frequency arbitrarily based on your ordering, you can set the fourier coefficients arbitrarily and in so doing recover any waveform you want.

Also, your point about commutativity is more subtle than you think; it fails for an infinite sum because you have an infinite space in which to rearrange things. Sure, the terms cancel eventually, but you can keep sticking the negative terms farther and farther back in a pattern so that by the time they've cancelled earlier positive terms, there's already a bunch of new positive terms to take their place. The subtlety comes from the fact that you can keep doing this forever, and you can do it in a way where the sum eventually converges to a specific value.

But don't take my word for it. This is an extremely well-known and basic result in mathematical analysis (the fancy math term for calculus and related topics). Again, see links above, or go straight to a proof [0]. If you want a deeper understanding, check out Rudin's Principle's of Mathematical Analysis [1], which explains this and other fun math stuff very well.

[edit] Just to be crystal clear, the Riemann series theorem does not apply to partial sums, which is what you are saying; if you do an infinite sum on a conditionally convergent series (like the alternating harmonic sum, a variation on which I used in my example), then your final result can literally be any number you want based on how you order the terms in the series. You can set it up so that the infinite sum keeps getting closer an closer to an arbitrary value. If this sounds nonintuitive, it's because infinite phenomena are subtle and nonintuitive!! This is a very cool example of how weird things get once you start dealing with the infinite.

[0] https://en.wikipedia.org/wiki/Riemann_series_theorem#Proof

[1] https://www.amazon.com/Principles-Mathematical-Analysis-Inte...

I wouldn't recommend Spivak for self-study and a first exposure to analysis. It's known to be notoriously difficult even for good students.

I learned from this Dover book[1]; I think it's pretty good. From there you might move up to Baby Rudin[2], but it might have a lot of typos, or big gaps in the exposition that are taken to be obvious but require several steps to fill in, since that was certainly the case for Papa Rudin[3].

[1]: https://www.amazon.com/Introduction-Analysis-Dover-Books-Mat... [2]: https://www.amazon.com/Principles-Mathematical-Analysis-Inte... [3]: https://www.amazon.com/Real-Complex-Analysis-Higher-Mathemat...