There is plenty of math that does not lie on the academic beaten path (i.e. arithmetic -> algebra -> calculus --> etc.) I would recommend trying to figure out what area of math you are most interested in first, that way you will be enthusiastic about learning it. Once you have the mathematical momentum from that you will probably be able to learn whatever math you need to. A couple of books that helped me to find a direction were Mathematical Ideas[1] and Concepts of Modern Mathematics[2], which cover many different areas with just enough depth to give you a feel for each. If you are wanting to learn math solely for CS I would also recommend something like Schaum's Outline of Discrete Mathematics[3].

[1]http://www.amazon.com/Mathematical-Ideas-Edition-Charles-Mil...

[2]http://www.amazon.com/Concepts-Modern-Mathematics-Dover-Book...

[3]http://www.amazon.com/Schaums-Outline-Discrete-Mathematics-R...