MIT press has an excellent book, "The First Computers- History and Architectures"[1] that discusses a number of potential "firsts" based on varying criteria- well worth a read, if you're into this sort of thing. I'd agree that the Z1 was the first complete, working machine that I consider a computer.

[1]http://www.amazon.com/First-Computers--History-Architectures...