Probabilists often think of a uniform distribution over the whole real line as giving an infinitesimal (but non-zero) probability at each point. The intuition is very similar to the way physicists understand the Dirac delta function. Like the Dirac delta, an improper prior doesn't have any formally acceptable definition as a stand-alone mathematical object [0], but it does have a good definition that covers the way it's used.
Now, why does a uniform distribution correspond to an "uninformative" or "unbiased" prior in the first place? Because the uniform is the unconstrained maximum-entropy distribution. If you don't have rock-solid intuition about the concept of entropy, I recommend starting with Arieh Ben-Naim's book [1].
0. But with nonstandard analysis, it's a simple thing to give a rigorous, intuitive definition of a uniform distribution whose support includes as much of the real line as we could possibly care about.
Now, why does a uniform distribution correspond to an "uninformative" or "unbiased" prior in the first place? Because the uniform is the unconstrained maximum-entropy distribution. If you don't have rock-solid intuition about the concept of entropy, I recommend starting with Arieh Ben-Naim's book [1].
0. But with nonstandard analysis, it's a simple thing to give a rigorous, intuitive definition of a uniform distribution whose support includes as much of the real line as we could possibly care about.
1. A Farewell to Entropy, http://www.amazon.com/FAREWELL-ENTROPY-Statistical-Thermodyn.... It's a probability book masquerading as a physics book.