It would be more expensive, but generally only 2-3%. Which, for a short trip where I'm expecting to spend maybe 50 Euros or few hundred New Zealand dollars, is a fair tradeoff versus the amount of my time it would take to get the slightly more favourable rate in advance.

(Of course, if I *can* get the more favourable rate in advance without going way out of my way, I will.)

Anything more than that, yeah, getting it in advance is nice.