If the Input Sensitivity on my amplifier is 250mV - 4V, do I want to
watch what kind of power the preamped outputs are coming out of the
head units I'm considering getting?

To phrase that a different way; If I get a deck that has 5 Volt
pre-amped outputs, would that be a problem for the amplifier?

Or are they completely seperate specs that shouldn't be
compared/contrasted, and I should quit worrying?

~Mister.Lull