If you buy a 20 or 30 dollar microphone (like audio-technica) and hook
it up to your sound hard using a 1/4/ to 1/8 adapter you should be
well off. The weekest link in your setup is the cheap desktop
microphone.
If you really want a great set up get a 60 dollar mixer, a 30 dollar
microphone, and a better sound card. Yeah..the sound card will run you
150 or so (and get one that has RCA or 1/4 inputs).
I'm researching digital voice recorders myself at the moment. What is
important is the memory that comes with it, the sampling rate, and
the frequency responce rate.