In answer my own question: according to some recent wiki posts, others with this device are having a similar issue. It seems that more reverse engineering is required to fix the problem.

It's possible that the noise is hardware related, perhaps due to external interference? The wiring is especially conducive to inducted spikes in the signal. Make sure that all cables and wires are kept a reasonable distance from power cables, wireless devices, and network cables.

I assume you've already run alsamixer and futzed with the volume levels there? I've seen some background noise get introduced when my Line-In and Mic levels were too high...

I've recently purchased an Edirol UA-4FX usb sound mixer, which is supported in a "standard" usb sound device mode which supports 16bit at 44.1Khz., and an "advanced" mode which allows up to 24bit at 96Khz as well as enabling use of the onboard midi ports. Alsa supports both the standard and advanced modes on this device, however, when I am in advanced mode, I am getting some extra background noise which is especially noticable at lower volumes and certain frequencies. I have attached a tar file containing two mp3s comparing the output of the standard and advanced modes. I've tried linux kernels 2.6.28, 2.6.29, and the git sources alsa code all with the same results. Is anyone else with this device having a similar issue?

Note: the file was too big to attach, so here's some urls for each:

Standard:
http://209.90.109.2/test-standard-mode.wav.mp3

Advanced:
http://209.90.109.2/test-advanced-mode.wav.mp3