The problem is (probably) that the computer generates noise on the USB ground. The noise current drains to the safety ground at the power amp through the USB cable and the RCA interconnects. The cables and connectors have some small but not zero resistance, which converts the current to a voltage that is effectively in series with the music.
There are "USB ground isolators" - google that phrase. I got 377,000 results when I did. They are not expensive.