It could be a problem with the digital audio interface. When you are using the codec as a slave, it is necessary to provide the exact correct timing. If you are using a non-standard clock and the on board PLL, there is a good chance the internal audio will be out of sync with your external timing (causing noise).
I would try to use the codec as the audio master.
I also noticed that you are using a class d amp with no LP filter between the codec and the amp. You should have a LP filter such as a simple RC filter with a pole around 40 kHz.