From your description, it sounds as if something stresses the device (or the system - for example the power supply) resulting in the class-D modulator misbehaving grossly.
The rationale for my question was that when you stop the I2S interface, the noise continues, which points to a rather drastic failure mode. If this happens at full volume settings, the part will draw fairly high currents for loud audio signals, which may cause short drops in the power supply, which may result in unpredictably behavior. The fact that it recovers when you disable/enable the part also supports this assumption.
However, because this also happens when you disconnect the speaker altogether, it does not appear to be related to the device stressing the power supply.
In any case, this sounds like there's something wrong on the system level or that the device is damaged. Please check that all power supplies are within the required limits (all the time - even short glitches are a problem).