Hi Eduardo,
The self-test mode is used to check the internal logic of the DAC. The real purpose of this mode is to test timing and logic functionality during production. All units that are shipped have passed this self-test. Unless you have severely damaged the chip, there shouldn't be any reason to run this mode. Just to clarify what self-test mode does, it essentially applies a known input to the digital logic and compares the output from the logic against the expected value. If it makes it through correctly, the logic is considered valid. The data input bus is ignored during this test because the self test pattern is internally generated. It is not meant as an analog output test.
FIFO_err occurs if the read and write pointers of the input FIFO collide. This could be due to incorrect clock setup (wrong frequency, not synchronized) or because you are not synchronizing the FIFO. It sounds like your clock frequencies are correct, so you must not be synchronizing the FIFO. To do that, first make sure your clocks are stable, then issue a SYNC event. This can be done by providing a rising edge on the SYNC input (see paragraph on page 39 of datasheet) or using the SW_sync through SPI. See the recommended startup sequence on page 40, specifically step 8.
After the FIFO is synchronized, you need to clear the "FIFO_ERR" by writing all 0's to the STATUS4 register. Then read back STATUS4 to see the state of the alarm.
Regards,
Matt Guibord