Hi Alexander,
That time domain definitely looks wrong since your signal should be centered at code 8192. Let's get back to the default parallel configuration. First, set the following jumpers as such:
JP8: Short 1-2
JP9 Short 1-2
JP10 Short 1-2
JP11 Short 1-2
This will put the ADC in parallel configuration mode. Next, set JP14 for "2'S COMP, LVDS". Then power on the board and take a capture. You should be able to get a good capture now, with the signal centered at code 8192.
If your SNR is still low at the point, please check the clock and analog inputs. Both should have tight bandpass filters to remove harmonics and wideband noise. The clock should be at least 1 Vpp at the ADC (after filter/transformer losses).
Regards,
Matt Guibord