Hello Tianyi,
The ADC12D800RFRB3 uses the same schematics and layouts as the design package above. Thank you for pointing out the absence of FPGA source code, which I have attached below:
(Please visit the site to view this file)
For the clock amplitude, we specify a limit 0.4Vpp (-7dBm into 100Ω differential load). Since the clock receiver is detecting the zero crossing we expect that it will work below this level, but noise/jitter starts to have an impact on performance so we cannot recommend reliable operation.
However, if the board has not been modified then it is unlikely that clock amplitude is the problem. All of the boards we release have been checked for functionality and performance so we would be very surprised to see a hardware failure.
As one more check, when you toggle the "Clock Selection" from external back to internal, the "Sampling Rate" may not update until you take a capture. Can you try a capture event and let us know if it still has a time out error?
Thanks!