Hi Lei,
Welcome to the TI E2E forums! From your description I'm not sure you are getting the bad results.
Are you enabling the PGA buffer?
The buffer limits the absolute input range to the ADC. However, if your inputs are 0V and 40mV, I don't see an issue when using the buffer.
Is your ground really 0V or the same as the ADC's AGND?
If the AIN0 to AGND connection is done with a jumper wire and on a breadboard, you may be introducing parasitics into your circuit and measuring something other than what your expecting. Try probing AIN0 and AGND and see if there is a difference in voltage potential (it may be more than just a DC voltage potential).
Is it possible the ADS1256 is getting reset at some point and measuring AIN0-AIN1 (default), instead of AIN1-AIN0?
Do you have a schematic you could share with me?
Best Regards,
Chris