floating-point

Python float vs Decimal: why 0.1 × 100000 is exactly 10000 in binary64, rounding and precision

Jan. 13, 11:00

Stop Losing Precision: Use Python time.time_ns() for Integer Nanosecond Timestamps, Not Floats

Nov. 8, 01:00

JAX vmap numeric drift across batch sizes on GPU float32: causes, reproduction, fixes

Oct. 18, 01:00

Silent Data Corruption in NumPy/Numba complex128 Arrays: Hardware Memory Errors, MemTest86, ECC

Sep. 23, 05:00

1