IPPM software. Considered unlikely because the relevant part is identical to that of the DFN IPPM system and such an effect is not observed there.
OS/hardware. Because packets are timestamped in user space, there could be considerable delays due to buffering on the NIC/in the kernel or due to kernel/user-space interaction.