7i96 issue read/write

Hi,

I run two cnc machines from a single PC box, one cnc lathe one cnc router, both are running 7i96. Each machine has independent 7i96 and PSU's etc. I never need to run these at the same time, so use one PC machine to run them (switching the network cable and LCNC instance as necessary)

Recently, I have been getting sporadic errors with both machines with:

hm2/hm2_7i96.0: error queuing read! iter="number"
or
hm2/hm2_7i96.0: error queuing write! iter="number"

This happens on both the cnc router and lathe (with different network cables) so, I expect it is something with the local machine/networking.

The strange thing is, this works fine until recently. Nothing has changed on the machine (it is not network/internet active) so no updates or software changes.

Is there some initial things I can do to test what might be happening, bios settings are all good from what I can see. I have used this exact machine for LCNC for many years and it is solid, although 7i96 is recent ish to me, only on and off use for the past 12 months or so.

Latency figures are very low, but I can run new tests to check that out to see if something might be found there. No hyper threading enabled. Motherboard is a gigabyte, I have read on my limited searching that certain kernels with Debian can result in this issue with certain network hardware, is there something there I can look at? 

I am running 2.9..0-pre0-3879-g9a26e69fe (I need dual synchronised axes in the Y for my router).

I just find it quite strange that this has just started to occur. And, on both machines would suggest it is the networking. This sounds very unlikely but, is there any potential for issues with temperature? I'm running a long program just now and it 'so far' is running just fine, earlier when the space was cold, it errors out. I seem to encounter it more when space is cold, maybe a long shot and I can't think of why that would matter. 5V supplies are all over specified and test out fine.... Any things I could try?

CPU: E-350D Dual core (isolcpus set to 1 for lowest latency here)
Network: Realtek GbE Lan 

Thanks, best, John

Edit: running latency test as we speak, so far Servo is 12k, Base 10k. I'll leave it to run for a while to see if I get any spikes. Although I am not sure that makes a lot of difference here. I ran two long programs when the machine was warm, dry run, no issues. V. Strange.

I ran a few tests after changing ini settings.

Increased servo from 1,000,000 to 2,000,000

Changed joint P values from 1000 to 500 (a suggestion read elsewhere)

One program ran fine, the next had same issue.

I have disconnected all vfds during tests to eliminate noise from equation as best as possible. Still no joy. If there is a solid recommendation for an Ethernet card/chipset for debian/mesa im happy to get one to see if that remedies the situation.

If no software was changed, those errors sound like a hardware
failure somewhere, perhaps a loss of link?

does dmesg show any link up/down transitions? ( other than at host or 7I96 power up )

Yes. It does seem to be a loss of connection. No physical issues with wiring but I will check as you say and report back. In cases where it occurs, lcnc halts all comms to drivers (no movement possible) restarting lcnc resolves without having to touch anything physical. Seems so many packets fail and it halts. Im away from workshop atm but will check asap

Typical error steam for ref. Edit. Photo too large. Ill send later

The error you get does suggest something odd with the host hardware.
Normally if you lose the link, you will get "error finishing read"

Here's the result of either setting the interface down
or simple pulling the RS45:

hm2/hm2_7i92.0: error finishing read! iter=2782471
hm2/hm2_7i92.0: error finishing read! iter=2782471

I did this at least 20 times and never got any queue related errors