SUCCESS!
For documentation purposes here was my problem:
I was using an older laptop machine which initially showed "workable" max jitter of 90,000ns without any tweaking or tuning and running normal test loads as described
here
. This was
far from perfect, but enough to get some motors turning.
HOWEVER, I would occasionally see a spike to > 6,000,000ns. Being a new to linuxcnc I dismissed this as anomalies and when I didn't see any of these in a multi-hour test run I convinced myself that it wasn't a big deal and that I would look into it later.
So what was happening is exactly as PCW described. Those rare spikes of delay were actually very consistent when my computer would attempt to connect to the 7i76e board. That would cause the watchdog to flip and eventually loose the serial connection. The watchdog has_bit was true. In my case I could even flip this to false and it would run for a while then fail again.
I replaced the laptop with another machine that, after 5 minutes of tuning (I found
this thread
helpful), had 45,000ns of max latency with no spikes. The latency-plot tool was very helpful.
I know this was totally a noob problem, so thanks for the help. I chose to make a linuxcnc based machine because I wanted to actually know and understand the details of my machine and I am not disappointed.