threading - spindle encoder synchonization issues

I think I got to the bottom of it. There were some locations I could twist the spindle (and hence spindle encoder) to manually that resulted in continued increases or decrease of the spindle position. This told me it was still noise so I dug into it.

I have no idea why, but on the mesa 7i76 the encoder input can be differential or single ended. I was using it in differential mode because I thought it would be more noise immune. I also had a "backpack" sort of thing on my encoder that made the single-ended outputs 3.3V differential. In this mode it uses a Renesas ISL32173 chip to take in the differential signal through what amounts to a comparator and output a logical signal to some counter in the FPGA. The issue is that the chip is designed to be really sensitive and a differential signal over 0.2V is considered high. That is nuts on a signal that is 3.3V or 5V differential. Hence, despite all my work it was still noise, but I decided to just go back to 5V single-ended at 5V directly out of the encoder and now all is well... despite the noise levels being higher in absolute terms the signal to noise ratio is much much better and I can cut perfect threads! (Yeah I had my scope on it a lot throughout this.)

I don't know why there is no warning in the datasheet about this. 200 mV of noise on a 3.3V-5V signal should NOT be causing this kind of issue. Typical levels of a high bit on 3v3 logic is >2V. I think for some reason they used an RS-422 chip and it was not the right choice in my opinion... especially without a giant flashing warning.

Maybe I'll mention this to Peter at mesanet. They make a great product... but I think this was a bit of an oversight.

In this mode it uses a Renesas ISL32173 chip to take in the differential signal through what amounts to a comparator and output a logical signal to some counter in the FPGA. The issue is that the chip is designed to be really sensitive and a differential signal over 0.2V is considered high. That is nuts on a signal that is 3.3V or 5V differential.

200mV diff voltage threshold comes from the EIA-422 standard. i believe every compliant receiver IC does the same. nevertheless, usually they perform nicely in pretty harsh noise environment. i suspect your "backpack" may be a problem, not the 7i76

Yes, I think that there's a misunderstanding of how differential signaling works here.

The high input sensitivity allows detection of small differential signals but the differential noise
immunity is equal to the differential signal level from the source.  Because it is a differential
receiver designed for a high noise environment, it has a wide common mode range (-7 to +12V)
meaning it can detect small differential signals (> 200 mv) accurately even with -7 to +12V of
common mode noise. This is why RS-422 signaling is used for encoders.

If the 7I76 single ended setting works better than the differential setting on supposedly
differential signals it suggests that there is some hardware issue.

I would check first that you have true differential signals from the encoder.

To do this, use a voltmeter across the differential pairs (A,/A B,/B IDX,/IDX)
and measure the voltage as you jog the encoder a bit to get both states.

One state should read a positive voltage and the other a negative voltage.
in no case should the absolute differential voltage be less than 2V.