Actual position and Commanded position differ, using cia402 [SOLVED]

I'm very new to LinuxCNC, and using a CNC machine custom built and configured by someone else. Running into some errors that none of us could figure out, help would be very very appreciated.

We are using cia402 and ethercat to run a 9-axis system, including xyz and 6 rotation axis. We noticed that the actual and commanded positions of the xyz axes are always off by a factor of 80 , e.g. if we send the Gcode to move X axis by 10mm, it moves by 10mm, the commanded position is 10, but the 'actual position' is 0.125. We can still run .ngc files, they do exactly what we expect them to do, but at the end we get motor position error on these three axes.

We've tried increasing the FERROR and MIN_FERROR in the .ini file

...

[AXIS_X]
MIN_LIMIT         = -5
MAX_LIMIT         = 395
MAX_VELOCITY         = 500
MAX_ACCELERATION     = 250

[JOINT_0]
TYPE             = LINEAR
MAX_VELOCITY         = 500
MAX_ACCELERATION     = 250
FERROR             = 200
MIN_FERROR         = 50
MIN_LIMIT         = -1
MAX_LIMIT         = 1200
HOME            = 0
HOME_OFFSET        = 0
HOME_SEQUENCE        = 1
HOME_IGNORE_LIMITS    = YES
# nach github.com/dbraun1981/hal-cia402/blob/main/README.md

HOME_SEARCH_VEL     = 5.0
HOME_LATCH_VEL         = -2
HOME_FINAL_VEL        = 1
HOME_USE_INDEX    = FALSE

...
 .hal:

loadrt [KINS]KINEMATICS
loadrt [EMCMOT]EMCMOT servo_period_nsec=[EMCMOT]SERVO_PERIOD num_joints=[KINS]JOINTS

loadusr -W lcec_conf ECOMODUL_ethercat_conf.xml
loadrt lcec
loadrt cia402 count=9

###########################################################
# Functions servo-thread
###########################################################

addf lcec.read-all servo-thread
addf cia402.0.read-all servo-thread
addf cia402.1.read-all servo-thread
addf cia402.2.read-all servo-thread
 ...

addf motion-command-handler servo-thread
addf motion-controller servo-thread #########################################
#nets
#########################################
net emc-enable => iocontrol.0.emc-enable-in
sets emc-enable 1 #config
#testing Werte für pos-scale fehlen
#x
setp cia402.0.csp-mode 1
setp cia402.0.pos-scale 8000
#y
setp cia402.1.csp-mode 1
setp cia402.1.pos-scale 8000 
#z
setp cia402.2.csp-mode 1
setp cia402.2.pos-scale 8000 ...

#from servo(ethercat) to cia402 net x-statusword      lcec.0.8Axis.xcia-statusword => cia402.0.statusword
net x-opmode-display  lcec.0.8Axis.xopmode-display => cia402.0.opmode-display
net x-drv-act-pos     lcec.0.8Axis.xactual-position => cia402.0.drv-actual-position net y-statusword      lcec.0.8Axis.ycia-statusword => cia402.1.statusword
net y-opmode-display  lcec.0.8Axis.yopmode-display => cia402.1.opmode-display
net y-drv-act-pos     lcec.0.8Axis.yactual-position => cia402.1.drv-actual-position  net z-statusword      lcec.0.8Axis.zcia-statusword => cia402.2.statusword
net z-opmode-display  lcec.0.8Axis.zopmode-display => cia402.2.opmode-display
net z-drv-act-pos     lcec.0.8Axis.zactual-position => cia402.2.drv-actual-position

...
#from cia402 to servo(ethercat) net x-controlword         cia402.0.controlword     => lcec.0.8Axis.xcia-controlword
net x-modes-of-operation  cia402.0.opmode     => lcec.0.8Axis.xopmode
net x-drv-target-pos      cia402.0.drv-target-position   => lcec.0.8Axis.xtarget-position net y-controlword         cia402.1.controlword     => lcec.0.8Axis.ycia-controlword
net y-modes-of-operation  cia402.1.opmode     => lcec.0.8Axis.yopmode
net y-drv-target-pos      cia402.1.drv-target-position   => lcec.0.8Axis.ytarget-position net z-controlword         cia402.2.controlword     => lcec.0.8Axis.zcia-controlword
net z-modes-of-operation  cia402.2.opmode     => lcec.0.8Axis.zopmode
net z-drv-target-pos      cia402.2.drv-target-position   => lcec.0.8Axis.ztarget-position

...
#from motion to cia
net x-enable    <= joint.0.amp-enable-out => cia402.0.enable
net x-amp-fault => joint.0.amp-fault-in   <= cia402.0.drv-fault
net x-pos-cmd   <= joint.0.motor-pos-cmd  => cia402.0.pos-cmd
net x-pos-fb    => joint.0.motor-pos-fb   <= cia402.0.pos-fb net y-enable    <= joint.1.amp-enable-out => cia402.1.enable
net y-amp-fault => joint.1.amp-fault-in   <= cia402.1.drv-fault
net y-pos-cmd   <= joint.1.motor-pos-cmd  => cia402.1.pos-cmd
net y-pos-fb    => joint.1.motor-pos-fb   <= cia402.1.pos-fb net z-enable    <= joint.2.amp-enable-out => cia402.2.enable
net z-amp-fault => joint.2.amp-fault-in   <= cia402.2.drv-fault
net z-pos-cmd   <= joint.2.motor-pos-cmd  => cia402.2.pos-cmd
net z-pos-fb    => joint.2.motor-pos-fb   <= cia402.2.pos-fb

langdons, sometimes it is better not to answer. It should be obvious from the post that this is an ethercat machine.

I don't see anything obvious wrong in the setup.
8000 is a lot, 8000 steps or ticks or whatever per mm but that is probably right since it moves right.
What you can do to debug this is to use "halshow" and look at all pins in the chain from joint.0.motor-pos-cmd to the cia402 pin, to the driver pin, and the return from the driver pin, to the cia402 pins, to joint.0.motor-pos-fb pin.

I can't jog or do anything after running a program, I get motor position error on xyz axes afterwards and have to restart linuxcnc before I can reconnect to the machine. Same thing if I run a few lines in MDI mode and switch back to manual control.

Yes the x, y and z axes are driven by leadscrews.

Yes they are stepper motors.

I did double check the unit last week, we have everything set in mm. I wish it was as simple as that.

Did you try with halshow? That is the way to debug this.
When linuxcnc is still running, start halshow in a terminal window. The machine can have e-stopped, that's ok.
Just type halshow at the prompt.
Navigate through pins and investigate the values of the pins outlined above.
This should hopefully give an understanding of where things go wrong.

I just tried that, not sure if I found something useful but something strange did come up.
The pins I looked at were:
joint.0.pos-cmd
joint.0.pos-fb
cia402.0.drv-actual-position
cia402.0.drv-target-position
joint.0.motor-pos-cmd
joint.0.motor-pos-fb
cia402.0.pos-cmd
cia402.0.pos-fb
axis.x.pos-cmd
cec.0.8Axis.xactual-position
lcec.0.8Axis.xtarget-position

all seem like they're what they should be except axis.x.pos-cmd, after I sent G0 X50 in MDI mode, it was 50, everything was fine, then I switched to manual control mode, it changed to 0.625. That is also when I get the motor position error.

I took some screenshots at startup, after homing, after sending a G0X50 command and after I got the error. I don't have anywhere to host the images though, so I put them in the attachment in case that helps.


 

Suddenly a big shift in axis.x.pos-cmd while joint.0.pos.cmd stays? When you shift between MDI and Manual.
The translation between joints and axis is done in the kinematics module but there are more things involved I don't know so much about.
I see you have a userkins module, that one becomes suspicious of course.

You can follow the signals, ms by ms, using the halscope utility.
Maybe interesting to do that when you shift between MDI and manual.

Judging from the halshow output you EtherCAT drives behave as they should. The problem isn't there.
 

I checked the userkins.comp file, found nothing strange, but in userkins.c, it was

int kinematicsForward ...
   ...
    pos->tran.x = j[0]/80; // X coordinate
    pos->tran.y = j[1]/80; // Y coordinate
    pos->tran.z = j[2]/80; // Z coordinate

while in userkins.comp or either of the kinematicsInverse functions, the /80 was not there. I tried compiling again, nothing changed. I tried deleting the /80 from userkins.c, everything is now working. I just have no idea why..

Could be, but I can't figure out where it's coming from, so looks like I'm stuck with a temporary fix of a possible temporary fix for now.

I remember Python giving me so much grief over the tabs and spaces in my first year of learning to program! At least these days IDEs take care of that problem.

C just went completely over my head when I tried to learn it. I can mostly read it but I cannot write anything in C, it's pretty frustrating