Need guidence to improve latency and speed

Hi all,

It’s been (far) too long since I last had time for CNC projects. Fortunately, things have settled down a bit lately, and I’ve been able to pick up the hobby again. However, getting my machine up and running has proven to be more of a challenge than expected.

My old Intel setup (with a 4790K) had both a faulty motherboard and RAM, so I replaced it with an ASUS AM4 board and a Ryzen 5600 CPU. Since then, I've been struggling with what seems to be very high latency.

To make matters worse, I used to run LinuxCNC 2.4, but I’ve now reinstalled everything using the latest stock ISO (2.9 with the PREEMPT-RT kernel). Unfortunately, this means my old configuration is no longer compatible.

I’ve uploaded my current settings, including the BIOS changes I’ve made in an attempt to reduce latency. At this point, my first goal is simply to get the axes moving at a reasonable rate. Right now, I can’t get it to move faster than about 1 mm/sec (rough estimate) before I run into either a joint 0 following error or an error about the base thread timing.

If anyone has tips or suggestions based on my setup or config files, I’d really appreciate the help. The machine used to run at a speed much faster than the tool could handle, so I know it's capable — I just need to get the configuration right again.

I’m aware that my .hal and .ini files likely need significant changes. I've done my best to port over the old configuration, but clearly, it’s not quite there yet.

Thanks in advance for any insights or advice!

The images are from 1 to 12, replace the number in the URL. I cant post that many links
www.doingcode.nl/images/linuxcnc/1.jpg

There's quite a few threads regarding latency, there's a quite a long one from just this week.
Your max speed will be a function of step per rev and your base thread.

You could try isolating the last 2 cores via the kernel command line.

Hi, thanks for your response!
Do you happen to have a link to that specific thread? I might have missed it.

I've read a lot of posts and tried many things in both the BIOS and on the CNC machine over the past month, but nothing seems to help so far. I haven’t tried any bootloader or CLI-based options yet, so that might be worth exploring next.

That's why i have the feeling i am missing something obvious because this is my first AMD CPU machine. In the past i had 2 intel systems before and never had this extreme latency. Also i always used my AMD Firepro graphics card. So the GPU did not change over time.


::EDIT:: Alright isolating cores seems to be a step in the correct direction. I changed my GRUB conf to:

GRUB_CMDLINE_LINUX_DEFAULT="quiet isolcpus=2,3 nohz_full=2,3 rcu_nocbs=2,3 idle=poll processor.max_cstate=0 amd_pstate=disable"
to free up my core 2 and 3. If i do a latency tests i now have a servo thread < 25 000 and a base thread < 120 000. So that is better. I think i it still not ideal to have such a high basethread... still the:
- joint 0 following error
- unexpected real-time delay

errors even if i start linuxcnc on the isolated cores via #!/bin/bash
taskset -c 2,3 linuxcnc /home/npostma/linuxcnc/configs/CNC/CNC.ini

Validated with `ps -eLo pid,psr,comm | grep -i linuxcnc` that it did run on the isolated cores. I also see in HTOP that the cores are isolated. No processes go to them.
 

Thanks again!

Keep playing with different combinations of cores isolated using isolcpus till you find what gives you the best result. Your Ryzen has 6 of them to choose from, I'd test different combinations of 1, 2, and 3 cores isolated, like 1,3,5 or 3,4,5 or 0,1,2, or 4,5, or 0,1...

Hi Todd,

Thanks for your response — I’ll definitely give that a try, as it feels like my last resort with this current setup.

I've already disabled everything I possibly could in the BIOS. The only things I haven't touched yet are the memory timings and voltages. If this doesn’t work either, I’m honestly not sure what else is left to tweak.

I'm also considering switching to a Mesa card, possibly the 7i76EU, but I have zero experience with Mesa hardware. I’ll need to figure out whether it will work with my three DM542A stepper drivers, and also with my VFD, which is currently controlled over USB using: `loadusr -Wn vfd hy_vfd -n vfd -d /dev/ttyUSB0 -p none -r 9600`

Things i disabled next to the obvious, like settings for fixed clock speeds (fclock 1066, bclock 1000) disabling power management and disabling hardware that is not being used (BT, Wireless, serial, audio, fancontrol (disabled fixed on 100%)):

BIOS Settings (set to Disabled):

SMT / Hyperthreading
→ Disables logical cores to reduce cache sharing and latency spikes

C1E, C6, and other C-States
→ Prevents the CPU from entering sleep states, ensuring consistent timing

Streaming Stores Control
→ Avoids aggressive memory write optimizations that can disrupt real-time behavior

DRAM Scrub Time
→ Turns off background memory error checking to avoid unexpected memory bus usage

GMI Encryption Control
→ Disables inter-chiplet encryption on Infinity Fabric to reduce overhead and latency

XGMI TXEQ Mode
→ Prevents signal equalization tuning that could introduce instability or variability

DRAM Map Intervention
→ Disables memory remapping logic that can introduce unpredictable latency

DF Sync Flood Propagation
→ Prevents error broadcasts across chiplets, reducing the chance of system-wide stalls

Mesa is definitely the way to go. It "future proofs" your setup. Swapping machines is easy, latency is less of an issue. If your current machine goes "poof" getting back & running is a simple task.

Tho it really depends on your budget....shipping to the Southern Hemisphere is brutal.

The easiest way, to prevent a total rewire is grab a 7i92T and that can plug into your existing parallel port breakout boards (or more correctly the breakout boards cab plug into the 7i92T). That'll get you going, later if you really feel the need a 7i76 can be plugged into the 7i92T.

I'm running a 7i92,7i76, another card for extra Smart Serial ports an analog smart serial card for the spindle and a 7i73 for the pendant.......Hasn't missed a beat in all the years I've been using them. Worked fine on the dual core J3355B system I used for a bit. Asrock J3355B-ITX. Upgraded to a system with a smaller foot print

Regarding isolating cores, from the dark spells I've read it would seem to isolate the cores on the same die, well that was the recommendations for Intel.

I wouldn’t mind rewiring everything — my setup is quite simple, and the CNC machine is fairly small. For me, building and tweaking the CNC is part of the fun of the hobby.

I’ve read that many of you use various Mesa cards. I was hoping the 7i76EU alone would be a good starting point to learn how to work with Mesa hardware. Do you know if that’s the case? I saw some brutal shipment costs (like 200 dollar plus, so i will have to search for an European supplier, i'm in the Netherlands)

If the 7i76EU would allow me to completely move away from the parallel port, that would be a big plus — especially since I’m currently limited by base-thread latency.

[EDIT]

I see a seller with reasonable shipping costs, not sure if its a trusted compane EUSurplus?I think i am planning to switch to a Mesa board and wanted to check if the 7i76EU is the right fit for my setup. This is what I’m working with:

• 3x DM542A stepper drivers (they use step/dir signals)
• 3x LJ12A3-4-Z/BY inductive proximity sensors (used for homing)
• 1x Huanyang VFD (spindle, controllable via 0–10V or RS-485 Modbus (I think because i now use it over USB)

From what I’ve researched, the Mesa 7i76EU seems like the best option. Here’s why:

• It supports up to 5 stepper/servo axes via step/dir signals (perfect for my DM542A drivers)
• It has 32 opto-isolated inputs which should work well with the 24V NPN homing sensors
• It has 16 outputs, which are configurable (sourcing/sinking)
• It includes an analog spindle output (0–10V) for VFD speed control
• It supports RS-485 communication if I decide to use Modbus for the Huanyang VFD
• It isolates field I/O power from logic power, which helps protect the system

Important to note:

• The sensors might need a pull-down resistor to give clean logic signals when connected to 24V. I already solderd a pc to convert the 30V signal back to a 5V system with opto's
• I will need to supply 24V field power to the board 
• I understand the 7i76EU requires a second Ethernet interface on my PC (either a second NIC or a USB↔Ethernet adapter)
• I still need a PREEMPT-RT kernel, but I won't have to worry about tight base-thread latency anymore

Does this sound correct to those of you with experience using this board?Thanks in advance!

 

Thanks! I think i will dive into the deep and order a 7i76EU. I don't need that much I/O now, but not sure in the future. I like to play with the idea of having more axis and sensors like endstops to the machine. Also i have coolant pump and airlines for blowing chips away. Its manually controlled at this moment, but automating this is on my list . One last question:

eusurplus.com/index.php?route=product/product&product_id=50

Is referring to a 'daughter' card. Would that imply that i need something extra?