Ethercat random jitter fix

More
5 months 3 weeks ago #346280 by grandixximo
It could work, after lots of testing, I think the noise papagno is experiencing is mostly caused by the servo thread spiking above 1ms, so widening the cycle time could certainly help.
The main reason why Debian 10 does not grind in his case, is mostly because Debian 10 with kernel 4.19 gets lower latency, and this is pretty universal IMO, Debian 13 Kernel 6.12 has overall worse latency, although with system setup tooling that you can get from Rodw, the latency can be improved a lot.
There is only one slight latency regression with the new linuxcnc-ethercat, the modulo operation that I currently must use to do phasing is causing slightly more latency on the servo-thread, and I want to fix that, I am preparing patches to modify Linuxcnc to expose a function to allow proper native phasing, so that I can lighten up linuxcnc-ethercat to match the older performance for both positive and negative refClockSyncCycles modes. Previously I was undecided if going that path, or using Sascha Ittner method/hack of moving ecrt_master_application_time out of lcec_main, but I found that Sascha's hack causes issues with papagno's hardware, I tried this with built binaries from Sittner repo, with refClockSyncCycles -1 nothing would move, therefore I am now convinced the proper path is to have a native linuxcnc phasing function, this will require updating both linuxcnc, and rebuilding linuxcnc-ethercat against the new linuxcnc-dev headers, hopefully the patches will be ready and merged soon, will try to keep you guys updated on the progress here, I know there are people following in the background ;-)
The following user(s) said Thank You: besriworld, andrax, tiagounderground

Please Log in or Create an account to join the conversation.

More
5 months 2 weeks ago #346367 by papagno-source
Good morning everyone.
After a few weeks of testing on a station with drives and I/O peripherals, we managed to achieve good results with GRandixximo.
The work done by Grandixximo significantly improves EtherCAT performance, but it must be combined with the kernel optimizations that Rodw has developed.
Using these two optimizations, on the EtherCAT and kernel sides, we are approaching Debian 10 performance.
The Debian 13 solution, when stressed by simultaneously opening multiple browsers and graphical tools, sporadically made noise for a fraction of a second, both with refclocksync -1 and 1, but I'm sure that using a PC more suitable for EtherCAT, stable machines can be achieved.

On Debian 10, the drives do not emit any noise under any conditions.

Grandixximo will now prepare the configuration packages and documentation.
As soon as they are ready, I will be able to test them on a few other machines to confirm everything.

We thank all the participants in this discussion.

Please Log in or Create an account to join the conversation.

More
5 months 2 weeks ago #346368 by rodw
Replied by rodw on topic Ethercat random jitter fix
This is great news. There has been a lot of ethercat related development activity driven by grandixximo on both Linuxcnc and the linuxcnc-ethercat hal driver. When a few more things come together, I will be refreshing my ISO's which will put everything on the one ISO. 

Its great that Grandixximo is now the owner of the  linuxcnc-ethercat repository and he's implemented a number of changes with more in the pipleine. He's also got a PR with LinuxCNC that will provide a "hook" in linuxCNC to allow synchronizing the servo thread with Ethercat before the servo thread starts which should further improve results and perhaps remove that little squeak even..

Thank you for your endorsement of my kernel tuning optimizations. Its very valuable feedback being able to state Trixie (Ddebian 13) is as good as Debian 10 once tuned. Having a system that queries your PC as it is optimised has reduced manually tuned jitter by 50% in my testing. I have never been very rigorous about this testing as it needs to be installed on hardware  and I have to stuff around compiling a wifi driver every times so its easy to loose data but it seems to improve with every iteration as its given me a place to record every new optimisation I unearth.

At risk of upsetting some, you can find out more about my optimisations on my Youtube channel  www.youtube.com/@MrRodW or the companion website  mrrodw.au/  
The following user(s) said Thank You: andrax

Please Log in or Create an account to join the conversation.

More
5 months 2 weeks ago #346379 by andrax
Replied by andrax on topic Ethercat random jitter fix
Thank you very much for all your hard work. I’ve been following this thread the whole time, and now that the changes are being incorporated into the official repository, I’ll update my repositories and test the new EtherCAT HAL driver.Note 1:From what I’ve read, there does appear to be a dependency between the software and hardware.Note 2:I would ask you to investigate whether Debian, the base thread and the servo thread can each be assigned to a single CPU core.This would further improve stability.This is also done by Beckhoff and Codesys.It makes their runtime independent of the operating system and guarantees real-time capability. 

Please Log in or Create an account to join the conversation.

More
5 months 2 weeks ago #346385 by rodw
Replied by rodw on topic Ethercat random jitter fix

Note 2:I would ask you to investigate whether Debian, the base thread and the servo thread can each be assigned to a single CPU core.This would further improve stability.This is also done by Beckhoff and Codesys.It makes their runtime independent of the operating system and guarantees real-time capability. 
Yes, I believe this is the correct approach but I've never run a base thread or looked at the thread instantiation code in detail. :)
With a single isolated core (normal case) both threads should end up there.

Ethercat and Mesa configs should not need a base thread

Please Log in or Create an account to join the conversation.

More
5 months 2 weeks ago #346389 by grandixximo
There is no need for base-thread on ethercat machines
base-thread should be set to 0, which will disable it, you can hook up everything to servo-thread, also the "no floating point" situation is completely stripped out form 2.10 master, and disabled on next 2.9 release, 2.9.9 whenever that ships...
The following user(s) said Thank You: rodw, NWE

Please Log in or Create an account to join the conversation.

Time to create page: 0.140 seconds
Powered by Kunena Forum