Bookworm on Armbian 7i76e errors

More
26 Feb 2023 02:35 #265379 by rodw
Its still a work in progress but we have been able to build a PREEMPT_RT kernel using Armbian for an Odroid N2+ to  try and get away from the error finishing read problem. This is a 6 core BIG.little CPU (2.4 Ghz and 2.0 Ghz respectively) so we are getting decent latency figues with isolcpus=2,3,4,5 (cores 4 & 5 are the little cores)
 
We are using kernel 6.2 and Debian Bookworm/Xfce and LInuxcnc 2.9 installed from the bookworm debs
Ping times are decent and even the first ping does not seem to be any slower
Ping:
odroid@odroidn2:~$ ping 10.10.10.1
PING 10.10.10.1 (10.10.10.1) 56(84) bytes of data.
64 bytes from 10.10.10.1: icmp_seq=1 ttl=64 time=0.075 ms
64 bytes from 10.10.10.1: icmp_seq=2 ttl=64 time=0.081 ms
64 bytes from 10.10.10.1: icmp_seq=3 ttl=64 time=0.081 ms
--- 10.10.10.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2032ms
rtt min/avg/max/mdev = 0.075/0.079/0.081/0.003 ms

Today I finally got it talking to a Mesa 7i76e  and noticed an odd error:
hm2/hm2_7i76e.0: Smart Serial port 0: DoIt not cleared from previous servo thread. Servo thread rate probably too fast. This message will not be repeated, but the hm2_7i76e.0.sserial.0.fault-count pin will indicate if this is happening frequently.

But when I look at the fault pin, there is no fault.
halcmd show pin hm2_7i76e.0.sserial.port-0.fault-count
Component Pins:
Owner Type Dir Value Name
36 u32 OUT 0x00000000 hm2_7i76e.0.sserial.port-0.fault-count

So the serial port error appears to be spurious and only happens on startup. Could this be a driver bug?
I also wonder if this could be caused by the load placed on the CPU when loading the QT5 based QTplasmac screen.
An explanation would be appreciated.
 
Attachments:

Please Log in or Create an account to join the conversation.

More
26 Feb 2023 02:53 - 26 Feb 2023 02:54 #265381 by PCW
The "DoIt not cleared" error does indicate a real time issue.

It means the last (write) transaction from the previous thread
did not complete before the next thread started.

In my experience the long communication delays
are mostly related to intensive video operations
or saturated CPUs.
Last edit: 26 Feb 2023 02:54 by PCW.

Please Log in or Create an account to join the conversation.

More
26 Feb 2023 02:58 #265382 by rodw
Thanks, what about the lack of erorr count?

Please Log in or Create an account to join the conversation.

More
26 Feb 2023 03:25 - 26 Feb 2023 04:00 #265383 by PCW
I think only fatal errors are registered (too many faults relative to good cycles)
A count could be added (just like the new packet-error-total)
Last edit: 26 Feb 2023 04:00 by PCW.

Please Log in or Create an account to join the conversation.

More
26 Feb 2023 03:48 #265384 by rodw
I played around changing isolcpus and watched CPU load  for each core in htop.
It seems the overhead from QT based GUI's sometimes saturate one RT CPU core briefly on startup.
One of the little RT cores sometimes goes to 100% utilisation briefly when opening QTplasmac.
Axis on the other hand only goes to 67% utilisation
Once QTplasmac opens, the CPU load on that core falls to a fairly constant 90-92%

But what I am observing, despite 3 cores being allocated to Real TIme with isolcpus, only one core is used by linuxcnc
I wonder if there is a way to put hm2-eth on a seperate thread on another core? That could be a sure fire way to fix error finishing read issues. 
I might try adding another thread and see where it ends up.

Maybe as the world gets mor complex, we should have the ability to specify a core to run a thread on...

Please Log in or Create an account to join the conversation.

More
26 Feb 2023 03:53 #265385 by rodw
hmm, I was using an existing config and one of the files in the ini file was incorrect which raised an error. Correcting the path seemed to have solved it. CPU core is at 90% max

But on the third attempt loading QTplasmac, that core has dropped to 2.5% utilisation. Very strange.......

Please Log in or Create an account to join the conversation.

Moderators: PCWjmelson
Time to create page: 0.159 seconds
Powered by Kunena Forum