Script for automated testing of computer latency.

More
16 Oct 2020 08:54 #186263 by seuchato
Arrrgh!
lsblk --version
lsblk from util-linux 2.33.1
Got the problem now. I develop on OpenSuse, then test on debian LnixCNC iso.
Thanks a ton, need to find another way to determine which local disc has most free space.

Greez
chris

Please Log in or Create an account to join the conversation.

More
16 Oct 2020 09:24 - 16 Oct 2020 09:25 #186264 by BeagleBrainz
df -l -x tmpfs | grep -v udev
df (GNU coreutils) 8.28
Last edit: 16 Oct 2020 09:25 by BeagleBrainz.
The following user(s) said Thank You: seuchato

Please Log in or Create an account to join the conversation.

More
16 Oct 2020 09:37 #186269 by tommylight
Separate home:
cnc@SSD240HIK:~$ mount | grep -Ei "on /home"
/dev/sda2 on /home type ext4 (rw,relatime,data=ordered)
cnc@SSD240HIK:~$
cnc@SSD240HIK:~$ lsblk -b -o MOUNTPOINT,FSAVAIL -x FSAVAIL | grep "/" | awk '!seen[$0]++' | sort -r -n -k2 | sed -n 1p
/home 143709351936
cnc@SSD240HIK:~$
The following user(s) said Thank You: seuchato

Please Log in or Create an account to join the conversation.

More
16 Oct 2020 17:05 - 16 Oct 2020 17:07 #186297 by seuchato
BB, tommy and who else might be interested:

df -l -x tmpfs | grep -v udev | sort -r -n -k4 | sed -n 1p

Sample output of an OpenSuse PC:

df -l -x tmpfs | grep -v udev
Filesystem     1K-blocks    Used Available Use% Mounted on
devtmpfs         1947284       8   1947276   1% /dev
/dev/sda2       69905408 6411708  62747300  10% /
/dev/sda2       69905408 6411708  62747300  10% /tmp
/dev/sda2       69905408 6411708  62747300  10% /srv
/dev/sda2       69905408 6411708  62747300  10% /opt
/dev/sda2       69905408 6411708  62747300  10% /boot/grub2/i386-pc
/dev/sda2       69905408 6411708  62747300  10% /.snapshots
/dev/sda2       69905408 6411708  62747300  10% /usr/local
/dev/sda2       69905408 6411708  62747300  10% /boot/grub2/x86_64-efi
/dev/sda2       69905408 6411708  62747300  10% /var
/dev/sda2       69905408 6411708  62747300  10% /root
/dev/sda6      108299608  331460 107968148   1% /home
localhost:~ # df -l -x tmpfs | grep -v udev | sort -r -n -k4
/dev/sda6      108299608  331420 107968188   1% /home
/dev/sda2       69905408 6411728  62747296  10% /var
/dev/sda2       69905408 6411728  62747296  10% /usr/local
/dev/sda2       69905408 6411728  62747296  10% /tmp
/dev/sda2       69905408 6411728  62747296  10% /srv
/dev/sda2       69905408 6411728  62747296  10% /root
/dev/sda2       69905408 6411728  62747296  10% /opt
/dev/sda2       69905408 6411728  62747296  10% /boot/grub2/x86_64-efi
/dev/sda2       69905408 6411728  62747296  10% /boot/grub2/i386-pc
/dev/sda2       69905408 6411728  62747296  10% /.snapshots
/dev/sda2       69905408 6411728  62747296  10% /
devtmpfs         1947284       8   1947276   1% /dev
Filesystem     1K-blocks    Used Available Use% Mounted on
localhost:~ # df -l -x tmpfs | grep -v udev | sort -r -n -k4 | sed -n 1p
/dev/sda6      108299608  331420 107968188   1% /home

This should return the block device that has most free space. Can you confirm this?

Greez
chris
Last edit: 16 Oct 2020 17:07 by seuchato. Reason: added sample output

Please Log in or Create an account to join the conversation.

More
16 Oct 2020 17:51 #186298 by tommylight
cnc@SSD240HIK:~$ df -l -x tmpfs | grep -v udev | sort -r -n -k4 | sed -n 1p
/dev/sda2 200810752 52103456 138436988 28% /home
cnc@SSD240HIK:~$
The following user(s) said Thank You: seuchato

Please Log in or Create an account to join the conversation.

More
19 Oct 2020 23:57 #186635 by seuchato
Hi all

This is work in progress.

In order to provide a suitable I/O load, I plan to include a cleaned up version of the attached scriptled "test2" in lcnc-hw.

It does the following:
  • determines the block device with most free space on it (thanks BeagleBrainz)
  • calculates largest file sizes for two (intention was to use identical sizes) files (at least for now)(thanks to tommylight for pointing me to dd)
    • 1st file with a large block size
    • 2nd file with a small block size of only 512 bytes. That proves to be slow, very slow indeed, if compared to the large block size
  • reads from /dev/urandom into two files
  • copies/moves the file generated
  • deletes the files
  • integrated in lcnc-hw, it will loop over above steps until
    • real time delay occurs
    • Ctrl C is pressed
    • Time limit is achieved

Sample output:

./test2
941921040 (size available on device)
 Blocksize: 1M, count: 898
923795456 bytes (924 MB, 881 MiB) copied, 44 s, 21.0 MB/s
898+0 Datensätze ein
898+0 Datensätze aus
941621248 bytes (942 MB, 898 MiB) copied, 44.9813 s, 20.9 MB/s
done /home/1603149946-1.tmp
250880 bytes (251 kB, 245 KiB) copied, 13 s, 19.3 kB/s
500+0 Datensätze ein
500+0 Datensätze aus
256000 bytes (256 kB, 250 KiB) copied, 13.4213 s, 19.1 kB/s
done /home/1603149946-2.tmp

What I'd like to know:
  • Does this represent a suitable model to simulate I/O load?
  • What size is the free Ram for instance on an RPi with say only 2 GB Ram? (run "free" in a terminal while running attached script AND lcnc-hw)
  • Can you see an effect on latency when running "test2"

A word of caution: I fried an intel 128 G NVMe ssd when starting "test2" looping in two terminals without a pause. Nothing happened to the sata based ssd I use now. I guess NVMe disc should be equipped with cooling when loaded.

Greez
chris
Attachments:

Please Log in or Create an account to join the conversation.

More
01 Nov 2020 03:07 #187967 by BeagleBrainz
After reading this post forum.linuxcnc.org/18-computer/39371-res...cnc?start=170#187963 and seeing the unusable figures I think it may be an idea to check if the kernel is a RT kernel. Old Mate has booted into a "Generic" kernel without noticing.
The following user(s) said Thank You: seuchato

Please Log in or Create an account to join the conversation.

More
01 Nov 2020 03:24 #187972 by seuchato
I will add a test on RT/RTAI. If it does not find such a kernel, it will post a warning and not do the very latency test at all.

Please Log in or Create an account to join the conversation.

More
03 Nov 2020 22:29 - 03 Nov 2020 22:32 #188275 by seuchato
Hi all

Took a while, but new version, out soon:

#-----------------------------------------------------------------#
# Todo: (o = open, x = cancelled at least for the time being)     
# x skip the .lat file by default but leave an option to have it. 
#   If, a graphical output will no longer be possible.            
# o add a graphical output (one pic = 1000 words)                 
# o make sure "abort" in UserOptions now really aborts            
# o add data on nics for mesa ethernet/hostmot                    
# o rework mouse keyboard related data, as output is not always   
#   correct                                                       
#-----------------------------------------------------------------
#                                                                 
# V20201104pre                                                    
# . option to just get the hw data and skip latency test (-info)  
# . corrected basebin setting if basethread is selected manually  
# . added info on Uefi or bios boot                               
# . more detailed output for RAM data                             
# . better output regarding blockdevices                          
# . added script to block pcspkr                                  
# . added write permission check for user to run the very         
#   latency test                                                  
# . added check on *rt* kernel with a warning and prevent         
#   the very latency test from running                           



# . better and more compact data on RAM                           
# o added disc io, CPU and RAM load                               
# . indent all secondary lines to make reading better              


Regarding disc io, CPU and RAM load :
I dug deep into the internet. Came out that large block reads from /dev/urandom into a temporary file provided the largest CPU and Ram load. on a xeon W3635 system with 8 GiBs Ram I got to -p <pidoftask> CPU load is jumping from little above 10% to a full 100%, where must of the time it was above 50%.

The attached "stresstest" can be run alone. as root or sudoed. I have yet to find out a way to weave the subroutine "DiscLoad" into the script in such way, that it terminates gracefully on CTRL-C or script end. Imho CPULoad and RAMLoad subroutines, as well as the low blocksize reads in DiscLoad (proposed 512 bytes) can be put aside. There is now new info to be gained from them.

I'd like to kindly ask the community to check "stresstest" and report what is their conclusion. Meanwhile, I'll continue to integrate "DiscLoad" into the script.

Tia
chris
Attachments:
Last edit: 03 Nov 2020 22:32 by seuchato. Reason: added stresstest.zip
The following user(s) said Thank You: tommylight, my1987toyota

Please Log in or Create an account to join the conversation.

More
17 Nov 2020 00:56 - 17 Nov 2020 00:58 #189567 by my1987toyota
well hopefully this is what you were looking for sorry for taking so long life has a bad habit of taking my plans and throwing them in the trash
cnc001@debian:~$ sudo sensors-detect
[sudo] password for cnc001: 
Sorry, try again.
[sudo] password for cnc001: 
# sensors-detect revision $Revision$
# System: LENOVO H215 [IdeaCentre]
# Board: LENOVO Tilapia CRB
# Kernel: 4.19.0-11-rt-amd64 x86_64
# Processor: AMD Athlon(tm) II X2 250 Processor (16/6/3)

This program will help you determine which kernel modules you need
to load to use lm_sensors most effectively. It is generally safe
and recommended to accept the default answers to all questions,
unless you know what you're doing.

Some south bridges, CPUs or memory controllers contain embedded sensors.
Do you want to scan for them? This is totally safe. (YES/no): y
Module cpuid loaded successfully.
Silicon Integrated Systems SIS5595...                       No
VIA VT82C686 Integrated Sensors...                          No
VIA VT8231 Integrated Sensors...                            No
AMD K8 thermal sensors...                                   No
AMD Family 10h thermal sensors...                           Success!
    (driver `k10temp')
AMD Family 11h thermal sensors...                           No
AMD Family 12h and 14h thermal sensors...                   No
AMD Family 15h thermal sensors...                           No
AMD Family 16h thermal sensors...                           No
AMD Family 17h thermal sensors...                           No
AMD Family 15h power sensors...                             No
AMD Family 16h power sensors...                             No
Intel digital thermal sensor...                             No
Intel AMB FB-DIMM thermal sensor...                         No
Intel 5500/5520/X58 thermal sensor...                       No
VIA C7 thermal sensor...                                    No
VIA Nano thermal sensor...                                  No

Some Super I/O chips contain embedded sensors. We have to write to
standard I/O ports to probe them. This is usually safe.
Do you want to scan for Super I/O sensors? (YES/no): y
Probing for Super-I/O at 0x2e/0x2f
Trying family `National Semiconductor/ITE'...               No
Trying family `SMSC'...                                     No
Trying family `VIA/Winbond/Nuvoton/Fintek'...               No
Trying family `ITE'...                                      Yes
Found `ITE IT8720F Super IO Sensors'                        Success!
    (address 0xe80, driver `it87')
Probing for Super-I/O at 0x4e/0x4f
Trying family `National Semiconductor/ITE'...               No
Trying family `SMSC'...                                     No
Trying family `VIA/Winbond/Nuvoton/Fintek'...               No
Trying family `ITE'...                                      No

Some systems (mainly servers) implement IPMI, a set of common interfaces
through which system health data may be retrieved, amongst other things.
We first try to get the information from SMBIOS. If we don't find it
there, we have to read from arbitrary I/O ports to probe for such
interfaces. This is normally safe. Do you want to scan for IPMI
interfaces? (YES/no): y
Probing for `IPMI BMC KCS' at 0xca0...                      No
Probing for `IPMI BMC SMIC' at 0xca8...                     No

Some hardware monitoring chips are accessible through the ISA I/O ports.
We have to write to arbitrary I/O ports to probe them. This is usually
safe though. Yes, you do have ISA I/O ports even if you do not have any
ISA slots! Do you want to scan the ISA I/O ports? (yes/NO): y
Probing for `National Semiconductor LM78' at 0x290...       No
Probing for `National Semiconductor LM79' at 0x290...       No
Probing for `Winbond W83781D' at 0x290...                   No
Probing for `Winbond W83782D' at 0x290...                   No

Lastly, we can probe the I2C/SMBus adapters for connected hardware
monitoring devices. This is the most risky part, and while it works
reasonably well on most systems, it has been reported to cause trouble
on some systems.
Do you want to probe the I2C/SMBus adapters now? (YES/no): y
Using driver `i2c-piix4' for device 0000:00:14.0: ATI Technologies Inc SB600/SB700/SB800 SMBus
Module i2c-dev loaded successfully.

Next adapter: SMBus PIIX4 adapter at 0b00 (i2c-0)
Do you want to scan it? (yes/NO/selectively): y
Client found at address 0x50
Probing for `Analog Devices ADM1033'...                     No
Probing for `Analog Devices ADM1034'...                     No
Probing for `SPD EEPROM'...                                 Yes
    (confidence 8, not a hardware monitoring chip)
Probing for `EDID EEPROM'...                                No
Client found at address 0x51
Probing for `Analog Devices ADM1033'...                     No
Probing for `Analog Devices ADM1034'...                     No
Probing for `SPD EEPROM'...                                 Yes
    (confidence 8, not a hardware monitoring chip)

Next adapter: SMBus PIIX4 adapter at 0b20 (i2c-1)
Do you want to scan it? (yes/NO/selectively): y

Next adapter: Radeon i2c bit bus 0x90 (i2c-2)
Do you want to scan it? (yes/NO/selectively): y
Client found at address 0x4a
Probing for `National Semiconductor LM75'...                No
Probing for `National Semiconductor LM75A'...               No
Probing for `Dallas Semiconductor DS75'...                  No
Probing for `National Semiconductor LM77'...                No
Probing for `Analog Devices ADT7410/ADT7420'...             No
Probing for `Analog Devices ADT7411'...                     No
Probing for `Maxim MAX6642'...                              No
Probing for `Texas Instruments TMP435'...                   No
Probing for `National Semiconductor LM73'...                No
Probing for `National Semiconductor LM92'...                No
Probing for `National Semiconductor LM76'...                No
Probing for `Maxim MAX6633/MAX6634/MAX6635'...              No
Probing for `NXP/Philips SA56004'...                        No
Client found at address 0x4b
Probing for `National Semiconductor LM75'...                No
Probing for `National Semiconductor LM75A'...               No
Probing for `Dallas Semiconductor DS75'...                  No
Probing for `National Semiconductor LM77'...                No
Probing for `Analog Devices ADT7410/ADT7420'...             No
Probing for `Analog Devices ADT7411'...                     No
Probing for `Maxim MAX6642'...                              No
Probing for `Texas Instruments TMP435'...                   No
Probing for `National Semiconductor LM92'...                No
Probing for `National Semiconductor LM76'...                No
Probing for `Maxim MAX6633/MAX6634/MAX6635'...              No
Probing for `NXP/Philips SA56004'...                        No
Probing for `Analog Devices ADT7481'...                     No

Next adapter: Radeon i2c bit bus 0x91 (i2c-3)
Do you want to scan it? (yes/NO/selectively): y

Next adapter: Radeon i2c bit bus 0x92 (i2c-4)
Do you want to scan it? (yes/NO/selectively): y

Next adapter: Radeon i2c bit bus 0x93 (i2c-5)
Do you want to scan it? (yes/NO/selectively): y
y

Next adapter: Radeon i2c bit bus 0x14 (i2c-6)
Do you want to scan it? (yes/NO/selectively): 

Now follows a summary of the probes I have just done.
Just press ENTER to continue: 

Driver `k10temp' (autoloaded):
  * Chip `AMD Family 10h thermal sensors' (confidence: 9)

Driver `it87':
  * ISA bus, address 0xe80
    Chip `ITE IT8720F Super IO Sensors' (confidence: 9)

To load everything that is needed, add this to /etc/modules:
#----cut here----
# Chip drivers
it87
#----cut here----
If you have some drivers built into your kernel, the list above will
contain too many modules. Skip the appropriate ones!

Do you want to add these lines automatically to /etc/modules? (yes/NO)y
Successful!

Monitoring programs won't work until the needed modules are
loaded. You may want to run '/etc/init.d/kmod start'
to load them.

Unloading i2c-dev... OK
Unloading cpuid... OK
Last edit: 17 Nov 2020 00:58 by my1987toyota.
The following user(s) said Thank You: seuchato

Please Log in or Create an account to join the conversation.

Time to create page: 0.456 seconds
Powered by Kunena Forum