How We Test Linux
Download as PPTX, PDF•2 likes•905 views
The document outlines the various methodologies for testing Linux, including a discussion on the architecture, frameworks, and tools used for testing. Key components include kernel self-tests, the Linux Test Project, and the use of benchmarks such as lmbench. Additionally, it highlights the structure of test cases and continuous integration processes involving Jenkins.
How We Test Linux
- 1. 1 How we Test Linux Dmytro Tokarskyy 2018-06-04
- 2. 2 Agenda 1. What Linux is for us 2. Frameworks and Tools 3. Kernel Continuous Integration 4. "Leg" Tests
- 3. 33 What Linux is for us
- 4. 4 Linux Architecture Hardware Desktop Environment Android System Libraries System Libraries FS USB Network UART ...
- 7. 7 How we get Linux Linux Mainline Source Tree Board Vendor Linux Source Tree Customer Linux Source Tree Target Source TreePorting
- 8. 8 Expectation from Testing Improvements / Degradation Functional Tests Benchmarks
- 10. 10 Kernel Self Tests • Started and 2014 by Shuah Khan, Linux Kernel Developer at Samsung Research America • Set of tests included to the Linux Kernel source tree • A set of developer unit and regression tests • Tests exercise individual code paths in the kernel • Own test harnesses • Test results are reported in the TAP13 format • Links - tools/testing/selftests/ - https://kselftest.wiki.kernel.org/ - https://lwn.net/Articles/608959/
- 11. 11 Kernel Self Test Cases • Function Tracer (ftrace) • Linux Sockets • Virtual Memory • Timers
- 12. 12 Linux Test Project • The Linux Test Project is a joint project started by SGI, developed and maintained by IBM, Cisco, Fujitsu, SUSE, Red Hat and others • A goal to deliver test suites to validate the reliability, robustness, and stability of Linux. • The LTP testsuite contains a collection of tools for testing the Linux kernel and related features.
- 13. 13 Linux Test Project Coverage • Collection of tools for testing the Linux kernel and related features - Kernel-related tests such as filesystems, io, ipc, memory management, scheduler, and system calls - Network tests including tests for ipv6, multicast, nfs, rpc, sctp, and network related user commands - Tests for user level commands commonly used in application development such as ar, ld, ldd, nm, objdump, and size - Miscellaneous tests that do not fit into one of the other categories such as crash, f00f, and a floating point math set of tests
- 14. 14 Linux Test Project Test Case • “Classical” structure of tests - set up - test themself - tear down • Four results of tests - TPASS - Test has passed - TFAIL - Test has failed - TCONF - Test is not appropriate for current configuration - TBROK - Something has failed in test preparation phase • Source code - https://github.com/linux-test-project/ltp
- 15. 15 LMBench lmbench is a suite of simple, portable, ANSI/C microbenchmarks for UNIX/POSIX. ● Bandwidth benchmarks ○ Cached file read ○ Memory copy (bcopy) ○ Memory read ● Latency benchmarks ○ Context switching ○ Process creation ○ Signal handling ○ System call overhead ○ Memory read latency ● Miscellaneous ○ Processor clock rate calculation
- 17. 17 Jenkins Pipeline Get Tools Download Kernel Build Kernel Pack Rootfs Deploy SelfTest LTP LMBench
- 19. 19 Jenkins Pipeline Stage View
- 20. 20 Test Station Scheme • Dedicated server runs Jenkins CI • MRB’s rootfs placed to NFS at another server • Jenkins controls MRBs, NFS via SSH/SCP • USB controlled 4-channel relay “presses” an ignition button by CI command
- 22. 2222 "Leg" Tests
- 23. 23 Apple CarPlay
- 24. 24 Kernel Stress Testing: USB Stack
- 25. 25 Kernel Stress Testing: SATA Interface
- 26. 26 ALSA Sound
- 27. 27 Graphics
- 28. 28 Watchdog
- 29. 29 Something More... • Functional Tests - Bootloader - UART - Power Management - File Systems - Ethernet • Benchmarks - Iperf: maximum achievable bandwidth on IP networks - IOZone: file system benchmark utility - Glmark2: OpenGL 2.0 and ES 2.0 benchmark