SlideShare a Scribd company logo
December 8-10, 2020
Virtual Event
Ivannikov Institute for System Programming
Russian Academy of Sciences
Nikita Chertok, Mikhail Chupilko, Alexander Kamkin
Online Test Program Generator for
RISC-V Microprocessors
MicroTESK Framework
MicroTESK Framework
Specifications
(nML)
Specifications
Translator
Test Programs
(ASM)
Test Program
Generator
Binary Code
Analyzer
Tool
Builders
Processor
Model
Coverage
Simulator
Metadata
lui a0,0xdead
ori a0,a0,0x0
lui a1,0xbeef
ori a1,a1,0xf
add t0,a0,a1
Simulator
(C/C++)
Specifications
(WhyML)
…
Ref. Simulator
(QEMU)
Specifications
Debug
1
https://forge.ispras.ru/projects/microtesk
op add (rd: X, rs1: X, rs2: X)
syntax = format("add %s, %s, %s",
rd.syntax, rs1.syntax, rs2.syntax)
image = format("0000000%s%s000%s0110011",
rs2.image, rs1.image, rd.image)
action = {
rd = rs1 + rs2;
}
mode X (i: card(5)) = XREG[i]
syntax = format("%s",
if i==0 then ZERO().syntax
elif i==1 then RA().syntax
elif i==2 then SP().syntax
elif i==3 then GP().syntax
elif i==4 then TP().syntax
elif i>=5 && i<=7 then
Temp(coerce(card(3), i-5)).syntax
elif i>=8 && i<=9 then
Saved(coerce(card(4), i-8)).syntax
elif i>=10 && i<=17 then
Func(coerce(card(3), i-10)).syntax
elif i>=18 && i<=27 then
Saved(coerce(card(4), i-16)).syntax
else
Temp(coerce(card(3), i-25)).syntax
image = format("%5s", i)
nML Language Overview
2
#ifdef RV64I
let XLEN = 64
...
#else
...
#endif
let MEMORY_SIZE_IN_WORDS = 2 ** (XLEN – 2)
shared mem MEM[MEMORY_SIZE_IN_WORDS, WORD]
type WORD = card(32)
type XWORD = card(XLEN)
type FLOAT32 = float(23, 8)
reg XREG [32, XWORD]
reg PC [XWORD]
Specifications
3
Specifications Characteristic Value
RISC-V Instruction Set Manual Vol. I: User-Level ISA (v. 2.2) 145 pages (94 pages in chapters 2-18)
Base Extension Base Extension In Total
RV32I M F C P
24 pages 44 pages 68 pages
RV32E A D B V
RV64I Q J N
2 pages 24 pages 26 pages
RV128I L T
# Specified Instructions 230 instructions
ISA Specifications Size (nML) 4 900 LOC (w/o comments)
MMU Specification Size (MMUSL) 460 LOC (Sv32, Sv39, and Sv48)
Offline Test Program Generation
4
Design Under Test
(RTL, FPGA)
lui a0, 0xdead
ori a0, a0, 0x0
lui a1, 0xbeef
ori a1, a1, 0xf
add t0, a0, a1
sub t1, a0, t1
add t0, t0, t1
Test Programs
(ASM)
Reference Simulator
(C/C++)
Traces (Tarmac) /
Signatures
0x2000: lui ...
0x2004: ori ...
0x2008: lui ...
0x200c: ori ...
0x2010: add ...
0x2014: sub ...
0x2018: add ...
Comparator
(Python)
0x2000: lui ...
0x2004: ori ...
0x2008: lui ...
0x200c: ori ...
0x2010: add ...
0x2014: sub ...
0x2018: bug ...
Traces (Tarmac) /
Signatures
What’s Inside?
?
5
MicroTESK Test Program Generator (Offline)
MicroTESK Test Program Generator
Specifications (nML)
Translator
Test Templates (Ruby)
lui a0, 0xdead
ori a0, a0, 0x0
lui a1, 0xbeef
ori a1, a1, 0xf
add t0, a0, a1
sub t1, a0, t1
add t0, t0, t1
Test Programs
(Assembly Code)
Processor Model
Generation Core
Specification Engineer
Verification Engineer
6
Autogen
https://forge.ispras.ru/projects/microtesk-riscv
Test Templates (Offline)
 Test templates similar to RISC-V Foundation’s Tests
https://github.com/riscv/riscv-tests
 Test templates similar to RISC-V Torture Test Generator
https://github.com/ucb-bar/riscv-torture 7
class MyTemplate < RiscVBaseTemplate
def run
block(:combinator => 'product',
:compositor => 'random') {
iterate {
xor x(_), x(_), x(_) do situation(...) end
lui x(_), _
}
iterate {
and x(_), x(_), x(_)
or x(_), x(_), x(_)
}
}.run
end
end
# Single Test Case
# Initialization
ori a7, a7, 0x2d7
slli a7, a7, 0xb
ori a7, a7, 0x1
slli a7, a7, 0xb
ori a7, a7, 0x3d2
ori t3, t3, 0x164
slli t3, t3, 0xb
ori t3, t3, 0x52b
slli t3, t3, 0xb
ori t3, t3, 0x24e
# Stimulus
and s4, a7, a7
xor s8, s4, t3
2 × 2 = 4 test cases
• Combinatorial Brute Force
• Randomization
• Constraint Solving
• Self Checking
Code is distributed under
Apache License, v2.0
Online Test Generation
Offline Test Generation
lui a0, 0xdead
ori a0, a0, 0x0
lui a1, 0xbeef
ori a1, a1, 0xf
add t0, a0, a1
sub t1, a0, t1
add t0, t0, t1
Test Program Generator
(Standalone Software)
Test Programs
(Binaries)
Design Under Test
(RTL, FPGA, IC)
DUT executes loaded
test programs
lui a0, 0xdead
ori a0, a0, 0x0
lui a1, 0xbeef
ori a1, a1, 0xf
add t0, a0, a1
sub t1, a0, t1
add t0, t0, t1
Test Program Generator
(Binary Inside DUT)
Test Programs
(Binaries)
Design Under Test
(FPGA, IC)
DUT executes
test generator
that produces
test programs
Tests programs are
loaded to memory
8
Online TPG Requirements
 Autonomy
 Self-generation of a variety of tests from a template
 Self-checking of tests results
 Performance
 Test massiveness prevails over test directedness
 Simple techniques (brute force, randomization, mutation, etc.)
 Robustness
 No loss of control (exception handling, timers, etc.)
 Saving and restoring generator state
9
Online TPG Architecture
Control
Unit
Allocator
Executor
Test Oracle
add …
sub …
...
add …
nop
sub …
Generator
Transformer
Residential Part Test Image
External Part
Control
Console
Generator
Commands
Parameters
Statistics
Coverage
Failures
.text
110011010100000…
110011011100000…
...
.data
1101111010101101
.text
110011010100000…
110010000000000…
110011011100000…
.data
1101111010101101
Test w/ mutant (QED)
Verdict
.data
1101111010101101
1011111011101111
.data
1101111010101101
1011101011101111
Test Results
Test Code
Allocation, linking, etc.
Signatures
Metadata
Allocates code in memory
Gets signatures and compares them
Requests next test case
Transfers control to
tests and returns back
Deviation
10
MicroTESK Test Program Generator (Online)
MicroTESK Test Program Generator
Specifications (nML)
Translator
Test Templates (Ruby)
Test Generator
(Code in C)
Processor Model
Generation Core
Specification Engineer
Verification Engineer
11
Autogen
https://forge.ispras.ru/projects/microtesk-riscv
Generator
Transformer
Test Templates (Online)
 Test template is translated to C language:
combinators, compositors, preparators, etc.
 Generator in C implements a certain interface:
can be obtained by other means (e.g., Autogen)
class MyTemplate < RiscVBaseTemplate
def run
block(:combinator => 'product',
:compositor => 'random') {
iterate {
xor x(_), x(_), x(_) do ... end
lui x(_), _
}
iterate {
and x(_), x(_), x(_)
or x(_), x(_), x(_)
}
}.run
end
end
// MyTemplate (simplified)
OP *iterate_0[] = { {XOR}, {LUI} };
OP *iterate_1[] = { {AND}, {OR} };
// :combinator => 'product'
for (i[0] = 0; i[0] < 2; i[0]++)
for (i[1] = 0; i[1] < 2; i[1]++) {
OP *combine[] = tpg_malloc(...)
combine[0] = iterate_0[i[0]];
combine[1] = iterate_1[i[1]];
// :compositor => 'random'
OP compose[] = tpg_malloc(...);
k = j[0] = j[1] = 0;
while (j[0] < 1 && j[1] < 1) {
while (j[s = rand() % 2] == 2);
compose[k++] = combine[s][j[s]++];
}
// choose registers, generate data,
// insert initialization code, etc.
...
// get the test’s binary image
for (int n = 0; n < N; n++) {
binary[n] = test[n].image();
}
} 12
Online TPG Features
 Test generation techniques
 Combinatorial brute-force generation (“perebor”)
 Randomization
 No constraint solving!
 Test result checking techniques
 Precalculated signatures returned by generator
 Equivalent (modulo inputs) transformations
 Dead code injection (NOPs between instructions, etc.)
 Quick error detection (QED)
13
Future Work Directions
 RISC-V specifications
 Specifying more subsets (RV128I, Q, L, B, T, J, P, V, and N)
 Online test program generation
 Error localization and diagnostics
 Test coverage tracking and test completeness assessment
 Advanced test program generation
 Model-based generation (modeling compiler transformations)
 Mutations and coverage-directed fuzzing
 Binary code equivalence checking
 Verifying optimizing compilers (including JIT and AOT) 14
http://www.microtesk.org
microtesk-support@ispras.ru
Thank
You!

More Related Content

What's hot (20)

PDF
Board Support Package Fact Sheet | Manual Guide
Embitel Technologies - A VOLKSWAGEN GROUP COMPANY
 
PDF
Pci express modi
proma_goswami
 
PDF
RISC-V Online Tutor
RISC-V International
 
PDF
LCU13: An Introduction to ARM Trusted Firmware
Linaro
 
PDF
LAS16 111 - Raspberry pi3, op-tee and jtag debugging
96Boards
 
PDF
UVM Methodology Tutorial
Arrow Devices
 
PPT
I2C And SPI Part-23
Techvilla
 
PDF
Session 8,9 PCI Express
Subhash Iyer
 
PDF
Cracking Digital VLSI Verification Interview: Interview Success
Ramdas Mozhikunnath
 
PPTX
SOC Verification using SystemVerilog
Ramdas Mozhikunnath
 
PPTX
Static partitioning virtualization on RISC-V
RISC-V International
 
PPT
SPI Bus Protocol
Sudhanshu Janwadkar
 
PPTX
Introduction about APB Protocol
Pushpa Yakkala
 
DOCX
Intellectual property in vlsi
Saransh Choudhary
 
PDF
How to create SystemVerilog verification environment?
Sameh El-Ashry
 
PPTX
GCC for ARMv8 Aarch64
Yi-Hsiu Hsu
 
ODP
Introduction to Optee (26 may 2016)
Yannick Gicquel
 
PDF
FIFO Design
Arrow Devices
 
PDF
Trusted firmware deep_dive_v1.0_
Linaro
 
PPTX
AXI Protocol.pptx
Yazan Yousef
 
Board Support Package Fact Sheet | Manual Guide
Embitel Technologies - A VOLKSWAGEN GROUP COMPANY
 
Pci express modi
proma_goswami
 
RISC-V Online Tutor
RISC-V International
 
LCU13: An Introduction to ARM Trusted Firmware
Linaro
 
LAS16 111 - Raspberry pi3, op-tee and jtag debugging
96Boards
 
UVM Methodology Tutorial
Arrow Devices
 
I2C And SPI Part-23
Techvilla
 
Session 8,9 PCI Express
Subhash Iyer
 
Cracking Digital VLSI Verification Interview: Interview Success
Ramdas Mozhikunnath
 
SOC Verification using SystemVerilog
Ramdas Mozhikunnath
 
Static partitioning virtualization on RISC-V
RISC-V International
 
SPI Bus Protocol
Sudhanshu Janwadkar
 
Introduction about APB Protocol
Pushpa Yakkala
 
Intellectual property in vlsi
Saransh Choudhary
 
How to create SystemVerilog verification environment?
Sameh El-Ashry
 
GCC for ARMv8 Aarch64
Yi-Hsiu Hsu
 
Introduction to Optee (26 may 2016)
Yannick Gicquel
 
FIFO Design
Arrow Devices
 
Trusted firmware deep_dive_v1.0_
Linaro
 
AXI Protocol.pptx
Yazan Yousef
 

Similar to Online test program generator for RISC-V processors (20)

PDF
Shantanu's Resume
Shantanu Telharkar
 
PDF
PT-4054, "OpenCL™ Accelerated Compute Libraries" by John Melonakos
AMD Developer Central
 
PDF
Architecture for Massively Parallel HDL Simulations
DVClub
 
PDF
main
Ian Masliah
 
PDF
Fpga computing
rinnocente
 
PDF
Shantanu's Resume
Shantanu Telharkar
 
PPT
Virtual platform
sean chen
 
PDF
Unit testing on embedded target with C++Test
Engineering Software Lab
 
PDF
Abstracting Vector Architectures in Library Generators: Case Study Convolutio...
ETH Zurich
 
PPT
Stephan berg track f
Alona Gradman
 
PDF
Reconfigurable Model for RISC Processors
Thiago Lima
 
PPTX
Is Advanced Verification for FPGA based Logic needed
chiportal
 
PPTX
Test pattern Generation for 4:1 MUX
UrmilasSrinivasan
 
PPTX
Crossing the Boundaries: Development Strategies for (P)SoCs
Andreas Koschak
 
PDF
Practical RISC-V Random Test Generation using Constraint Programming
ed271828
 
PDF
digitaldesign-s20-lecture3b-fpga-afterlecture.pdf
Duy-Hieu Bui
 
PDF
Pharo Virtual Machine: News from the Front
ESUG
 
PDF
Auto Tuning
Hemanth Kumar Mantri
 
PPT
SystemVerilog-20041201165354.ppt
ravi446393
 
PDF
Can FPGAs Compete with GPUs?
inside-BigData.com
 
Shantanu's Resume
Shantanu Telharkar
 
PT-4054, "OpenCL™ Accelerated Compute Libraries" by John Melonakos
AMD Developer Central
 
Architecture for Massively Parallel HDL Simulations
DVClub
 
Fpga computing
rinnocente
 
Shantanu's Resume
Shantanu Telharkar
 
Virtual platform
sean chen
 
Unit testing on embedded target with C++Test
Engineering Software Lab
 
Abstracting Vector Architectures in Library Generators: Case Study Convolutio...
ETH Zurich
 
Stephan berg track f
Alona Gradman
 
Reconfigurable Model for RISC Processors
Thiago Lima
 
Is Advanced Verification for FPGA based Logic needed
chiportal
 
Test pattern Generation for 4:1 MUX
UrmilasSrinivasan
 
Crossing the Boundaries: Development Strategies for (P)SoCs
Andreas Koschak
 
Practical RISC-V Random Test Generation using Constraint Programming
ed271828
 
digitaldesign-s20-lecture3b-fpga-afterlecture.pdf
Duy-Hieu Bui
 
Pharo Virtual Machine: News from the Front
ESUG
 
SystemVerilog-20041201165354.ppt
ravi446393
 
Can FPGAs Compete with GPUs?
inside-BigData.com
 
Ad

More from RISC-V International (20)

PDF
WD RISC-V inliner work effort
RISC-V International
 
PDF
RISC-V Zce Extension
RISC-V International
 
PPTX
London Open Source Meetup for RISC-V
RISC-V International
 
PPTX
Ziptillion boosting RISC-V with an efficient and os transparent memory comp...
RISC-V International
 
PDF
Standardizing the tee with global platform and RISC-V
RISC-V International
 
PDF
Semi dynamics high bandwidth vector capable RISC-V cores
RISC-V International
 
PPTX
Security and functional safety
RISC-V International
 
PPTX
Reverse Engineering of Rocket Chip
RISC-V International
 
PPTX
RISC-V NOEL-V - A new high performance RISC-V Processor Family
RISC-V International
 
PPTX
RISC-V 30910 kassem_ summit 2020 - so_c_gen
RISC-V International
 
PDF
RISC-V 30908 patra
RISC-V International
 
PPTX
RISC-V 30907 summit 2020 joint picocom_mentor
RISC-V International
 
PPTX
RISC-V 30906 hex five multi_zone iot firmware
RISC-V International
 
PPTX
RISC-V 30946 manuel_offenberg_v3_notes
RISC-V International
 
PDF
RISC-V software state of the union
RISC-V International
 
PDF
Ripes tracking computer architecture throught visual and interactive simula...
RISC-V International
 
PPTX
Porting tock to open titan
RISC-V International
 
PPTX
Open j9 jdk on RISC-V
RISC-V International
 
PDF
Open source manufacturable pdk for sky water 130nm process node
RISC-V International
 
PPTX
Klessydra t - designing vector coprocessors for multi-threaded edge-computing...
RISC-V International
 
WD RISC-V inliner work effort
RISC-V International
 
RISC-V Zce Extension
RISC-V International
 
London Open Source Meetup for RISC-V
RISC-V International
 
Ziptillion boosting RISC-V with an efficient and os transparent memory comp...
RISC-V International
 
Standardizing the tee with global platform and RISC-V
RISC-V International
 
Semi dynamics high bandwidth vector capable RISC-V cores
RISC-V International
 
Security and functional safety
RISC-V International
 
Reverse Engineering of Rocket Chip
RISC-V International
 
RISC-V NOEL-V - A new high performance RISC-V Processor Family
RISC-V International
 
RISC-V 30910 kassem_ summit 2020 - so_c_gen
RISC-V International
 
RISC-V 30908 patra
RISC-V International
 
RISC-V 30907 summit 2020 joint picocom_mentor
RISC-V International
 
RISC-V 30906 hex five multi_zone iot firmware
RISC-V International
 
RISC-V 30946 manuel_offenberg_v3_notes
RISC-V International
 
RISC-V software state of the union
RISC-V International
 
Ripes tracking computer architecture throught visual and interactive simula...
RISC-V International
 
Porting tock to open titan
RISC-V International
 
Open j9 jdk on RISC-V
RISC-V International
 
Open source manufacturable pdk for sky water 130nm process node
RISC-V International
 
Klessydra t - designing vector coprocessors for multi-threaded edge-computing...
RISC-V International
 
Ad

Recently uploaded (20)

PDF
DevBcn - Building 10x Organizations Using Modern Productivity Metrics
Justin Reock
 
PPTX
Webinar: Introduction to LF Energy EVerest
DanBrown980551
 
PPTX
AI Penetration Testing Essentials: A Cybersecurity Guide for 2025
defencerabbit Team
 
PDF
Newgen Beyond Frankenstein_Build vs Buy_Digital_version.pdf
darshakparmar
 
PDF
Exolore The Essential AI Tools in 2025.pdf
Srinivasan M
 
PDF
CIFDAQ Weekly Market Wrap for 11th July 2025
CIFDAQ
 
PDF
Blockchain Transactions Explained For Everyone
CIFDAQ
 
PDF
Presentation - Vibe Coding The Future of Tech
yanuarsinggih1
 
PPTX
WooCommerce Workshop: Bring Your Laptop
Laura Hartwig
 
PDF
Log-Based Anomaly Detection: Enhancing System Reliability with Machine Learning
Mohammed BEKKOUCHE
 
PDF
Chris Elwell Woburn, MA - Passionate About IT Innovation
Chris Elwell Woburn, MA
 
PDF
CIFDAQ Market Insights for July 7th 2025
CIFDAQ
 
PDF
Using FME to Develop Self-Service CAD Applications for a Major UK Police Force
Safe Software
 
PDF
Python basic programing language for automation
DanialHabibi2
 
PDF
SWEBOK Guide and Software Services Engineering Education
Hironori Washizaki
 
PDF
Transcript: New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
PPTX
From Sci-Fi to Reality: Exploring AI Evolution
Svetlana Meissner
 
PDF
Newgen 2022-Forrester Newgen TEI_13 05 2022-The-Total-Economic-Impact-Newgen-...
darshakparmar
 
PDF
Achieving Consistent and Reliable AI Code Generation - Medusa AI
medusaaico
 
PDF
Reverse Engineering of Security Products: Developing an Advanced Microsoft De...
nwbxhhcyjv
 
DevBcn - Building 10x Organizations Using Modern Productivity Metrics
Justin Reock
 
Webinar: Introduction to LF Energy EVerest
DanBrown980551
 
AI Penetration Testing Essentials: A Cybersecurity Guide for 2025
defencerabbit Team
 
Newgen Beyond Frankenstein_Build vs Buy_Digital_version.pdf
darshakparmar
 
Exolore The Essential AI Tools in 2025.pdf
Srinivasan M
 
CIFDAQ Weekly Market Wrap for 11th July 2025
CIFDAQ
 
Blockchain Transactions Explained For Everyone
CIFDAQ
 
Presentation - Vibe Coding The Future of Tech
yanuarsinggih1
 
WooCommerce Workshop: Bring Your Laptop
Laura Hartwig
 
Log-Based Anomaly Detection: Enhancing System Reliability with Machine Learning
Mohammed BEKKOUCHE
 
Chris Elwell Woburn, MA - Passionate About IT Innovation
Chris Elwell Woburn, MA
 
CIFDAQ Market Insights for July 7th 2025
CIFDAQ
 
Using FME to Develop Self-Service CAD Applications for a Major UK Police Force
Safe Software
 
Python basic programing language for automation
DanialHabibi2
 
SWEBOK Guide and Software Services Engineering Education
Hironori Washizaki
 
Transcript: New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
From Sci-Fi to Reality: Exploring AI Evolution
Svetlana Meissner
 
Newgen 2022-Forrester Newgen TEI_13 05 2022-The-Total-Economic-Impact-Newgen-...
darshakparmar
 
Achieving Consistent and Reliable AI Code Generation - Medusa AI
medusaaico
 
Reverse Engineering of Security Products: Developing an Advanced Microsoft De...
nwbxhhcyjv
 

Online test program generator for RISC-V processors

  • 1. December 8-10, 2020 Virtual Event Ivannikov Institute for System Programming Russian Academy of Sciences Nikita Chertok, Mikhail Chupilko, Alexander Kamkin Online Test Program Generator for RISC-V Microprocessors
  • 2. MicroTESK Framework MicroTESK Framework Specifications (nML) Specifications Translator Test Programs (ASM) Test Program Generator Binary Code Analyzer Tool Builders Processor Model Coverage Simulator Metadata lui a0,0xdead ori a0,a0,0x0 lui a1,0xbeef ori a1,a1,0xf add t0,a0,a1 Simulator (C/C++) Specifications (WhyML) … Ref. Simulator (QEMU) Specifications Debug 1 https://forge.ispras.ru/projects/microtesk
  • 3. op add (rd: X, rs1: X, rs2: X) syntax = format("add %s, %s, %s", rd.syntax, rs1.syntax, rs2.syntax) image = format("0000000%s%s000%s0110011", rs2.image, rs1.image, rd.image) action = { rd = rs1 + rs2; } mode X (i: card(5)) = XREG[i] syntax = format("%s", if i==0 then ZERO().syntax elif i==1 then RA().syntax elif i==2 then SP().syntax elif i==3 then GP().syntax elif i==4 then TP().syntax elif i>=5 && i<=7 then Temp(coerce(card(3), i-5)).syntax elif i>=8 && i<=9 then Saved(coerce(card(4), i-8)).syntax elif i>=10 && i<=17 then Func(coerce(card(3), i-10)).syntax elif i>=18 && i<=27 then Saved(coerce(card(4), i-16)).syntax else Temp(coerce(card(3), i-25)).syntax image = format("%5s", i) nML Language Overview 2 #ifdef RV64I let XLEN = 64 ... #else ... #endif let MEMORY_SIZE_IN_WORDS = 2 ** (XLEN – 2) shared mem MEM[MEMORY_SIZE_IN_WORDS, WORD] type WORD = card(32) type XWORD = card(XLEN) type FLOAT32 = float(23, 8) reg XREG [32, XWORD] reg PC [XWORD]
  • 4. Specifications 3 Specifications Characteristic Value RISC-V Instruction Set Manual Vol. I: User-Level ISA (v. 2.2) 145 pages (94 pages in chapters 2-18) Base Extension Base Extension In Total RV32I M F C P 24 pages 44 pages 68 pages RV32E A D B V RV64I Q J N 2 pages 24 pages 26 pages RV128I L T # Specified Instructions 230 instructions ISA Specifications Size (nML) 4 900 LOC (w/o comments) MMU Specification Size (MMUSL) 460 LOC (Sv32, Sv39, and Sv48)
  • 5. Offline Test Program Generation 4 Design Under Test (RTL, FPGA) lui a0, 0xdead ori a0, a0, 0x0 lui a1, 0xbeef ori a1, a1, 0xf add t0, a0, a1 sub t1, a0, t1 add t0, t0, t1 Test Programs (ASM) Reference Simulator (C/C++) Traces (Tarmac) / Signatures 0x2000: lui ... 0x2004: ori ... 0x2008: lui ... 0x200c: ori ... 0x2010: add ... 0x2014: sub ... 0x2018: add ... Comparator (Python) 0x2000: lui ... 0x2004: ori ... 0x2008: lui ... 0x200c: ori ... 0x2010: add ... 0x2014: sub ... 0x2018: bug ... Traces (Tarmac) / Signatures
  • 7. MicroTESK Test Program Generator (Offline) MicroTESK Test Program Generator Specifications (nML) Translator Test Templates (Ruby) lui a0, 0xdead ori a0, a0, 0x0 lui a1, 0xbeef ori a1, a1, 0xf add t0, a0, a1 sub t1, a0, t1 add t0, t0, t1 Test Programs (Assembly Code) Processor Model Generation Core Specification Engineer Verification Engineer 6 Autogen https://forge.ispras.ru/projects/microtesk-riscv
  • 8. Test Templates (Offline)  Test templates similar to RISC-V Foundation’s Tests https://github.com/riscv/riscv-tests  Test templates similar to RISC-V Torture Test Generator https://github.com/ucb-bar/riscv-torture 7 class MyTemplate < RiscVBaseTemplate def run block(:combinator => 'product', :compositor => 'random') { iterate { xor x(_), x(_), x(_) do situation(...) end lui x(_), _ } iterate { and x(_), x(_), x(_) or x(_), x(_), x(_) } }.run end end # Single Test Case # Initialization ori a7, a7, 0x2d7 slli a7, a7, 0xb ori a7, a7, 0x1 slli a7, a7, 0xb ori a7, a7, 0x3d2 ori t3, t3, 0x164 slli t3, t3, 0xb ori t3, t3, 0x52b slli t3, t3, 0xb ori t3, t3, 0x24e # Stimulus and s4, a7, a7 xor s8, s4, t3 2 × 2 = 4 test cases • Combinatorial Brute Force • Randomization • Constraint Solving • Self Checking Code is distributed under Apache License, v2.0
  • 9. Online Test Generation Offline Test Generation lui a0, 0xdead ori a0, a0, 0x0 lui a1, 0xbeef ori a1, a1, 0xf add t0, a0, a1 sub t1, a0, t1 add t0, t0, t1 Test Program Generator (Standalone Software) Test Programs (Binaries) Design Under Test (RTL, FPGA, IC) DUT executes loaded test programs lui a0, 0xdead ori a0, a0, 0x0 lui a1, 0xbeef ori a1, a1, 0xf add t0, a0, a1 sub t1, a0, t1 add t0, t0, t1 Test Program Generator (Binary Inside DUT) Test Programs (Binaries) Design Under Test (FPGA, IC) DUT executes test generator that produces test programs Tests programs are loaded to memory 8
  • 10. Online TPG Requirements  Autonomy  Self-generation of a variety of tests from a template  Self-checking of tests results  Performance  Test massiveness prevails over test directedness  Simple techniques (brute force, randomization, mutation, etc.)  Robustness  No loss of control (exception handling, timers, etc.)  Saving and restoring generator state 9
  • 11. Online TPG Architecture Control Unit Allocator Executor Test Oracle add … sub … ... add … nop sub … Generator Transformer Residential Part Test Image External Part Control Console Generator Commands Parameters Statistics Coverage Failures .text 110011010100000… 110011011100000… ... .data 1101111010101101 .text 110011010100000… 110010000000000… 110011011100000… .data 1101111010101101 Test w/ mutant (QED) Verdict .data 1101111010101101 1011111011101111 .data 1101111010101101 1011101011101111 Test Results Test Code Allocation, linking, etc. Signatures Metadata Allocates code in memory Gets signatures and compares them Requests next test case Transfers control to tests and returns back Deviation 10
  • 12. MicroTESK Test Program Generator (Online) MicroTESK Test Program Generator Specifications (nML) Translator Test Templates (Ruby) Test Generator (Code in C) Processor Model Generation Core Specification Engineer Verification Engineer 11 Autogen https://forge.ispras.ru/projects/microtesk-riscv Generator Transformer
  • 13. Test Templates (Online)  Test template is translated to C language: combinators, compositors, preparators, etc.  Generator in C implements a certain interface: can be obtained by other means (e.g., Autogen) class MyTemplate < RiscVBaseTemplate def run block(:combinator => 'product', :compositor => 'random') { iterate { xor x(_), x(_), x(_) do ... end lui x(_), _ } iterate { and x(_), x(_), x(_) or x(_), x(_), x(_) } }.run end end // MyTemplate (simplified) OP *iterate_0[] = { {XOR}, {LUI} }; OP *iterate_1[] = { {AND}, {OR} }; // :combinator => 'product' for (i[0] = 0; i[0] < 2; i[0]++) for (i[1] = 0; i[1] < 2; i[1]++) { OP *combine[] = tpg_malloc(...) combine[0] = iterate_0[i[0]]; combine[1] = iterate_1[i[1]]; // :compositor => 'random' OP compose[] = tpg_malloc(...); k = j[0] = j[1] = 0; while (j[0] < 1 && j[1] < 1) { while (j[s = rand() % 2] == 2); compose[k++] = combine[s][j[s]++]; } // choose registers, generate data, // insert initialization code, etc. ... // get the test’s binary image for (int n = 0; n < N; n++) { binary[n] = test[n].image(); } } 12
  • 14. Online TPG Features  Test generation techniques  Combinatorial brute-force generation (“perebor”)  Randomization  No constraint solving!  Test result checking techniques  Precalculated signatures returned by generator  Equivalent (modulo inputs) transformations  Dead code injection (NOPs between instructions, etc.)  Quick error detection (QED) 13
  • 15. Future Work Directions  RISC-V specifications  Specifying more subsets (RV128I, Q, L, B, T, J, P, V, and N)  Online test program generation  Error localization and diagnostics  Test coverage tracking and test completeness assessment  Advanced test program generation  Model-based generation (modeling compiler transformations)  Mutations and coverage-directed fuzzing  Binary code equivalence checking  Verifying optimizing compilers (including JIT and AOT) 14