verification_planning_systemverilog_uvm_2020
3 likes•1,389 views
This document provides an overview of a training session on SystemVerilog for verification. The agenda includes verification planning, course contents on SystemVerilog basics and verification techniques, chip design flow, old verification languages, verification approaches, and a case study on verifying an arithmetic logic unit. Verification planning concepts like test plans, features and test types, specifications extraction, and measurements are also discussed.
verification_planning_systemverilog_uvm_2020
- 1. SystemVerilog For Verification Training Session 0: Verification Planning Sameh El-Ashry Senior Digital Verification Engineer
- 2. S. ElAshry © 2020 Agenda 1. Road Map. 2. Course Contents. 3. Chip Flow. 4. Verification Basics. 5. Verification Planning and Management. 6. Case Study. 7. Verification Planning Tools. 8. References.
- 3. S. ElAshry © 2020 Course Contents ▪ Verification Planning. ▪ Testbench basics using Verilog. ▪ SystemVerilog basics (Data Types). ▪ Processes and Threads. ▪ OOP in SystemVerilog. ▪ How to create a testbench SystemVerilog Environment ? ▪ Randomization and Constraints. ▪ Coverage. ▪ Assertions.
- 4. S. ElAshry © 2020 Chip Design Flow 7/19/2020 4 Design Specifications Floor Planning Technology Library SDF & Parasitics Design Entry (Schematic/HDL) Functional Verification & UPF Verification Logic & Test Synthesis Layout Design Placement & Routing Static Timing Analysis Gate Level Simulation Formal Verification Power Aware Verification (UPF) Physical Verification Tapeout Front End Back End Verification Signoff Floor Planning & CTS (Verilog or VHDL Language) (SystemVerilog Language and UVM)
- 5. S. ElAshry © 2020 Old Languages for Verification (VHDL vs.. Verilog) ▪ What language should I use? ▪ This is usually dictated by one’s experience and personal preference. ▪ Typically, when working with a language, you do not notice the things that are simple to do, instead you notice the frustrations and how easy it would be if you were using the other language. ▪ Both languages are not enough for verification (by themselves). ▪ Both languages are equal in terms of the area under the learning curve. VHDL’s learning curve is steeper, but Verilog’s goes on much further.
- 6. S. ElAshry © 2020 Why we need functional verification? ▪ To build confidence and stay in business. ▪ A primary purpose for functional verification is to detect failures so that bugs can be identified and corrected before it gets shipped to costumer. ▪ A single mistake may lead to a chip failure. ▪ Not all bugs are caused by coding errors. There are possibilities that error may in the specification itself.
- 7. S. ElAshry © 2020 60% - 80% time spent in verification – WHY?? ▪ Product time-to-market. ▪ Cost of bugs over time. ▪ Longer a bug goes undetected, the more expensive it is. ▪ Bug found early (designer simulation) has little cost. ▪ Finding a bug at chip/system has moderate cost. ▪ Requires more debug time and isolation time. ▪ Could require new algorithm, which could effect schedule and cause board rework. ▪ Finding a bug in System Test (test floor) requires new ‘spin’ of a chip. ▪ Finding bug in customer’s environment can cost hundreds of millions and worst of all – Reputation.
- 8. S. ElAshry © 2020 What is Verification? ▪ Not a testbench. ▪ Not a series of testbenches.
- 9. S. ElAshry © 2020 What is a testbench? ▪ A “testbench” usually refers to the code used to create a pre-determined input sequence to a design, then optionally observe the response. ▪ Generic term used differently across industry. ▪ Always refers to a testcase. ▪ Most commonly (and appropriately), a testbench refers to code written (VHDL, Verilog, SystemVerilog, SystemC .. etc) at the top level of the hierarchy. ▪ The testbench is often simple, but may have some elements of randomness. ▪ Completely closed system.
- 10. S. ElAshry © 2020 Ways to Reduce Verification Time ▪ Parallelism: Add more resources. ▪ Abstraction: Higher level of abstraction (i.e. C vs.. Assembly). ▪ Automation: Tools to automate standard processes. • Requires standard processes. • Not all processes can be automated. • Regression automation. • Test Pass/Fail reporting automation. ▪ Scripting languages. (Makefiles, Perl, Python).
- 11. S. ElAshry © 2020 Functional Verification Approaches ▪ Black-Box Approach ▪ White-Box Approach ▪ Grey-Box Approach
- 12. S. ElAshry © 2020 Black-Box Verification Approach • The black box has inputs, outputs, and performs some function. • The specifications function document may be well documented...or not. • To verify a black box, you need to understand the function and be able to predict the outputs based on the inputs. • The black box can be a full system, a chip, or a unit of a chip. • Disadvantages: ❑ Can only control inputs and observe outputs, thus it lacks controllability and observability. ❑ It is difficult to locate source of problem. • Advantages: ❑ Independent of implementation. ❑ I.E. could be VHDL, Verilog, C model, Arch model, behavioral, RTL, etc. Some piece of logic design written in VHDL or Verilog Inputs Outputs
- 13. S. ElAshry © 2020 White-Box Verification Approach ▪ White box verification means that the internal facilities are visible and utilized by the testbench stimulus. ▪ Examples: Unit/Module level verification. ▪ Opposite of Black-Box. ▪ Full visibility and controllability of internals. ▪ Quickly setup. ▪ Tightly integrated with implementation. ▪ Testbench must change if Design Under Verification (DUV) changes.
- 14. S. ElAshry © 2020 Grey-Box Verification Approach ▪ Grey box verification means that a limited number of facilities are utilized in a mostly black-box environment. ▪ Example: Most environments! Prediction of correct results on the interface is occasionally impossible without viewing an internal signal.
- 15. S. ElAshry © 2020 Verification VS. Test ▪ Two often confused. ▪ Purpose of test is to verify that the design was manufactured properly. ▪ Verification is to ensure that the design meets the functionality intent. Specification Silicon Verification HW Design Net list Test Fabrication
- 16. S. ElAshry © 2020
- 17. S. ElAshry © 2020 Phases of Verification Analyze Coverage Extract Code Coverage Writing Tests Building Testbench Verification Plan
- 18. S. ElAshry © 2020 Detailed Phases Of Verification
- 19. S. ElAshry © 2020 Verification Plan Concept ▪ In test plan, we prepare a road map for how do achieve the goal, it is a living document. ▪ Test plan includes, introduction, assumptions, list of test cases, list of features to be tested, approach, deliverables, resources, risks and exit (sign off) criteria. ▪ Test plan helps verification engineer to understand how the verification should be done. ▪ A test plan could come in many forms, such as a spreadsheet, a document or a simple text file. ▪ Test plan also contains the descriptions of TestBench architecture and description of each component and its functionality.
- 20. S. ElAshry © 2020 Case Study: Arithmetic Logic Unit (ALU) ▪ What is your input? Standard document? Specifications document?
- 21. S. ElAshry © 2020 Features and Tests ▪ The functionality to be tested should be broken down into more detailed list of features that are to be verified. ▪ This list of features gives an initial list of tests that are to be executed for verification. ▪ A test plan should be reviewed thoroughly by all the relevant experts of design.
- 22. S. ElAshry © 2020 Different kinds of Tests (Around 12 types) ▪ Black box tests: System level testing is mostly black box testing. Tests are based on the requirement and functionality of the design‚ there is no knowledge of the internal design or code. ▪ White box tests: Block level tests are mostly white box tests based on internal logic of the RTL. Tests are based on coverage of code statements‚ branches‚ paths and conditions.
- 23. S. ElAshry © 2020 Different kinds of Tests2 ▪ Unit tests: These are to test particular function or code at micro scale. This is also done by block designers as it requires detailed knowledge of internal code and design. ▪ Integration testing: testing of combined parts of blocks to determine if they function together correctly. This is different than toplevel where all the system is tested together.
- 24. S. ElAshry © 2020 Different kinds of Tests3 ▪ End-to-end testing: involves testing of a complete application environment in a situation that mimics real-world use‚ for example running PHY transmitter and receiver at the same simulation environment. ▪ Sanity testing: typically an initial testing effort to determine if a new RTL version is performing well enough to accept it for a major testing effort.
- 25. S. ElAshry © 2020 Different kinds of Tests4 ▪ Regression testing: re-running tests after fixes or modifications of the RTL or its environment. It is generally difficult to determine how much regression running is needed‚ especially near the end of the design cycle. Automated regression setup can be especially useful for this type of testing. ▪ Acceptance testing: final testing based on specifications of the customer‚ or based on use by end-users/customers over some limited period of time.
- 26. S. ElAshry © 2020 Different kinds of Tests5 ▪ Stress testing: term often used interchangeably with ‘load’ and ‘performance’ testing. Also used to describe such tests as system functional testing while under unusually heavy loads‚ heavy repetition of certain actions or inputs‚ input of large stimulus for chip capacity/buffer management. ▪ Performance testing: term often used interchangeably with ‘stress’ and ‘load’ testing. To determine system performance under stress or normal conditions and see if they match expected values.
- 27. S. ElAshry © 2020 Different kinds of Tests6 ▪ Corner tests: Verification engineers work with designers to define additional tests that address the corner cases for optimum coverage. ▪ Negative testing: Deliberately inserting some errors in the test stimulus and check if RTL catches it and that system does not hang on seeing negative input that is out of specified range
- 28. S. ElAshry © 2020 Specifications Extraction (Design and Verification Requirements) (Features)
- 29. S. ElAshry © 2020 Measurements ▪ Tests (Inherited form the Driving) • Test1. • Test2. ▪ Checking • Scoreboard. • Assertion. ▪ Coverage • Covergroup. • Coverpoint. • Crosspoint.
- 30. S. ElAshry © 2020 Verification Requirements
- 31. S. ElAshry © 2020 Building Testbench • In this phase, the verification environment is developed. – Verilog verification environment (Directed). – SystemVerilog verification environment. – UVM verification environment. • Each verification component can be developed one by one or if more than one engineer is working it can be developed parallel. • Writing the coverage module can be done at any time. It is preferred to write down the coverage module first as it gives some idea of the verification progress.
- 32. S. ElAshry © 2020 Mapping to Verilog Environment For ALU Checking Module Driver Module Monitor Module Environment Module Test Case Module Top Module DUT
- 33. S. ElAshry © 2020 Spec Mapping and Linking to the Testbench
- 34. S. ElAshry © 2020 Spec Mapping and Linking to the Testbench2 Plan Test Metric Checker Metric Cov Metric Feature Scripts Makefile Excel Sheet Reports Testbench Files
- 35. S. ElAshry © 2020 Final Mapping Result Show Using DVE (Tools Example) Reference: https://semiengineering.com/coverage-driven-verification-isnt-complete-low-power-metrics/
- 36. S. ElAshry © 2020 Verification Planning with Questa® Verification Management (Tools Example) https://verificationacademy.com/verification-horizons/june-2017-volume-13-issue-2/verification-planning-with- questa-verification-management
- 37. S. ElAshry © 2020 Sign off criteria - verification metrics ▪ (98% to 100%) code coverage. ▪ (100%) functional coverage. ▪ Test status: All tests are passing.
- 38. S. ElAshry © 2020 Mini Verification Plan Project ▪ Extract the verification requirements for the APB bus. (Features) ▪ Generate all measurements for each feature • Test • Checker • Coverage ▪ Add tags for spec linking and mapping. ▪ Write python script to parse the tags from the excel sheet, you need to parse the tags columns only, the tags columns should be stored in lists, print the lists in external text file. ▪ Enhance the python script to search for tags inside the testbench files. ▪ Generate reports.
- 39. S. ElAshry © 2020 Busses: the glue that connects the pieces Timers Central Processing Unit Software Hardware Internal External System Buses AHB/APB ldr (read) str (write) ISA USART DAC/ADC Internal & External Memory GPIO/INT C Assembly Machine Code Interrupts bl (int)
- 40. S. ElAshry © 2020 AHB Vs APB ▪ Advanced/AMBA High-performance Bus (AHB) ▪ Advanced Peripheral Bus (APB) AHB • High performance • Pipelined operation • Burst transfers • Multiple bus masters • Split transactions APB • Low power • Latched address/control • Simple interface • Suitable of many peripherals
- 41. S. ElAshry © 2020 References ▪ Makefile Introduction ▪ https://www.gnu.org/software/make/manual/html_node/Introduction. html ▪ https://makefiletutorial.com/ ▪ Python ▪ https://www.tutorialspoint.com/python/index.htm ▪ Reading Excel sheet using python ▪ https://www.geeksforgeeks.org/reading-excel-file-using-python/
- 42. Thank You ! SystemVerilog Training Presented by Sameh El-Ashry [email protected] www.linkedin.com/in/sameh-elashry-22b5603b Session 0 https://scholar.google.com/citations?hl=en&user=UD8UHsoAAAAJ