13 superscalar
- 1. Topics Left • Superscalar machines • IA64 / EPIC architecture • Multithreading (explicit and implicit) • Multicore Machines • Clusters • Parallel Processors • Hardware implementation vs microprogramming
- 2. Chapter 14 Superscalar Processors • Definition of Superscalar • Design Issues: - Instruction Issue Policy - Register renaming - Machine parallelism - Branch Prediction - Execution • Pentium 4 example
- 3. What is Superscalar? • “Common” instructions (arithmetic, load/store, conditional branch) can be executed independently. • Equally applicable to RISC & CISC, but more straightforward in RISC machines. • The order of execution is usually assisted by the compiler. A Superscalar machine executes multiple independent instructions in parallel. They are pipelined as well.
- 4. Example of Superscalar Organization • 2 Integer ALU pipelines, • 2 FP ALU pipelines, • 1 memory pipeline (?)
- 6. Limitations of Superscalar • Dependent upon: - Instruction level parallelism possible - Compiler based optimization - Hardware support • Limited by — Data dependency — Procedural dependency — Resource conflicts
- 7. (Recall) True Data Dependency (Must W before R) ADD r1, r2 r1+r2 r1 MOVE r3, r1 r1 r3 • Can fetch and decode second instruction in parallel with first LOAD r1, X x (memory) r1 MOVE r3, r1 r1 r3 • Can NOT execute second instruction until first is finished Second instruction is dependent on first (R after W)
- 8. (recall) Antidependancy (Must R before W) ADD R4, R3, 1 R3 + 1 R4 ADD R3, R5, 1 R5 + 1 R3 • Cannot complete the second instruction before the first has read R3
- 9. (Recall) Procedural Dependency • Can’t execute instructions after a branch in parallel with instructions before a branch, because? Note: Also, if instruction length is not fixed, instructions have to be decoded to find out how many fetches are needed
- 10. (recall) Resource Conflict • Two or more instructions requiring access to the same resource at the same time — e.g. two arithmetic instructions need the ALU • Solution - Can possibly duplicate resources — e.g. have two arithmetic units
- 11. Effect of Dependencies on Superscalar Operation Notes: 1) Superscalar operation is double impacted by a stall. 2) CISC machines typically have different length instructions and need to be at least partially decoded before the next can be fetched – not good for superscalar operation
- 12. Instruction-level Parallelism – degree of • Consider: LOAD R1, R2 ADD R3, 1 ADD R4, R2 These can be handled in parallel. • Consider: ADD R3, 1 ADD R4, R3 STO (R4), R0 These cannot be handled in parallel. The “degree” of instruction-level parallelism is determined by the number of instructions that can be executed in parallel without stalling for dependencies
- 13. Instruction Issue Policies • Order in which instructions are fetched • Order in which instructions are executed • Order in which instructions update registers and memory values (order of completion) Standard Categories: • In-order issue with in-order completion • In-order issue with out-of-order completion • Out-of order issue with out-of-order completion
- 14. In-Order Issue -- In-Order Completion Issue instructions in the order they occur: • Not very efficient • Instructions must stall if necessary (and stalling in superpipelining is expensive)
- 15. In-Order Issue -- In-Order Completion (Example) Assume: • I1 requires 2 cycles to execute • I3 & I4 conflict for the same functional unit • I5 depends upon value produced by I4 • I5 & I6 conflict for a functional unit
- 16. In-Order Issue -- Out-of-Order Completion (Example) How does this effect interrupts? Again: • I1 requires 2 cycles to execute • I3 & I4 conflict for the same functional unit • I5 depends upon value produced by I4 • I5 & I6 conflict for a functional unit
- 17. Out-of-Order Issue -- Out-of-Order Completion • Decouple decode pipeline from execution pipeline • Can continue to fetch and decode until the “window” is full • When a functional unit becomes available an instruction can be executed (usually in as much in- order as possible) • Since instructions have been decoded, processor can look ahead
- 18. Out-of-Order Issue -- Out-of-Order Completion (Example) Note: I5 depends upon I4, but I6 does not Again: • I1 requires 2 cycles to execute • I3 & I4 conflict for the same functional unit • I5 depends upon value produced by I4 • I5 & I6 conflict for a functional unit
- 19. Register Renaming to avoid hazards • Output and antidependencies occur because register contents may not reflect the correct ordering from the program • Can require a pipeline stall • One solution: Allocate Registers dynamically (renaming registers)
- 20. Register Renaming example Add R3, R3, R5 R3b:=R3a + R5a (I1) Add R4, R3, 1 R4b:=R3b + 1 (I2) Add R3, R5, 1 R3c:=R5a + 1 (I3) Add R7, R3, R4 R7b:=R3c + R4b (I4) • Without “subscript” refers to logical register in instruction • With subscript is hardware register allocated: R3a R3b R3c Note: R3c avoids: antidependency on I2 output dependency I1
- 21. Recaping: Machine Parallelism Support • Duplication of Resources • Out of order issue hardware • Windowing to decouple execution from decode • Register Renaming capability
- 22. Speedups of Machine Organizations (Without Procedural Dependencies) • Not worth duplication of functional units without register renaming • Need instruction window large enough (more than 8, probably not more than 32)
- 23. Branch Prediction in Superscalar Machines • Delayed branch not used much. Why? Multiple instructions need to execute in the delay slot. This leads to much complexity in recovery. • Branch prediction should be used - Branch history is very useful
- 24. View of Superscalar Execution
- 25. Committing or Retiring Instructions Results need to be put into order (commit or retire) • Results sometimes must be held in temporary storage until it is certain they can be placed in “permanent” storage. (either committed or retired/flushed) • Temporary storage requires regular clean up – overhead – done in hardware.
- 26. Superscalar Hardware Support • Facilities to simultaneously fetch multiple instructions • Logic to determine true dependencies involving register values and Mechanisms to communicate these values • Mechanisms to initiate multiple instructions in parallel • Resources for parallel execution of multiple instructions • Mechanisms for committing process state in correct
- 27. Example: Pentium 4 A Superscalar CISC Machine
- 28. Pentium 4 alternate view
- 29. Pentium 4 pipeline 20 stages !
- 30. a) Generation of Micro-ops (stages 1 &2) • Using the Branch Target Buffer and Instruction Translation Lookaside Buffer, the x86 instructions are fetched 64 bytes at a time from the L2 cache •The instruction boundaries are determined and instructions decoded into 1-4 118-bit RISC micro-ops • Micro-ops are stored in the trace cache
- 31. b) Trace cache next instruction pointer (stage 3) • The Trace Cache Branch Target Buffer contains dynamic gathered history information (4 bit tag) • If target is not in BTB - Branch not PC relative: predict branch taken if it is a return, predict not taken otherwise - For PC relative backward conditional branches, predict take, otherwise not taken
- 32. c) Trace Cache fetch (stage 4) • Orders micro-ops in program-ordered sequences called traces • These are fetched in order, subject to branch prediction • Some micro-ops require many micro-ops (CISC instructions). These are coded into the ROM and fetched from the ROM
- 33. d) Drive (stage 5) • Delivers instructions from the Trace Cache to the Rename/Allocator module for reordering
- 34. e) Allocate: register naming (stages 6, 7, & 8) • Allocates resources for execution (3 micro-ops arrive per clock cycle): - Each micro-op is allocated to a slot in the 126 position circular Reorder Buffer (ROB) which tracks progress of the micro-ops. Buffer entries include: - State – scheduled, dispatched, completed, ready for retire - Address that generated the micro-op - Operation - Alias registers are assigned for one of 16 arch reg (128 alias registers) {to remove data dependencies} • The micro-ops are dispatched out of order as resources are available • Allocates an entry to one of the 2 scheduler queues - memory access or not
- 35. f) Micro-op queuing (stage 9) • Micro-ops are loaded into one of 2 queues: - one for memory operations - one for non memory operations • Each queue operates on a FIFO policy
- 36. g) Micro-op scheduling (stages 10, 11, & 12) • The 2 schedulers retrieve micro-ops based upon having all the operands ready and dispatch them to an available unit (up to 6 per clock cycle) • If two micro-ops need the same unit, they are dispatched in sequence. h) Dispatch (stages 13 & 14)
- 37. i) Register file (stages 15 & 16) j) Execute: flags (stages 17 & 18) • The register files are the sources for pending fixed and FF operations • A separate stage is used to compute the flags
- 38. k) Branch check (stage 19) l) Branch check results (stage 20) • Checks flags and compares results with predictions • If the branch prediction was wrong: - all incorrect micro-ops must be flushed (don’t want to be wrong!) - the correct branch destination is provided to the Branch Predictor - the pipeline is restarted from the new target address