Accelerated Windows Software Trace Analysis training public slides

Windows Software
Trace Analysis
Dmitry Vostokov
Software Diagnostics Services

What’s it all About?
 General Trace Analysis Patterns
 Windows context
© 2012 Software Diagnostics Services

Prerequisites
Basic Windows troubleshooting

Training Goals
 Review tracing fundamentals
 Learn trace analysis patterns
 Practice finding patterns in logs

Training Principles
 Lots of pictures
 Pattern relationships
 Practical examples

Schedule Summary
Day 1
 Trace Analysis Fundamentals
 Trace Analysis Patterns
Day 2
 Trace Analysis Patterns
 Examples

Part 1: Fundamentals

Basic Concepts
 Software Trace (or Log)
 Process
 Thread
 Adjoint Thread
 Module (or Source)
 File
 Function
 Message (or Operation)
 Stack trace

Software Trace (Log)
 A sequence of formatted messages
 Arranged by time
 A narrative story

Process
 PID
 Session
 Image Name
 Modules (DLLs)
 Examples:
svchost.exe
notepad.exe
PID 1 PID 2
PID 3 PID 4

Thread
 TID
 CPU
 Context
PID
CPU 1 CPU 2
TID 1
TID 1
TID 2

Adjoint Thread
Debugging TV Frame 0x14

Exercise T0
1. Download Process Monitor
2. Trace system activity
3. Add more columns such as TID
4. Filter a thread based on TID
5. Reset filter
6. Filter an adjoint thread based on image
name svchost.exe
7. Filter an adjoint thread based on PID

Module / Source
 Module Name
 Source Folder
ApplicationA.exe
sourcelibrary
svchost.exe
ApplicationA.exe
ModuleB.dll
ModuleC.dll
sourcelibrary*.c

File and Function
// MainApp.c
foo () {
trace(“foo: entry”);
// do stuff
trace(“foo: exit”);
}
MainApp.c
MainApp.c: foo
MainApp.c

Trace Message
// MainApp.c
foo () {
int result = bar();
trace(“bar result: 5”);
trace(“foo: exit”);
}
Invariant Variable Invariant Variable …

Stack Trace
// MainApp.c
main() {
trace(“start”);
foo();
}
foo() {
bar();
}
bar() {
trace(“bar: entry”);
// do stuff
}
bar: entry
foo: entry
start
bar
foo
main
foo
main

Trace Recording Tools
 Process Monitor
 MessageHistory
 CDFControl

Trace Analysis Tools
 Process Monitor
 CDFControl
 CDFAnalyzer
 MS Office Excel

Minimal Trace Graphs
Time
# PID TID Time Message
No Module PID TID Date Time Message
-----------------------------------------------------------
1 ModuleA 4280 1736 5/28/2012 08:53:50.496 Trace message 1
2 ModuleB 6212 6216 5/28/2012 08:53:52.876 Trace message 2
[…]

Trace Formats
 ETW
 CDF
 CSV
 Free
 Mixed

Pattern-Driven Analysis
Pattern: a common recurrent identifiable problem together with a set of
recommendations and possible solutions to apply in a specific context
Checklist: http://www.dumpanalysis.org/blog/index.php/2011/03/10/software-
trace-analysis-checklist/
Patterns: http://www.dumpanalysis.org/blog/index.php/trace-analysis-
patterns/
Trace
Collection
Checklists Patterns Action

Pattern-Based Analysis
Software Trace
New Pattern
Discovery
Pattern
Catalog
+
Usage

Pattern Hierarchy
 Domain Independent
from IBM mainframes to mobile and embedded computers
 Domain Specific

Pattern Classification
 Vocabulary
 Error
 Trace as a Whole
 Large Scale
 Activity
 Message
 Block
 Trace Set

Part 2: Individual Patterns

Vocabulary Patterns
 Basic Facts
 Vocabulary Index

Basic Facts
 Problem Description
Application disappears after launch
 Software Trace
PID Message
-----------------------------------
…
3f6 Create process AppA: PID 4a5
4a5 AppA loads DLLC
…
3f6 Create process AppB: PID 5b8
5b8 AppB loads DLLD
…
Related Patterns
Vocabulary Index

Basic Facts Taxonomy
 Functional Facts
Example: Expected a dialog to enter data
 Non-functional Facts
Example: CPU consumption 100%
 Identification Facts
Application name, PID, user name

Vocabulary Index
Related Patterns
Basic Facts
Activity Region
 Problem Description
A user Test123 authentication failed
basic fact index
 Narrowing:

Error Patterns
 Error Message
 Exception Stack Trace
 False Positive Error
 Periodic Error ↓*
 Error Distribution
* ‘↓’ sign means that a pattern involves time dependency

Error Message
Related Patterns
False Positive Error
Periodic Error
Error Distribution
Adjoint Thread
Data Flow Explicit errors
 Implicit errors
 WinDbg command !error
0:000> !error c0000017
Error code: (NTSTATUS) 0xc0000017 (3221225495) - {Not Enough Quota} Not enough
virtual memory or paging file quota is available to complete the specified
operation.
0:000> !error 5
Error code: (Win32) 0x5 (5) - Access is denied.

Exception Stack Trace
No PID TID Message
------------------------
[…]
265799 8984 4216 ComponentA.Store.GetData threw exception: ‘System.Reflection.TargetInvocationException: DCOM connection to
server failed with error: ‘Exception from HRESULT: 0×842D0001′ —> System.Runtime.InteropServices.COMException (0×842D0001):
Exception from HRESULT: 0×842D0001
265800 8984 4216 === Exception Stack Trace ===
265801 8984 4216 at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
265802 8984 4216 at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
265803 8984 4216 at ComponentA.Store.GetData(Byte[] pKey)
265804 8984 4216 at ComponentA.App.EnumBusinessObjects()
[…]
Related Patterns
Error Message

Periodic Error ↓
Related Patterns
Error Message
Error Distribution
Message Invariant
No PID TID Message
-----------------------
[…]
36495 1788 2250 MyClass::Init: Cannot open connection “Client ID: 310″, status=5
[…]
[…]
36883 1788 1986 MyClass::Init: Cannot open connection “Client ID: 612″, status=5
[…]
Time

 Expected errors
 Not relevant to our problem
 Implementation details
Related Patterns
Error Message
Master Trace
Activity Region

Error Distribution
Time
Related Patterns
Partition
Activity Region

Trace as a Whole
 Partition
 Circular Trace ↓
 Message Density
 Message Current ↓
 Trace Acceleration ↓
 No Trace Metafile
 Empty Trace
 Missing Module
 Guest Module
 Truncated Trace ↓
 Visibility Limit
 Sparse Trace

Partition
Related Patterns
Significant Event
Truncated Trace
Adjoint Thread
Tail
Epilogue
Head
Time
Prologue
Core

Circular Trace ↓
Time
Problem
Repro
No Module PID TID Date Time Message
---------------------------------------------------------------
2 ModuleB 6212 6216 5/28/2009 08:53:52.876 Trace message 2
[…]
[…]
Related Patterns
Focus of Tracing

Message Density
D1 > D2
Similar relative density for 2 traces may shows correlation:
D11 / D21 = D12 / D22
For correlated messages different densities from 2 traces may
show different partition or system conditions:
D11 >> D12
Time
Related Patterns
Intra-correlation
Focus of Tracing
Relative Density
Partition

Message Current ↓
J1 > J2
Time
Time
10.100
10.200
10.100
12.100
Related Patterns
Significant Event
Activity Region
Message Density

Trace Acceleration ↓
Message current Ji < Jj, i < j < N
Partial message currents:
with respect to TID X Jk(TID=x)
with respect to PID Y Jk(PID=y)
with respect to PID X and TID Z Jk(PID=y & TID=z)
Jj
Jl
JN
Time
Ji
Jk
Jm
Related Patterns
Activity Region
Message Current
Thread of Activity
Adjoint Thread of Activitiy

No Trace Metafile
# Module PID TID Time Message
-------------------------------------------
[…]
21372 dllA 2968 5476 3:55:10.004 Calling foo()
21373 Unknown 2968 5476 3:55:10.004 Unknown GUID=A1E38F24-613D-4D71-B9F5-… (No Format Information found).
21374 Unknown 2968 5476 3:55:10.004 Unknown GUID=A1E38F24-613D-4D71-B9F5-… (No Format Information found)
21378 dllA 2968 5476 3:55:10.004 Calling bar()
[…]
Possible patterns to detect:
 Circular Trace
 Message Density
 Message Current
 Discontinuity
 Time Delta
 Trace Acceleration
Related Patterns
Thread of Activity

Empty Trace
Related Patterns
Truncated Trace
No Activity
Missing Module
 Small file size
 Very few trace messages
Always open a trace before sending to
someone else

Missing Module
Related Patterns
Discontinuity
Inter-Correlation
No Activity
Time
Missing
Tracing Best Practices

Guest Module
Related Patterns
Missing ModuleTime
Load: 3rdPartyActivity.dll

Truncated Trace ↓
Tail
Epilogue
Head
Time
Prologue
Core
Head
Prologue
Core
Related Patterns
Partition
Anchor Messages
Missing Module

Visibility Limit
Time
r
Related Patterns
Truncated Trace
Missing Module
Sparse Trace

Sparse Trace
Related Patterns
Missing Module
Visibility Limit
Time
Missing
L
PLOT

Large Scale Patterns
 Characteristic Block
 Background Modules
 Foreground Modules
 Layered Periodization
 Focus of Tracing
 Event Sequence Order ↓
 Trace Frames

Bird’s Eye Binary View

Characteristic Block
Time

Background Modules
Background:
Foreground:
Related Patterns
Foreground ModulesTime

Foreground Modules
Time
Time
Related Patterns
Background Modules
Module Foregrounding

Layered Periodization
Time
Time
Time

Focus of Tracing
Activity regions: Jm1, Jm2, Jm3
Related Patterns
Activity RegionTime
Jm1
Jm2
Jm3

Event Sequence Order ↓
Synchronization
Race Conditions
Deadlock
Related Patterns
Significant Event
Anchor Messages
Time
E1
E2
E3
E4
E5
Time
E2
E3
E4
E5
E1

Frames (Source Code)
Visual Studio 2012

Trace Frames
Related Patterns
Thread of Activity
Adjoint Thread
Truncated Trace
Discontinuity
Time

Activity Patterns
 Thread of Activity ↓
 Adjoint Thread of Activity ↓
 No Activity
 Activity Region
 Discontinuity ↓
 Time Delta ↓
 Glued Activity
 Break-in Activity ↓
 Resume Activity ↓
 Data Flow ↓

Thread of Activity ↓
Related Patterns
Discontinuity
Sparse Trace
Time
# PID TID Time Func Message

Adjoint Thread of Activity ↓
Related Patterns
Thread of Activity
Message Invariant
Time
Time

No Activity
Related Patterns
Discontinuity
Sparse Trace
Missing Module
Time
We expect this process
Causes: hang, wait chain,
deadlock, terminated threads,
CPU loop

Activity Region
Message current : Jm2 > max (Jm1,Jm3)
Time
Jm1
Jm2
Jm3
Related Patterns
Message Current

Discontinuity ↓
Time
Time
Related Patterns
Thread of Activity
Missing Module
Sparse Trace
Possible causes:
Blocked thread, IPC response
delay, wait chains, long
computation

Time Delta ↓
Related Patterns
Basic Facts
Thread of Activity
Discontinuity
Significant Event
# Module PID TID Time File Function Message
---------------------------------------------------------------------------------------------------
6060 dllA 1604 7108 10:06:21.746 fileA.c DllMain DLL_PROCESS_ATTACH
24480 dllA 1604 7108 10:06:32.262 fileA.c LaunchApp Exec Path: C:Program FilesCompanyAappB.exe
30 seconds of discontinuity till the end of full trace
Time

Glued Activity
ATID: Adjoint Thread ID
ImageA
ATID 2
ImageB
ATID 3
Time
# ATID TID Time Message
Related Patterns
Adjoint Thread
Time
Trace
Session
1
Trace
Session
2

Break-in Activity ↓
Related Patterns
Thread of Activity
Adjoint Thread
Discontinuity
Resume Activity
Time
Discontinuity

Resume Activity ↓
Related Patterns
Break-in Activity
Thread of Activity
Adjoint Thread
Time
Discontinuity

Data Flow ↓
Time
Related Patterns
Adjoint Thread
Message Invariant
[…]
DriverA: Device 0xA IRP 0xB
[…]
DriverB: Device 0xC IRP 0xB
[…]
DriverC: Device 0xD IRP 0xB
DriverC: Processing IRP 0xB
[…]

Message Patterns
 Significant Event
 Defamiliarizing Effect
 Anchor Messages
 Diegetic Messages
 Message Change ↓
 Message Invariant
 UI Message
 Original Message
 Implementation Discourse
 Opposition Messages
 Linked Messages
 Gossip ↓
 Counter Value
 Message Context
 Marked Messages
 Incomplete History
 Message Interleave
 Fiber Bundle

Significant Event
Related Patterns
Error Message
Basic Facts
Vocabulary Index
Time

Poetry of Software Traces
“Capturing delicate moments, one gives birth to a poetry of traces …”
Ange Leccia, Motionless Journeys, by Fabien Danesi

Defamiliarizing Effect
Time
Time
Related Patterns
Activity Region

Anchor Messages
Related Patterns
Vocabulary Index
Adjoint Thread
Message Interleave
Time
----------------------------------------------------------
24226 2656 3480 10:41:05.774 AppA.exe: DLL_PROCESS_ATTACH
108813 4288 4072 10:41:05.774 AppB.exe: DLL_PROCESS_ATTACH
112246 4180 3836 10:41:05.940 DllHost.exe: DLL_PROCESS_ATTACH
135473 2040 3296 10:41:12.615 AppC.exe: DLL_PROCESS_ATTACH
694723 1112 1992 10:44:23.393 AppD.exe: DLL_PROCESS_ATTACH
705891 1528 2592 10:44:42.307 regedit.exe: DLL_PROCESS_ATTACH
785231 2992 4912 10:45:26.516 AppE.exe: DLL_PROCESS_ATTACH
786523 3984 1156 10:45:26.605 powershell.exe: DLL_PROCESS_ATTACH
817979 4188 4336 10:45:48.707 wermgr.exe: DLL_PROCESS_ATTACH
834875 3976 1512 10:45:52.342 LogonUI.exe: DLL_PROCESS_ATTACH
835229 4116 3540 10:45:52.420 AppG.exe: DLL_PROCESS_ATTACH

Message Interleave
Related Patterns
Adjoint Thread
Anchor Messages
Time

Diegetic Messages
Time
Process PID 234 is OK
Status OK
Status OK
Status OK
Related Patterns
Anchor Messages

Message Change ↓
Related Patterns
Anchor Messages
Message Invariant
Adjoint Thread
Status = 0x0
Time
Status = 0x0
Status = 0x0
Status = 0xc0000017
Status = 0xc0000017
Status = 0xc0000017

Implementation Discourse
 Win32 API
 MFC
 Kernel Development
 COM
 C# / .NET
 C++
 Java
 …

Message Invariant
-------------------------------------------------------------------------------------------
[…]
2782 ModuleA 2124 5648 10:58:03.356 CreateObject: pObject 0×00A83D30 data ([…]) version 0×4
[…]
-------------------------------------------------------------------------------------------
[…]
4793 ModuleA 2376 8480 09:22:01.947 CreateObject: pObject 0×00BA4E20 data ([…]) version 0×5
[…]
Related Patterns
Trace Set

UI Message
Related Patterns
Activity Region
Significant Event
Thread of Activity
Adjoint Thread
--------------------------------------------------------------------------------
[…]
2782 ModuleA 2124 5648 10:58:03.356 CreateWindow: Title "..." Class "..."
[…]
3512 ModuleA 2124 5648 10:58:08.154 Menu command: Save Data
[…]
3583 ModuleA 2124 5648 10:58:08.155 CreateWindow: Title "Save As" Class "Dialog"
[... Data update and replication related messages ...]
4483 ModuleA 2124 5648 10:58:12.342 DestroyWindow: Title "Save As" Class "Dialog"
[…]
--------------------------------------------------------------------------------
[…]
2782 ModuleA 2124 5648 10:58:03.356 CreateWindow: Title "..." Class "..."
3512 ModuleA 2124 5648 10:58:08.154 Menu command: Save Data
3583 ModuleA 2124 5648 10:58:08.155 CreateWindow: Title "Save As" Class "Dialog"
4483 ModuleA 2124 5648 10:58:12.342 DestroyWindow: Title "Save As" Class "Dialog"
[…]

Original Message
---------------------------------------------------------------------------------------------------------
[…]
35835 ModuleA 12332 11640 18:27:28.720 LoadLibrary: Program FilesMyProductSystem32MyDLL.dll PID 12332
[…]
[…]
[…]
Related Patterns
Message Invariant
Adjoint Thread

Linked Messages
Time
CreateProcess AppB.exe
CreateProcess AppA.exe
ImageLoad AppB.exe
ImageLoad AppC.exe
ImageLoad AppA.exe
CreateProcess AppC.exe
Related Patterns
Adjoint Thread
# PID Message
---------------------------------------------
[…]
128762 1260 CreateProcess: PPID 1260 PID 6356
[…]
128785 6356 ImageLoad: AppA.exe PID 6356
[…]
[…]
131239 6280 ImageLoad: AppB.exe PID 6280
[…]
[…]
132906 8144 ImageLoad: AppC.exe PID 8144
[…]

Gossip ↓
Related Patterns
Adjoint Thread
Event Sequence Order
Message Interleave
# Module PID TID Message
[…]
26875 ModuleA 2172 5284 LoadImage: DeviceHarddiskVolume2WindowsSystem32notepad.exe PID 0x000000000000087C
26876 ModuleB 2172 5284 LoadImage: DeviceHarddiskVolume2WindowsSystem32notepad.exe, PID (2172)
26877 ModuleC 2172 5284 ImageLoad: fileName=notepad.exe, pid: 000000000000087C
[…]
[…]
26875 ModuleA 2172 5284 LoadImage: DeviceHarddiskVolume2WindowsSystem32notepad.exe PID 0×000000000000087C
[…]
33132 ModuleA 4180 2130 LoadImage: DeviceHarddiskVolume2WindowsSystem32calc.exe PID 0×0000000000001054
[…]

Counter Value
Module Variable
18:04:06 Explorer.EXE 3280 User Time: 8.4864544 seconds, Kernel Time: 9.5004609 seconds, Private Bytes: 42,311,680, Working Set: 10,530,816
Related Patterns
Adjoint Thread
Significant Event
Activity Region
Focus of Tracing
Characteristic Message Block
Performance-specific patterns:
Global Monotonicity
Constant Value

Message Context
Related Patterns
Significant Event
Anchor Message
Time
Context
<- Message

Marked Messages
Related Patterns
Master Trace
No Activity
Annotated messages:
session database queries [+]
session initialization [-]
socket activity [+]
process A launched [+]
process B launched [-]
process A exited [-]
[+] activity is present in a trace
[-] activity is undetected or not present

Fiber Bundle
I/O stack
Thread stack
trace
Trace
messages
Related Patterns

Incomplete History
Related Patterns
Opposition Messages
Sparse Trace
Truncated Trace
Master Trace
Code:
 Response-complete
 Exception-complete
 Call-complete

Opposition Messages
 open - close
 create – destroy (discard)
 allocate - free (deallocate)
 call - return
 enter - exit (leave)
 load - unload
 save - load
 lock - unlock
 map - unmap
Related Patterns
Incomplete History
Sparse Trace

Block Patterns
 Macrofunction
 Periodic Message Block
 Intra-Correlation

Macrofunction
------------------------------------------------------------
[…]
42582 DBClient 5492 9476 11:04:33.398 Opening connection
[…]
42585 DBClient 5492 9476 11:04:33.398 Sending SQL command
[…]
42589 DBServer 6480 10288 11:04:33.399 Executing SQL command
[…]
42592 DBClient 5492 9476 11:04:33.400 Closing connection
[…]

Periodic Message Block
Related Patterns
Periodic Error
Adjoint Thread
Invariant Message
Discontinuity
Time

Intra-Correlation
Handle: 00050586 Class: "Application A Class" Title: ""
Title changed at 15:52:4:3 to "Application A"
Title changed at 15:52:10:212 to "Application A - File1"
[…]
Process ID: 89c
Thread ID: d6c
[…]
Visible: true
Window placement command: SW_SHOWNORMAL
Placement changed at 15:54:57:506 to SW_SHOWMINIMIZED
Placement changed at 15:55:2:139 to SW_SHOWNORMAL
Foreground: false
Foreground changed at 15:52:4:3 to true
Foreground changed at 15:53:4:625 to false
[…]
Handle: 000D0540 Class: "App B" Title: "Application B"
[...]
Process ID: 3ac
Thread ID: bd4
[...]
Foreground: false
[…]
Related Patterns
Basic Facts
Activity Regions
WindowHistory WindowHistory64

Trace Set Patterns
 Master Trace
 Bifurcation Point
 Inter-Correlation
 Relative Density
 News Value
 Impossible Trace
 Split Trace

Master Trace
Related Patterns
Activity Regions
Background Modules
Foreground Modules
Event Sequence Order
Guest Module
Implementation Discourse
Bifurcation Point

Bifurcation Point
Software Trace Diagrams
Time
# PID TID Message
----------------------------------
[…]
25 2768 3056 Trace Statement A
26 3756 2600 Trace Statement B
27 3756 2600 Trace Statement C
[…]
149 3756 836 Query result: X
150 3756 836 Trace Statement 150.1
[…]
# PID TID Message
-----------------------------------
[…]
27 2768 3056 Trace Statement A
28 3756 2176 Trace Statement B
29 3756 2176 Trace Statement C
[…]
151 3756 5940 Query result: Y
[…]

Inter-Correlation
System
Tracing Tool
Tracing Tool
Trace File Trace File
Related Patterns
Intra-Correlation
Basic Facts
Discontinuity
Sparse Trace

Relative Density
Time
Time
1 / 1
3 / 1
Related Patterns
Message Density

News Value
Related Patterns
Inter-Correlation
Basic Facts
Master Trace
1
2
3
4
Time

Impossible Trace
-------------------------------
[…]
1001 ModuleA 202 404 foo: start
1002 ModuleA 202 404 foo: end
[…]
void foo()
{
TRACE("foo: start");
bar();
TRACE("foo: end");
}
void bar()
{
TRACE("bar: start");
// some code ...
TRACE("bar: end");
}
Related Patterns
Sparse Trace

Split Trace
Time
# PID TID Time Message # PID TID Time Message # PID TID Time Message
Related Patterns
Circular Trace

12.12.12
Related Patterns
Adjoint Thread
Discontinuity
Time Delta
Periodic Message Block
MM=DD=YY

Part 3: Practice Exercises

Links
Not included in Public Preview version

Process Monitor Examples
Exercises T1-T6

Exercise T1
 Goal: Learn how to identify application crashes
 Patterns: Background Modules, Adjoint Thread of Activity,
Discontinuity, Guest Module, Fiber Bundle

Exercise T2
 Goal: Learn how to identify CPU consumption, profile
processes and threads
 Patterns: Activity Region, Characteristic Message Block,
Periodic Message Block, Thread of Activity, No Activity,
Counter Value, Sparse Trace

Exercise T3
 Goal: Learn how to calculate message current and density
 Patterns: Activity Region, Thread of Activity, Time Delta,
Message Current, Message Density, Relative Density

Exercise T4
 Goal: Learn how to compare software traces
 Patterns: Master Trace, Characteristic Message Block,
Bifurcation Point

Exercise T5
 Goal: Learn process startup sequence for terminal services
session
 Patterns: Adjoint Thread of Activity, Anchor Messages,
Message Interleave

Exercise T6
 Goal: Learn how to work with split traces
 Patterns: Split Trace, Adjoint Thread of Activity, Fiber Bundle

Using Excel for Analysis
Debugging TV Frame 0x15

Resources
 TraceAnalysis.org
 Windows Internals, 6th ed.
 Inside Windows Debugging
 Introduction to Pattern-Driven Software Problem Solving
 Software Trace and Memory Dump Analysis
 Software Narratology: An Applied Science of Software Stories
 Introduction to Pattern-Driven Software Diagnostics
 Systemic Software Diagnostics
 Debugging TV
 Memory Dump Analysis Anthology (volumes 3, 4, 5, 6)

Q&A
Please send your feedback using the contact
form on DumpAnalysis.com

Thank you for attendance!

Accelerated Windows Software Trace Analysis training public slides

More Related Content

Similar to Accelerated Windows Software Trace Analysis training public slides (20)

More from Dmitry Vostokov (20)

Recently uploaded (20)

Accelerated Windows Software Trace Analysis training public slides