![Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca
Generating Discriminating Examples
9
Engineer: expressed
her intention φintended
φexpressed
Debugger: Infer Semantic
⇒ p[R] ⋀ …
P[R]: Pattern P
applied to relation R](https://image.slidesharecdn.com/models-2015-160623034732/85/Pattern-Based-Debugging-of-Declarative-Models-27-320.jpg)
![Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca
Generating Discriminating Examples
9
Engineer: expressed
her intention φintended
φexpressed
Debugger: Infer Semantic
⇒ p[R] ⋀ …
P[R]: Pattern P
applied to relation R
Debugger: Take a guess
that the model
might be underconstrained
w.r.t. φintended
p’ [R]
Weaken p to p’
p[R] p’ [R]
Strengthening is in reverse](https://image.slidesharecdn.com/models-2015-160623034732/85/Pattern-Based-Debugging-of-Declarative-Models-28-320.jpg)
![Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca
Generating Discriminating Examples
9
Engineer: expressed
her intention φintended
φexpressed
Debugger: Infer Semantic
⇒ p[R] ⋀ …
P[R]: Pattern P
applied to relation R
Debugger: Take a guess
that the model
might be underconstrained
w.r.t. φintended
p’ [R]
Weaken p to p’
p[R] p’ [R]
Strengthening is in reverse
Debugger: Mutate
the semantic
p’ [R] ⋀ ¬p[R] ⋀ …](https://image.slidesharecdn.com/models-2015-160623034732/85/Pattern-Based-Debugging-of-Declarative-Models-29-320.jpg)
![Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca
Generating Discriminating Examples
9
Engineer: expressed
her intention φintended
φexpressed
Debugger: Infer Semantic
⇒ p[R] ⋀ …
P[R]: Pattern P
applied to relation R
Debugger: Take a guess
that the model
might be underconstrained
w.r.t. φintended
p’ [R]
Weaken p to p’
p[R] p’ [R]
Strengthening is in reverse
Debugger: Mutate
the semantic
Debugger: Find
discriminating example
p’ [R] ⋀ ¬p[R] ⋀ …](https://image.slidesharecdn.com/models-2015-160623034732/85/Pattern-Based-Debugging-of-Declarative-Models-30-320.jpg)
![Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca
Generating Discriminating Examples
9
Engineer: expressed
her intention φintended
φexpressed
Debugger: Infer Semantic
⇒ p[R] ⋀ …
P[R]: Pattern P
applied to relation R
Debugger: Take a guess
that the model
might be underconstrained
w.r.t. φintended
p’ [R]
Weaken p to p’
p[R] p’ [R]
Strengthening is in reverse
Debugger: Mutate
the semantic
Debugger: Find
discriminating example
p’ [R] ⋀ ¬p[R] ⋀ …
Engineer: accepts or
rejects the discriminating
example](https://image.slidesharecdn.com/models-2015-160623034732/85/Pattern-Based-Debugging-of-Declarative-Models-31-320.jpg)
![Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca
Pattern-based Debugger’s components:
UI: DiscriminatingExample
12
❖ Back to the Linked list example
❖ Debugger finds: φexpressed ⇒ Total_Function[next],…
❖ Debugger generates a discriminating example, w.r.t.:
E ⊨ ¬Total_Function[next] ⋀ Partial_Function[next]
Engineer accepts E
φexpressed is overconstrained w.r.t. φintended
Fix by weakening
Engineer rejects E
Generate another example
❌✓](https://image.slidesharecdn.com/models-2015-160623034732/85/Pattern-Based-Debugging-of-Declarative-Models-34-320.jpg)
Pattern-Based Debugging of Declarative Models
- 1. Pattern-Based Debugging of Declarative Models Vajih Montaghami and Derek Rayside {vmontagh,drayside}@uwaterloo.ca Electrical and Computer Engineering University of Waterloo Waterloo, Ontario, Canada Fri 2 Oct 2015
- 2. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Say what you mean Adapted from: http://cloudcomputingcell.com/computer-programmer-images/ sig Node{next: Node} one sig Head extends Node{} pred StructuralConstr aints{ all n: Node | head 2
- 3. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Say what you mean Adapted from: http://cloudcomputingcell.com/computer-programmer-images/ sig Node{next: Node} one sig Head extends Node{} pred StructuralConstr aints{ all n: Node | head φintended 2
- 4. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Say what you mean Adapted from: http://cloudcomputingcell.com/computer-programmer-images/ sig Node{next: Node} one sig Head extends Node{} pred StructuralConstr aints{ all n: Node | head φexpressed φintended 2
- 5. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Say what you mean Adapted from: http://cloudcomputingcell.com/computer-programmer-images/ sig Node{next: Node} one sig Head extends Node{} pred StructuralConstr aints{ all n: Node | head ⇎φexpressed φintended 2
- 6. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Engineer intends to model a linear linked list Adapted from: http://cloudcomputingcell.com/computer-programmer-images/ sig Node{next: Node} one sig Head extends Node{} pred StructuralConstra ints{ all n: Node | one n.next} head 3
- 7. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Engineer intends to model a linear linked list Adapted from: http://cloudcomputingcell.com/computer-programmer-images/ sig Node{next: Node} one sig Head extends Node{} pred StructuralConstra ints{ all n: Node | one n.next} head head 3
- 8. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Adapted from: http://cloudcomputingcell.com/computer-programmer-images/ sig Node{next: Node} one sig Head extends Node{} pred StructuralConstra ints{ all n: Node | one n.next} head head ✗Underconstraint But, what engineer expresses is not what he intended 3
- 9. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Adapted from: http://cloudcomputingcell.com/computer-programmer-images/ sig Node{next: Node} one sig Head extends Node{} pred StructuralConstra ints{ all n: Node | one n.next} head ∅ head ✗Underconstraint But, what engineer expresses is not what he intended 3
- 10. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Adapted from: http://cloudcomputingcell.com/computer-programmer-images/ sig Node{next: Node} one sig Head extends Node{} pred StructuralConstra ints{ all n: Node | one n.next} head ∅ head ✗Underconstraint Engineer Strengthens the constraints, but gets nothing! ✗Full-Overconstraint 3
- 11. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Adapted from: http://cloudcomputingcell.com/computer-programmer-images/ sig Node{next: Node} one sig Head extends Node{} pred StructuralConstra ints{ all n: Node | one n.next} head ∅ head head ✗Underconstraint Engineer Strengthens the constraints, but gets nothing! ✗Full-Overconstraint 3
- 12. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Adapted from: http://cloudcomputingcell.com/computer-programmer-images/ sig Node{next: Node} one sig Head extends Node{} pred StructuralConstra ints{ all n: Node | one n.next} head ∅ head head ✗Underconstraint Engineer Weakens the constraints, but excludes some examples ✗Full-Overconstraint ✗Partial-Overconstraint 3
- 13. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Adapted from: http://cloudcomputingcell.com/computer-programmer-images/ sig Node{next: Node} one sig Head extends Node{} pred StructuralConstra ints{ all n: Node | one n.next} head ∅ head head ✗Underconstraint Engineer Weakens the constraints, and gets intended examples ✗Full-Overconstraint ✗Partial-Overconstraint ✓ head headhead 3
- 14. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca4 Can we invent a tool to guide this strengthening and weakening of the model?
- 15. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Idea: Discriminating Examples 5 φintended φexpressed
- 16. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Idea: Discriminating Examples 5 φintended φexpressed example
- 17. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Idea: Discriminating Examples 5 φintended φexpressed example discriminating example
- 18. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Mean what you say No bug φintended ⇔ φexpressed Underconstraint φintended ⇒ φexpressed Partial-overconstraint φintended ⇐ φexpressed Fully-overconstraint (︎∄i: I| i ⊨ φexpressed) ∧ (∃j: I| j ⊨ φintended) Loosely Intersecting Intention ∃i: I | (i ⊨ φintended ⋀ φexpressed) ⋀ (i ⊭ φintended ⇒ φexpressed) ⋀ (i ⊭ φexpressed ⇒ φintended) 6
- 19. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Mean what you say No bug φintended ⇔ φexpressed Underconstraint φintended ⇒ φexpressed Partial-overconstraint φintended ⇐ φexpressed Fully-overconstraint (︎∄i: I| i ⊨ φexpressed) ∧ (∃j: I| j ⊨ φintended) Loosely Intersecting Intention ∃i: I | (i ⊨ φintended ⋀ φexpressed) ⋀ (i ⊭ φintended ⇒ φexpressed) ⋀ (i ⊭ φexpressed ⇒ φintended) 6 φintended φexpressed
- 20. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Mean what you say No bug φintended ⇔ φexpressed Underconstraint φintended ⇒ φexpressed Partial-overconstraint φintended ⇐ φexpressed Fully-overconstraint (︎∄i: I| i ⊨ φexpressed) ∧ (∃j: I| j ⊨ φintended) Loosely Intersecting Intention ∃i: I | (i ⊨ φintended ⋀ φexpressed) ⋀ (i ⊭ φintended ⇒ φexpressed) ⋀ (i ⊭ φexpressed ⇒ φintended) 6 φintended φexpressed
- 21. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Mean what you say No bug φintended ⇔ φexpressed Underconstraint φintended ⇒ φexpressed Partial-overconstraint φintended ⇐ φexpressed Fully-overconstraint (︎∄i: I| i ⊨ φexpressed) ∧ (∃j: I| j ⊨ φintended) Loosely Intersecting Intention ∃i: I | (i ⊨ φintended ⋀ φexpressed) ⋀ (i ⊭ φintended ⇒ φexpressed) ⋀ (i ⊭ φexpressed ⇒ φintended) 6 φintended φexpressed
- 22. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Mean what you say No bug φintended ⇔ φexpressed Underconstraint φintended ⇒ φexpressed Partial-overconstraint φintended ⇐ φexpressed Fully-overconstraint (︎∄i: I| i ⊨ φexpressed) ∧ (∃j: I| j ⊨ φintended) Loosely Intersecting Intention ∃i: I | (i ⊨ φintended ⋀ φexpressed) ⋀ (i ⊭ φintended ⇒ φexpressed) ⋀ (i ⊭ φexpressed ⇒ φintended) 6 φintended φexpressed
- 23. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Mean what you say No bug φintended ⇔ φexpressed Underconstraint φintended ⇒ φexpressed Partial-overconstraint φintended ⇐ φexpressed Fully-overconstraint (︎∄i: I| i ⊨ φexpressed) ∧ (∃j: I| j ⊨ φintended) Loosely Intersecting Intention ∃i: I | (i ⊨ φintended ⋀ φexpressed) ⋀ (i ⊭ φintended ⇒ φexpressed) ⋀ (i ⊭ φexpressed ⇒ φintended) 6 φintended φexpressed
- 24. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Pattern-based debugging Existing tools Pattern-based Underconstraint Random examples Minimal examples Discriminating examples Help the engineer decide if certain properties need to be strengthened or weakened Overconstraint UnSAT-Core Subset of UnSAT-Core Accepting or Rejecting discriminating examples localizes area to fix Explanation None By Pattern Semantics 7 Better Understanding Increase Localization Best-effort Fix suggestion
- 25. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Context ❖ Model: A text written in a logical language ❖ Bug: a divergence between the engineers intended logical model and the expressed logic model 8 sig Node{next: Node} one sig Head extends Node{} pred StructuralConstraints{ all n: Node | one n.next} pred Noloop{ all n: Node | n !in n.^next} pred showLists{ StructuralConstraints Noloop} run showLists ❖ Logical language: Alloy ๏ First-order relational logic with closure ๏ Alloy Analyzer: Instance finder equipped with Inconsistent core finder in finite domain ๏ Bugs: - Unintended instances - Inconsistent constraints
- 26. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Generating Discriminating Examples 9 Engineer: expressed her intention φintended
- 27. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Generating Discriminating Examples 9 Engineer: expressed her intention φintended φexpressed Debugger: Infer Semantic ⇒ p[R] ⋀ … P[R]: Pattern P applied to relation R
- 28. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Generating Discriminating Examples 9 Engineer: expressed her intention φintended φexpressed Debugger: Infer Semantic ⇒ p[R] ⋀ … P[R]: Pattern P applied to relation R Debugger: Take a guess that the model might be underconstrained w.r.t. φintended p’ [R] Weaken p to p’ p[R] p’ [R] Strengthening is in reverse
- 29. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Generating Discriminating Examples 9 Engineer: expressed her intention φintended φexpressed Debugger: Infer Semantic ⇒ p[R] ⋀ … P[R]: Pattern P applied to relation R Debugger: Take a guess that the model might be underconstrained w.r.t. φintended p’ [R] Weaken p to p’ p[R] p’ [R] Strengthening is in reverse Debugger: Mutate the semantic p’ [R] ⋀ ¬p[R] ⋀ …
- 30. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Generating Discriminating Examples 9 Engineer: expressed her intention φintended φexpressed Debugger: Infer Semantic ⇒ p[R] ⋀ … P[R]: Pattern P applied to relation R Debugger: Take a guess that the model might be underconstrained w.r.t. φintended p’ [R] Weaken p to p’ p[R] p’ [R] Strengthening is in reverse Debugger: Mutate the semantic Debugger: Find discriminating example p’ [R] ⋀ ¬p[R] ⋀ …
- 31. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Generating Discriminating Examples 9 Engineer: expressed her intention φintended φexpressed Debugger: Infer Semantic ⇒ p[R] ⋀ … P[R]: Pattern P applied to relation R Debugger: Take a guess that the model might be underconstrained w.r.t. φintended p’ [R] Weaken p to p’ p[R] p’ [R] Strengthening is in reverse Debugger: Mutate the semantic Debugger: Find discriminating example p’ [R] ⋀ ¬p[R] ⋀ … Engineer: accepts or rejects the discriminating example
- 32. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Patterns Binary relations ❖ Relational properties ❖ 21 are commonly known ❖ used in structural modelling ❖ e.g.: acyclic, antisymmetric, irreflexive, … Ternary relations ❖ Inclusion and Ordering ❖ we define 160 ❖ used in behavioural modelling ❖ e.g.: ExpandHeadOfRight, … (ece.uwaterloo.ca/~vmontagh/alloy/debugger/) 10
- 33. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Pattern-based Debugger’s components 11
- 34. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Pattern-based Debugger’s components: UI: DiscriminatingExample 12 ❖ Back to the Linked list example ❖ Debugger finds: φexpressed ⇒ Total_Function[next],… ❖ Debugger generates a discriminating example, w.r.t.: E ⊨ ¬Total_Function[next] ⋀ Partial_Function[next] Engineer accepts E φexpressed is overconstrained w.r.t. φintended Fix by weakening Engineer rejects E Generate another example ❌✓
- 35. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Pattern-based Debugger’s components: Knowledge-base Consistency Graph ❖ Undirected graph ❖ Nodes: Patterns ❖ Edges: Two nodes are consistent over the same relation ❖ Nodes: 160 Edges: 6764 Implication Lattice ❖ Directed graph ❖ Nodes: Patterns ❖ Edges: The source node implies the sink node over the same relation ❖ Sources:12 Sinks:6 Longest Path:6 Acyclic Irreflexive Acyclic Partial-function 13
- 36. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Pattern-based Debugger’s components: Reasoner Generate discriminating examples ❖ Infer Semantic ❖ Semantic Mutation ❖ Generation tactics (see paper) ❖ antecedent satisfaction, ❖ disjunction vacuity detection ❖ conjunction vacuity detection 14 Localize the issue ❖ UnSAT-Core ❖ Correspondence tactics (see paper) ❖ quantifier unrolling ❖ function and predicate inlining
- 37. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca A case study ❖ Dijkstra’s Dining Philosophers algorithm ❖ The bogus model shipped with Alloy for years ❖ Was fixed in 2012 by team from KIT Germany ๏ Fix is subtle and was not explained in their publications ❖ In four interactions, the pattern-based debugger assists the engineer to Understand, Localize, and Fix the defects Source: wikipedia.org 15
- 38. Pattern-Based Debugging of Declarative Models {vmontagh,drayside}@uwaterloo.ca Summary and Future work Now ❖ Pattern-based debugging ๏ Discriminating examples ๏ Library of ternary patterns ๏ Generation tactics ๏ Correspondence tactics ❖ Dining Philosophers case study 16 Next ❖ Automating search procedure ❖ Improve pattern library ❖ More case studies