![“I love that [changes on the shelf configuration interface] reflected in the code
because, particularly, I am not familiar with Gosling. [...] It was kind of very cool
to see how they are talking to each other. That kind of allows me to
understand what is going on. I feel that that kind of teaches me Gosling.”
P3](https://image.slidesharecdn.com/finalvis2024-blendedinterfaces-250604161946-a702b21e/85/Learnable-and-Expressive-Visualization-Authoring-through-Blended-Interfaces-39-320.jpg)
Learnable and Expressive Visualization Authoring through Blended Interfaces
- 1. Learnable and Expressive Visualization Authoring through Blended Interfaces Sehi L’Yi, Astrid van den Brandt, Etowah Adams, Huyen N. Nguyen, Nils Gehlenborg
- 2. How can we design a system that enables users to easily create highly customized visualization?
- 3. How can we design a system that enables users to easily create highly customized visualization? Learnability Expressivity
- 9. Template-Based Shelf Configuration Visual Builder Code Editor Vega Editor Lyra Voyager Microsoft Excel Learnability Expressivity
- 10. Template-Based Shelf Configuration Code Editor Visual Builder How can we build a visualization authoring system that is both learnable and expressive? Learnability Expressivity ?
- 11. Our Ideas Learnability Studies Multimodal User Interaction 1 2
- 12. Our Ideas Multimodal User Interaction : InChorus (Srinivasan et al. CHI 2020); DataBreeze (Srinivasan et al. TVCG 2021); TouchPivot (Jo et al. CHI 2017); Data@Hand (Kim et al. CHI 2021)
- 13. Our Ideas Multimodal User Interaction : InChorus (Srinivasan et al. CHI 2020); DataBreeze (Srinivasan et al. TVCG 2021); TouchPivot (Jo et al. CHI 2017); Data@Hand (Kim et al. CHI 2021)
- 14. Our Ideas Multimodal User Interaction : InChorus (Srinivasan et al. CHI 2020); DataBreeze (Srinivasan et al. TVCG 2021); TouchPivot (Jo et al. CHI 2017); Data@Hand (Kim et al. CHI 2021)
- 15. Our Ideas Multimodal User Interaction : InChorus (Srinivasan et al. CHI 2020); DataBreeze (Srinivasan et al. TVCG 2021); TouchPivot (Jo et al. CHI 2017); Data@Hand (Kim et al. CHI 2021)
- 16. Our Ideas: InChorus (Srinivasan et al. CHI 2020); DataBreeze (Srinivasan et al. TVCG 2021); TouchPivot (Jo et al. CHI 2017); Data@Hand (Kim et al. CHI 2021) + Multimodal User Interaction Complementarity Interaction Martin, 1998
- 17. Our Ideas Multimodal User Interaction : InChorus (Srinivasan et al. CHI 2020); DataBreeze (Srinivasan et al. TVCG 2021); TouchPivot (Jo et al. CHI 2017); Data@Hand (Kim et al. CHI 2021) + Complementarity Interaction Martin, 1998
- 18. Our Ideas Multimodal User Interaction : InChorus (Srinivasan et al. CHI 2020); DataBreeze (Srinivasan et al. TVCG 2021); TouchPivot (Jo et al. CHI 2017); Data@Hand (Kim et al. CHI 2021) + + Martin, 1998 Complementarity Interaction
- 19. Our Ideas Multimodal User Interaction : Authoring Interface “1” Authoring Interface “2” Authoring Interface “3” + + Martin, 1998 Complementarity Interaction
- 20. Our Ideas Multimodal User Interaction 1 2 Learnability Studies
- 21. Our Ideas Learnability Studies : Visualization Literacy, Visualization Onboarding, etc Kwon and Lee, CHI 2016; Ruchikachorn and Mueller, TVCG 2015; Kim et al. CHI 2023; Wang et al. TVCG 2022; Stoiber et al. Vis. Psychol. 2023 ?
- 22. Our Ideas Learnability Studies : Kwon and Lee, CHI 2016; Ruchikachorn and Mueller, TVCG 2015; Kim et al. CHI 2023; Wang et al. TVCG 2022; Stoiber et al. Vis. Psychol. 2023 ? ! • Brushing & Linking • Difference • Morphing Linking ! Visualization Literacy, Visualization Onboarding, etc
- 23. Our Ideas Learnability Studies : ? ! Linking ! Learnable Authoring Interface Expressive Authoring Interface Visualization Literacy, Visualization Onboarding, etc
- 24. Our Ideas Multimodal User Interaction 1 2 Learnability Studies Cooperative Interactions Linking
- 25. Template-Based Shelf Configuration Code Editor Visual Builder Template-Based Shelf Configuration Visual Builder Code Editor Learnability Expressivity Cooperative Interactions Linking Multiple Authoring Interfaces Blended Interfaces
- 26. Template-Based Shelf Configuration Code Editor Visual Builder Shelf Configuration Visual Builder Code Editor Learnability Expressivity Blended Interfaces Cooperative Interactions Linking Multiple Authoring Interfaces
- 28. Natural Language Interface ChatGPT Code Editor Gosling Visualization Grammar Template-based Interface Shelf Configuration Interface Visualization Canvas
- 29. Natural Language Interface ChatGPT Code Editor Gosling Visualization Grammar Template-based Interface Shelf Configuration Interface Visualization Canvas
- 35. User Study Design Hands-on Sessions 15 mins Introduction 20 mins Self-Learning 15 mins Reproduction Interview 10 mins Task 1 Task 2 Task 3 Pre-Study Survey Post-Study Survey
- 36. Interaction Patterns “It is very nice to choose between templates, and then, in shelves, I can change x, y, and color. And then, if I really want to fine-tune the visualization, I could go into the code or ask AI to do it for me. It was very useful to have all four, and it wasn’t anything cluttered or something like that.” P6
- 37. Interaction Patterns “I didn’t know what to put in the code editor, so I just asked to see what the natural language interface changes. And, based on that, I can continue myself” P1
- 38. Survey Results
- 39. “I love that [changes on the shelf configuration interface] reflected in the code because, particularly, I am not familiar with Gosling. [...] It was kind of very cool to see how they are talking to each other. That kind of allows me to understand what is going on. I feel that that kind of teaches me Gosling.” P3
- 40. “The different kinds of scatterplots confused me sometimes. I was somehow expecting only one type of each plot instead of slight variations.” P1
- 41. Main Takeaway Combining and linking authoring interfaces can help learn a unfamiliar authoring system to create complex visualizations
- 42. • Screen scalability • How to combine authoring techniques without output interfaces? e.g., Visualization by Demonstration (Saket et al. 2017) • Would users always start with the most familiar interfaces? Open Questions • Screen scalability • How to combine authoring techniques without output interfaces? e.g., Visualization by Demonstration (Saket et al. 2017) • Would users always start with the most familiar interfaces?
- 43. Summary • A conceptual framework of blended interfaces to blend multiple authoring interfaces • An implementation of Blace for genomics data visualization • A user evaluation with domain experts (n=12) to demonstrate the effectiveness of blended interfaces sehilyi.com [email protected] SEHI L’YI @sehi_lyi @sehilyi.bsky.social NIH K99/R00 Postdoc Fellow Harvard Medical School Formative User Study Tomorrow