UX STRAT Online 2020: Victoria Sosik, Verizon
0 likes266 views
The document discusses democratizing user research at Verizon. It considers the pros and cons of democratizing research insights, experiences, tools, skills, usability research, and generative research. It outlines that democratization is not a binary decision and organizations must consider their specific context. The document then proposes a "Spot Testing" program at Verizon to start democratizing usability research in a limited, structured way by providing research tools and training designers to conduct small, defined tests under the guidance of user researchers. The program would aim to increase bandwidth for research while maintaining rigor and oversight.
UX STRAT Online 2020: Victoria Sosik, Verizon
- 1. Victoria Sosik Director of UX Research Developing a Company- Wide Approach to Design Research at Verizon UX STRAT Online • Sept 2020
- 2. 2 UX Researcher, Google Maps User-generated Content 2014 2020 Director of UX Research Verizon Customer Marketplace Insights 2015 PhD in Human-Computer Interaction Social computing, persuasive technology, positive psychology 2016 2017 2018 2019 UX Research Manager, Google Maps Local Search & Explore
- 3. 3 UX Researcher, Google Maps User-generated Content 2014 2020 Director of UX Research Verizon Customer Marketplace Insights 2015 PhD in Human-Computer Interaction Social computing, persuasive technology, positive psychology 2016 2017 2018 2019 UX Research Manager, Google Maps Local Search & Explore That’s not research! 2014 20202015 2016 2017 2018 2019
- 4. 4 UX Researcher, Google Maps User-generated Content 2014 2020 Director of UX Research Verizon Customer Marketplace Insights 2015 PhD in Human-Computer Interaction Social computing, persuasive technology, positive psychology 2016 2017 2018 2019 UX Research Manager, Google Maps Local Search & Explore That’s not research! 2014 20202015 2016 2017 2018 2019 Usability Science? What will we do?
- 5. 5 UX Researcher, Google Maps User-generated Content 2014 2020 Director of UX Research Verizon Customer Marketplace Insights 2015 PhD in Human-Computer Interaction Social computing, persuasive technology, positive psychology 2016 2017 2018 2019 UX Research Manager, Google Maps Local Search & Explore That’s not research! Pokerface won’t generate valid insights. 2014 20202015 2016 2017 2018 2019 Usability Science? What will we do?
- 6. 01 02 03 04 05 Overview Defining democratization Democratize Don’t democratize Choosing our path at Verizon The Spot Testing program
- 8. 8 re·search /ˈrēˌsərCH,rəˈsərCH/ noun the systematic investigation into and study of materials and sources in order to establish facts and reach new conclusions. verb investigate systematically.
- 9. 9 de·moc·ra·ti·za·tion /dəˌmäkrədəˈzāSH(ə)n/ noun the action of making something accessible to everyone.
- 10. 10 Are we democratizing… ...research insights? ...research experiences? ...research tools? ...research skills? ...usability research? ...generative research?
- 11. 11 Are we democratizing… ...research insights? ...research experiences? ...research tools? ...research skills? ...usability research? ...generative research? IT DEPENDS
- 12. Are we democratizing… ...research insights? ...research experiences? ...research tools? ...research skills? ...usability research? ...generative research? IT DEPENDS 12 Polaris, Kaleidoscope Urban Jungle UserTesting, UserZoom Training Courses
- 13. 13 “Democratizing research is largely a bad idea.” - Saswati Saha Mitra “Democratization is our job." - Behzod Sirjani
- 14. Democratize
- 15. 15 Builds empathy — Making insights and direct access to customers more accessible builds customer empathy. Increases bandwidth — Training team members to conduct research helps meet demand when it outpaces the size of the research team. Protects Researcher time — Enabling others to do some research leaves trained Researchers time for the most challenging, impactful problems. Brings rigor to others’ curiosity — Training others to do better research improves the quality of their learnings and the decisions they make from them. Protects customers — Providing ethics and data governance guidelines for those doing research protects customers and their data. Builds collaboration and awareness — Empowering others builds collaboration and awareness whereas claiming ownership inspires “rogue” actors.
- 16. 16 Builds empathy — Making insights and direct access to customers more accessible builds customer empathy. Increases bandwidth — Training team members to conduct research helps meet demand when it outpaces the size of the research team. Protects Researcher time — Enabling others to do some research leaves trained Researchers time for the most challenging, impactful problems. Brings rigor to others’ curiosity — Training others to do better research improves the quality of their learnings and the decisions they make from them. Protects customers — Providing ethics and data governance guidelines for those doing research protects customers and their data. Builds collaboration and awareness — Empowering others builds collaboration and awareness whereas claiming ownership inspires “rogue” actors.
- 17. 17 Builds empathy — Making insights and direct access to customers more accessible builds customer empathy. Increases bandwidth — Training team members to conduct research helps meet demand when it outpaces the size of the research team. Protects Researcher time — Enabling others to do some research leaves trained Researchers time for the most challenging, impactful problems. Brings rigor to others’ curiosity — Training others to do better research improves the quality of their learnings and the decisions they make from them. Protects customers — Providing ethics and data governance guidelines for those doing research protects customers and their data. Builds collaboration and awareness — Empowering others builds collaboration and awareness whereas claiming ownership inspires “rogue” actors.
- 18. 18 Builds empathy — Making insights and direct access to customers more accessible builds customer empathy. Increases bandwidth — Training team members to conduct research helps meet demand when it outpaces the size of the research team. Protects Researcher time — Enabling others to do some research leaves trained Researchers time for the most challenging, impactful problems. Brings rigor to others’ curiosity — Training others to do better research improves the quality of their learnings and the decisions they make from them. Protects customers — Providing ethics and data governance guidelines for those doing research protects customers and their data. Builds collaboration and awareness — Empowering others builds collaboration and awareness whereas claiming ownership inspires “rogue” actors.
- 19. 19 Builds empathy — Making insights and direct access to customers more accessible builds customer empathy. Increases bandwidth — Training team members to conduct research helps meet demand when it outpaces the size of the research team. Protects Researcher time — Enabling others to do some research leaves trained Researchers time for the most challenging, impactful problems. Brings rigor to others’ curiosity — Training others to do better research improves the quality of their learnings and the decisions they make from them. Protects customers — Providing ethics and data governance guidelines for those doing research protects customers and their data. Builds collaboration and awareness — Empowering others builds collaboration and awareness whereas claiming ownership inspires “rogue” actors.
- 20. 20 Builds empathy — Making insights and direct access to customers more accessible builds customer empathy. Increases bandwidth — Training team members to conduct research helps meet demand when it outpaces the size of the research team. Protects Researcher time — Enabling others to do some research leaves trained Researchers time for the most challenging, impactful problems. Brings rigor to others’ curiosity — Training others to do better research improves the quality of their learnings and the decisions they make from them. Protects customers — Providing ethics and data governance guidelines for those doing research protects customers and their data. Builds collaboration and awareness — Empowering others builds collaboration and awareness whereas claiming ownership inspires “rogue” actors.
- 22. 22 Not good for generative research — Non-Researchers aren’t trained to effectively analyze large amounts of data, and can leave insights “on the table”. Creates operational concerns — Democratization programs require resources for training, consultation, recruitment, tooling, and compliance. Takes more time than it’s worth — Democratization programs require significant Researcher time investment and may not deliver commensurate value. Leads to questionable quality — Those who only spend a short time learning about research won’t produce quality insights like trained Researchers. Creates insights silos — Non-Researchers rarely document findings, exacerbating challenges with access to and synthesis of research insights. “Cheapens” the research discipline — Democratization dilutes the skill set and value of the Research profession, propagating ideas that anyone can do our job.
- 23. 23 Not good for generative research — Non-Researchers aren’t trained to effectively analyze large amounts of data, and can leave insights “on the table”. Creates operational concerns — Democratization programs require resources for training, consultation, recruitment, tooling, and compliance. Takes more time than it’s worth — Democratization programs require significant Researcher time investment and may not deliver commensurate value. Leads to questionable quality — Those who only spend a short time learning about research won’t produce quality insights like trained Researchers. Creates insights silos — Non-Researchers rarely document findings, exacerbating challenges with access to and synthesis of research insights. “Cheapens” the research discipline — Democratization dilutes the skill set and value of the Research profession, propagating ideas that anyone can do our job.
- 24. 24 Not good for generative research — Non-Researchers aren’t trained to effectively analyze large amounts of data, and can leave insights “on the table”. Creates operational concerns — Democratization programs require resources for training, consultation, recruitment, tooling, and compliance. Takes more time than it’s worth — Democratization programs require significant Researcher time investment and may not deliver commensurate value. Leads to questionable quality — Those who only spend a short time learning about research won’t produce quality insights like trained Researchers. Creates insights silos — Non-Researchers rarely document findings, exacerbating challenges with access to and synthesis of research insights. “Cheapens” the research discipline — Democratization dilutes the skill set and value of the Research profession, propagating ideas that anyone can do our job.
- 25. 25 Not good for generative research — Non-Researchers aren’t trained to effectively analyze large amounts of data, and can leave insights “on the table”. Creates operational concerns — Democratization programs require resources for training, consultation, recruitment, tooling, and compliance. Takes more time than it’s worth — Democratization programs require significant Researcher time investment and may not deliver commensurate value. Leads to questionable quality — Those who only spend a short time learning about research won’t produce quality insights like trained Researchers. Creates insights silos — Non-Researchers rarely document findings, exacerbating challenges with access to and synthesis of research insights. “Cheapens” the research discipline — Democratization dilutes the skill set and value of the Research profession, propagating ideas that anyone can do our job.
- 26. 26 Not good for generative research — Non-Researchers aren’t trained to effectively analyze large amounts of data, and can leave insights “on the table”. Creates operational concerns — Democratization programs require resources for training, consultation, recruitment, tooling, and compliance. Takes more time than it’s worth — Democratization programs require significant Researcher time investment and may not deliver commensurate value. Leads to questionable quality — Those who only spend a short time learning about research won’t produce quality insights like trained Researchers. Creates insights silos — Non-Researchers rarely document findings, exacerbating challenges with access to and synthesis of research insights. “Cheapens” the research discipline — Democratization dilutes the skill set and value of the Research profession, propagating ideas that anyone can do our job.
- 27. 27 Not good for generative research — Non-Researchers aren’t trained to effectively analyze large amounts of data, and can leave insights “on the table”. Creates operational concerns — Democratization programs require resources for training, consultation, recruitment, tooling, and compliance. Takes more time than it’s worth — Democratization programs require significant Researcher time investment and may not deliver commensurate value. Leads to questionable quality — Those who only spend a short time learning about research won’t produce quality insights like trained Researchers. Creates insights silos — Non-Researchers rarely document findings, exacerbating challenges with access to and synthesis of research insights. “Cheapens” the research discipline — Democratization dilutes the skill set and value of the Research profession, propagating ideas that anyone can do our job.
- 28. 28 Not good for generative research — Non-Researchers aren’t trained to effectively analyze large amounts of data, and can leave insights “on the table”. Creates operational concerns — Democratization programs require resources for training, consultation, recruitment, tooling, and compliance. Takes more time than it’s worth — Democratization programs require significant Researcher time investment and may not deliver commensurate value. Leads to questionable quality — Those who only spend a short time learning about research won’t produce quality insights like trained Researchers. Creates insights silos — Non-Researchers rarely document findings, exacerbating challenges with access to and synthesis of research insights. “Cheapens” the research discipline — Democratization dilutes the skill set and value of the Research profession, propagating ideas that anyone can do our job.
- 29. Choosing our path at Verizon
- 30. 30 PROS CONS + Builds empathy + Increases bandwidth + Protects Researcher time + Builds collaboration and awareness + Brings rigor to others’ curiosity + Protects customers − Not good for generative research − Creates operational concerns − Takes more time than it’s worth − Creates insights silos − Leads to questionable quality − “Cheapens” the research discipline
- 31. 31 It’s not a binary decision. Research leaders need to think about the potential benefits and costs in the contexts of their organizations and decide what democratization looks like (or doesn’t) for them.
- 32. 32 PROS CONS + Builds empathy + Increases bandwidth ○ Large volume of iterative testing ○ 1 UX Researcher to about 26 Designers + Protects Researcher time ○ Increase bandwidth for more generative UX Research + Builds collaboration and awareness ○ Centralized Insights Organization ○ But easy for teams to reach out to vendors and go “rogue” + Brings rigor to others’ curiosity + Protects customers − Not good for generative research ○ Start small with clearly defined scope − Creates operational concerns ○ Existing ResearchOps Function − Takes more time than it’s worth ○ Balance oversight and Researcher time − Creates insights silos ○ Program process requires tracking alongside existing research trackers − Leads to questionable quality ○ Many Designers have research experience ○ Office Hour approval model ○ Templatized tests and analysis guides − “Cheapens” the research discipline ○ Clear, precise language ○ Contextualize within a broader UX Research program
- 33. 33 To start, our focus is democratizing ...research insights ...research experiences ...research tools ...research skills ...usability research ...generative research
- 35. 35 Spot Testing small, defined testing, not research CLEAR, PRECISE LANGUAGE
- 36. 36 Spot Testing is part of our wider UX Research Program Embedded Collaboration Ad Hoc Rapid Research Self-service Spot Testing CONTEXTUALIZE WITHIN A UXR PROGRAM
- 37. 37 Embedded Collaboration GOOD FOR: Generative or strategic projects, deep customer understanding, and ongoing research programs TIMELINE: Longer engagements, 2+ weeks depending on study HOW TO: Quarterly research prioritization through meetings with UXR portfolio Lead
- 38. 38 GOOD FOR: Well-defined, qualitative usability studies with 1-2 objectives TIMELINE: 1-2 week turnaround (depending on scope and audience) HOW TO: Submit a UXR brief to request a study—studies are prioritized every other Wednesday for kickoff the following Friday Ad Hoc Rapid Research
- 39. 39 GOOD FOR: Single objective, unmoderated tests that match an existing template TIMELINE: Quickest, 1-2 days depending on audience HOW TO: Trained Ambassadors create tests on-demand with custom templates, sign up for a consult during UXR-staffed Office Hours, launch test, and analyze results Self-service Spot Testing CLEARLY DEFINED SCOPE
- 40. 40 The Spot Testing Process Onboard to Program + Attend orientation + Take Basic Training + Admin sets up your UserTesting account 1 2 Setup Test + Create test on UserTesting Platform using provided templates 3 Submit Test for Review + Submit new Spot Test form + Attend Office Hours + UXR approves study for launch 4 5 Launch Pilot then Full Test + Pilot test with 1 participant + Launch Spot Test by adding participants as per training Analyze and Report Findings + Analyze results using guide + Submit final report form + Form shared with Portfolio UXRs
- 41. 41 The Spot Testing Process Onboard to Program + Attend orientation + Take Basic Training + Admin sets up your UserTesting account 1 2 Setup Test + Create test on UserTesting Platform using provided templates 3 Submit Test for Review + Submit new Spot Test form + Attend Office Hours + UXR approves study for launch 4 5 Launch Pilot then Full Test + Pilot test with 1 participant + Launch Spot Test by adding participants as per training Analyze and Report Findings + Analyze results using guide + Submit final report form + Form shared with Portfolio UXRs TEMPLATIZED PROCESS
- 42. 42 The Spot Testing Process Onboard to Program + Attend orientation + Take Basic Training + Admin sets up your UserTesting account 1 2 Setup Test + Create test on UserTesting Platform using provided templates 3 Submit Test for Review + Submit new Spot Test form + Attend Office Hours + UXR approves study for launch 4 5 Launch Pilot then Full Test + Pilot test with 1 participant + Launch Spot Test by adding participants as per training Analyze and Report Findings + Analyze results using guide + Submit final report form + Form shared with Portfolio UXRs TRACK ALONGSIDE OTHER RESEARCH
- 43. 43 Launch Approval Flow Attend Office Hours Spot Test approved for launch Spot Test assessment by UXR Designer edits offline and signs up for next Office Hours Spot Test on hold— UXR POC contacted Edits made in Office Hours Requires minor/ no revisions Requires moderate revisions Requires major revisions or not Spot Testing appropriate OFFICE HOUR APPROVAL MODEL
- 44. 44 Program Goal: Quick Approval and Launch Spot Test approved for launch Spot Test assessment by UXR Designer edits offline and signs up for next Office Hours Spot Test on hold— UXR POC contacted Edits made in Office Hours Requires minor/ no revisions Requires moderate revisions Requires major revisions or not Spot Testing appropriate Attend Office Hours BALANCE OVERSIGHT AND RESEARCHER TIME
- 45. In Summary
- 46. 46 Democratization is not a binary decision. Think about the potential benefits and costs in the contexts of your organization and make an intentional decision about what democratization does (or doesn’t) look like for you.
- 47. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
- 48. Appendix
- 49. 49 + Collum, M. The state of UX Research in 2020 + de la Nuez, A. Democratization of UX Insights: What Does This Really Mean? + Fuccella, J. Democracy or Tyranny in UX Research… Is there a Middle Ground? + Hammontree, M. The Future of UX Research + Kingman, K. Transforming strategic research capacity through democratization + McCloskey, M. Democratizing CX research: How our Design Team embraces human insights + Mitra, S.S. Undemocratising User Research + Ruiz, E.G. Scaling UX Research: how to train an army of (non)researchers + Sirjani, B. Democratization is our Job + Stevens, M. Escape Velocity: Finding Our Path in the Next Decade Referenced Work