SlideShare a Scribd company logo
UX in Action:
IBM Watson
#Utwebinar @UserTesting
Carol Smith
Sr Design Manager, IBM Watson
#UTwebinar
UserTesting’s on demand customer
insights platform enables companies
to create powerful products and
exceptional customer experiences.
We Partner With Thousands Of Customers
#UTwebinar
Across All Platforms, At All Stages Of Development
Watch	customers	interact	with:	
	
	
	
	
	
	
	
	
	
DESKTOP	 SMARTPHONE	 TABLET	
WEBSITES	 APPS	&	
PROGRAMS	
PROTOTYPES	&	
WIREFRAMES	
SURVEYS	 PHYSICAL		
PRODUCTS	
	
Anywhere people interact with your brand:
AT	HOME	 AT	A	STORE	 ON	THE	GO	
On any pla6orm or device:
#UTwebinar
Traditional
Testing
2 to 3 Weeks 2 to 3 Weeks1 Week
Results In Hours, Not Weeks. Save Time. Save Money.
#UTwebinar
Plan & Recruit Run Study
Analyze Data &
Write Report
1 Day1 Day
Analyze Data
& Iterate
Plan, Recruit
& Run Study
Within Sprint
2
Days
4-7
Weeks
Agenda
•  Welcome
•  IBM Design Thinking
•  Watson Design Team
•  Agile
•  Usability Testing
•  Q&A
Adam Rector
Webinar Manager, UserTesting
@iamadamrector
Carol Smith
Sr Design Manager, IBM Watson
#UTwebinar
@carologic
8	
		
Carol Smith
Sr Design Manager, IBM Watson
		
#UTwebinar
Carol Smith conducted her first usability study in 2000 and has been researching to
understand user needs and then advocating for improved designs on their behalf since
then. She has led projects for the US government, non-profits, and corporations in a wide
variety of industries. She currently manages a team of designers at IBM, creating and
improving Watson software products. She has presented over 70 talks and workshops
around the world and has collaborated with luminaries in the field on a variety of topics.
@carologic
Carol Smith
•  User Experience
•  Mix of consulting and in-house
•  Government, non-profits
and corps in many industries
•  Usability testing, research, interaction design
•  Web, software, mobile
•  MS, Human-Computer Interaction,
DePaul University, 2002
•  Sr. Design Manager at IBM Watson,
Pittsburgh, PA
•  My thoughts – not those of my employer
•  Twitter: @carologic
IBM
•  105 year old company “Big Blue”
•  Continuous reinvention
•  Patents: ATM, floppy disk, relational
database, SQL, UPC barcode
•  Over 170 countries
•  Nearly 380,000 employees
•  100’s of Designers
https://en.wikipedia.org/wiki/IBM and https://en.wikipedia.org/wiki/IBM_Selectric_typewriter
IBMDesignThinking
Human-centered outcomes at speed and scale
Understand people’s needs.
IBMDesignThinking
© 2016 IBM Corporation | IBM Confidential
Form intent.
IBMDesignThinking
© 2016 IBM Corporation | IBM Confidential
Deliver outcomes 
at speed and scale.
IBMDesignThinking
© 2016 IBM Corporation | IBM Confidential
IBMDesignThinking
© 2016 IBM Corporation | IBM Confidential
TheLoop
TheKeys
Principles
IBMDesignThinking
© 2016 IBM Corporation | IBM Confidential
TheLoop
Principles
TheKeys
A focus on user

outcomes
Multidisciplinary

teams
Restless

reinvention
IBMDesignThinking
© 2016 IBM Corporation | IBM Confidential
Principles
Principles
IBMDesignThinking
© 2016 IBM Corporation | IBM Confidential
TheLoop
TheKeys
IBMDesignThinking
TheLoop
Observe
 Reflect
 Make
© 2016 IBM Corporation | IBM Confidential
TheLoop
Principles
IBMDesignThinking
© 2016 IBM Corporation | IBM Confidential
TheKeys
Hills
 Playbacks
 Sponsor
users
IBMDesignThinking
© 2016 IBM Corporation | IBM Confidential
TheKeys
IBMDesignThinking
© 2016 IBM Corporation | IBM Confidential
TheLoop
TheKeys
Principles
Watson Designers and Guilds
Watson Design Team (~60 Designers)
Explore
& Discover (8)
Management
& Guild Leads
Director
Watson
Design
Director
Carol
Researchers Ix Designers
Viz
Designers
Staffing
•  Balance of skills and experience
•  Provide mentoring and guidance to less experienced folks
•  Best designers working on toughest problems
•  Focus on skill set
•  Enable team to do what they enjoy and practice skill
•  Encourage shadowing of others to learn new skills
•  Encourage depth and T shape
The Work
•  Individuals focus on one primary product/problem at a time
•  Secondary short projects
•  10% time is available
•  Tuesdays are ‘No Meeting’ day
•  Management facing constant change
•  Multiple projects
•  Customers around the world
•  Big challenges
Remote Research
•  Interviews
•  Observations
•  Limited, but helpful
•  Usability testing
•  Prototypes and working software
•  Moderated through screen shares
•  Unmoderated with usertesting.com
•  Card Sorting and/or co-creation activities (limited)
Ask for artifacts!
hHp://www.flickr.com/photos/camknows/						via						hHp://creaOvecommons.org/licenses/by-nc-sa/2.0/
In Person Research
•  At customer’s location
•  With sales or product owners (Offering Managers)
•  Initial meetings are typically to gain trust
•  Return to do additional work
•  Interviews
•  Observations
•  Usability Testing
•  Co-Creation activities
•  Limitations
•  Recording can be very difficult
•  Access only to whom they approve
Photo: http://koin.com/2016/07/12/how-to-protect-yourself-from-computer-camera-hackers/
Security
Conferences and Events
•  World of Watson
•  Design Studio Sessions
•  Listed as Feedback Sessions
•  Use social media and signage
to attract attendees
•  Primary Goal: Meet many
customers and build
relationships for future work.
•  Secondary Goal: Learn as much as possible through research
activities.
Workshops
•  Start with Design and product owners (Offering Management)
•  Incubator phase
•  Create Concept Cars and determine value
•  Review with Development teams
•  Collaboration sessions
•  Design, Development and Offering Management (QA and Content as well)
Remote Work and Collaboration
•  Screen sharing
•  SmartCloud, WebEx, GoToMeeting, etc.
•  Sharing ideas
•  Virtual post-its - Mural.ly
•  Real-time co-editing (Google Docs, Box)
•  Document cameras (higher quality)
This is only a small sample of available tools.
Endorsement is not implied by inclusion here.
Co-located and international teams
•  Communication
•  Skype, Google Hangout, Appear.in
•  Chatting
•  Slack
•  WhatsApp desktop version
This is only a small sample of available tools.
Endorsement is not implied by inclusion here.
IBM Design Thinking and Agile
•  Emphasize focus on:
•  Team collaboration
•  Continuous learning
•  Outcomes
•  Sprints or Kanban
•  3 week cadence
•  Demos near end
•  Daily squad standups
•  Weekly cross-team reviews
•  Usability test should take no more than 2 sprints to complete
Agile Manifesto
Full version: http://agilemanifesto.org/
Individuals and Interactions Customer Collaboration
over processes and tools over contract negotiation
Working Software Responding to Change
over comprehensive documentation over following a plan
We have come to value the items in the dark boxes more.
Agile and Prioritizing Work
•  Things move very quickly
•  Constant re-prioritization of work
•  Offering Management sets overall priority (fairly stable)
•  Teams pick what to work on
•  Dependencies
•  What we learn through research
•  What we learn through dev process
•  Time available (if no time to implement, don’t do the research)
•  Staff availability (other projects, personal leave, etc.)
•  Customer availability
Agile Frustrations via Spotify
Scaling Agile at Spotify via Slideshare of Vlad Mysla
http://www.slideshare.net/vmysla/scrum-at-spotify?qid=2345c3ad-7e68-4383-9673-9e715ff47a75&v=default&b=&from_search=14
squad member
Agile UX: The Good
•  Most important functionality is done first
•  Working together – not “over the wall”
•  Keep up with technology and environmental changes
•  Enables iteration of requirements
•  Less “design drift” and less wasted design
Agile UX: The Good (continued)
•  Issues get fixed
•  “Done” includes design
•  Satisfying to see designs in real use
•  Learn from actual product use
•  User data has effect on current release
Be Part of the Team
•  They’re not going to “stop the train” for you
•  Make UX processes Agile
•  Build trust by providing value of work
•  Attend daily standups/scrums
Design Adapts to Agile
•  Falling short of end goals is a constant
•  Focus on meeting user’s primary needs
•  Constant Improvement is key
•  Widen the Design family
•  Distribute the work
•  Empower entire squad to inform design
•  Agile is reactive – no time for predictive work
•  Prepare to react
Adapted from Jim Laing’s presentation at UX Pittsburgh, May 2014
Moderated Usability Testing
•  Everyone is invited to observe
•  Separate viewing room and project to a screen
•  Development, offering management, QA, content, etc.
•  Box note to track progress within Design team
•  Slack channel for larger group discussion and to ask questions
•  Discussions start during the study
•  Team gets excited – especially those that are new to usability testing
•  I have mixed emotions – sometimes wrong emphasis - but at least they
are communicating!
Moderated Usability Testing
•  Raise and/or discover significant issues through usability testing
•  Hard to argue with what you see
•  Have to more carefully vet our participants when testing high visibility
functionality
•  Have had situations with intense scrutiny – which we welcome!
•  Create highlight reels of most important topics
•  Optional editing – frequently we tell them at what time marker to watch
Unmoderated Usability testing
•  UserTesting.com
•  Clickable prototypes and working software
•  IBM Bluemix for hosting
•  One participant completed and watched before inviting more
•  Find issues with study design
•  Determine if getting desired results
•  If not, edit and re-start
•  Small studies
•  Up to 5 participants
•  Review and repeat as needed
Remote Tool Considerations
•  Prototype
•  Must be online (impossible to test if not online)
•  Document or clickable site/prototype
•  How is recruiting handled?
•  How selective can I be?
•  Can I provide my own list?
•  Limited number of observers?
•  Limited number of participants?
44
Remote Tool Considerations (cont)
•  Typically only one facilitator
•  Logging and video editing needs met?
•  Time on task, highlight video creation
•  Ease of use
•  Able to provide surveys before or after?
45
Towards Building Teams
•  Hire great people
•  Balance skills and experience
•  Enable to work to their strengths
•  Encourage to grow, attend webinars, conferences, etc.
•  Make time for 1:1’s
Questions & Answers
#UTwebinar
		
Q A
Carol Smith
	
in/CarolJSmith	
	
@Carologic	
	
slideshare.net/carologic
Shared Documents Aren’t Shared Understanding
Cartoon	by	Luke	BarreH			©	Jeff	PaHon,	all	rights	reserved,	www.AgileProductDesign.com
IBM Health Corps
•  Three weeks, completed entire software project
•  Partnered with American Cancer Society
•  Built a tool to help Ministries of Health in low-income countries get
access to more complete and cheaper therapies for cancer.
Blog: https://ibmhealthcorpsblog.wordpress.com/2016/09/29/delivering-high-value-in-a-short-time-colocation-agile-and-design-thinking/
Video: https://www.instagram.com/p/BK9N5K_jEN_/?taken-by=ibmhealthcorps
World Community Grid – unused computing
Web Site: https://www.worldcommunitygrid.org/discover.action
More about Watson
•  CBS News, 60 Minutes on AI
Video: http://www.cbsnews.com/news/60-minutes-artificial-intelligence-charlie-rose-robot-sofia/

More Related Content

What's hot (10)

PDF
Product Development for startup
Aslam Hirani
 
PDF
How to Run Research in Agile Sprints by Democratizing It Across Teams
UserZoom
 
PDF
Telling Your UX Metrics Story - The 21st Century Metrics Model
UserZoom
 
PDF
Scrum - Product Owner
Márcio Oya
 
PDF
Agile Tour 2016 Pune - Innovation vs iteration by Kanchan Tandel
India Scrum Enthusiasts Community
 
PPTX
Exploring UX in the Enterprise: The Industry’s Hottest Trends & Insights from...
UserZoom
 
PDF
Why should I care about the Minimum Viable Product (MVP)
Nektarios Sylligardakis
 
PPTX
Just do it
Steve Owens
 
PDF
Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...
UserZoom
 
PPTX
UX in a Dual Track Agile World
Andrea Neuhoff
 
Product Development for startup
Aslam Hirani
 
How to Run Research in Agile Sprints by Democratizing It Across Teams
UserZoom
 
Telling Your UX Metrics Story - The 21st Century Metrics Model
UserZoom
 
Scrum - Product Owner
Márcio Oya
 
Agile Tour 2016 Pune - Innovation vs iteration by Kanchan Tandel
India Scrum Enthusiasts Community
 
Exploring UX in the Enterprise: The Industry’s Hottest Trends & Insights from...
UserZoom
 
Why should I care about the Minimum Viable Product (MVP)
Nektarios Sylligardakis
 
Just do it
Steve Owens
 
Benchmarking Mini-series Part #2: Conducting Quick, Cost-Effective UX Benchma...
UserZoom
 
UX in a Dual Track Agile World
Andrea Neuhoff
 

Viewers also liked (20)

PPTX
RWorks Management Information
Tony Redmond
 
PDF
Google design
Chih-cheng Wang
 
PDF
Product development at NerdWallet: Why user research is pivotal
UserTesting
 
PDF
Google Design Sprinting Webinar
Rafiq Elmansy
 
PDF
Ibm watson - how it works, and what it means for society beyond winning jeo...
Rick Bouter
 
PDF
Emotional Design for Google Launchpad
Borrys Hasian
 
PDF
How To Run Your Own Guerrilla Usability Testing
jtcchan
 
PDF
What Will You Build with IBM Watson on Bluemix?
Swami Chandrasekaran
 
PDF
Beyond Keyword Search with IBM Watson Explorer Webinar Deck
MC+A
 
PDF
Bring IBM Watson to your telephone
Brian Pulito
 
PPTX
IBM Watson Work Services Development
Van Staub, MBA
 
PDF
IBM Watson - Cognitive Robots
Jouko Poutanen
 
PDF
Putting IBM Watson to Work.. Saxena
Manoj Saxena
 
PDF
Agile and continuous delivery – How IBM Watson Workspace is built
Vincent Burckhardt
 
DOCX
Modelo completo de contrato de compra e venda de veículo alienado
Dieison Moraes
 
PDF
IBM Watson in Healthcare
Anders Quitzau
 
PDF
Introduction to the IBM Watson Data Platform
Margriet Groenendijk
 
PDF
Solving Design Problem in 2.5 Hours with Google Design Sprint
Borrys Hasian
 
PDF
IBM Watson & Cognitive Computing - Tech In Asia 2016
Nugroho Gito
 
PDF
IBM Watson Question-Answering System and Cognitive Computing
Rakuten Group, Inc.
 
RWorks Management Information
Tony Redmond
 
Google design
Chih-cheng Wang
 
Product development at NerdWallet: Why user research is pivotal
UserTesting
 
Google Design Sprinting Webinar
Rafiq Elmansy
 
Ibm watson - how it works, and what it means for society beyond winning jeo...
Rick Bouter
 
Emotional Design for Google Launchpad
Borrys Hasian
 
How To Run Your Own Guerrilla Usability Testing
jtcchan
 
What Will You Build with IBM Watson on Bluemix?
Swami Chandrasekaran
 
Beyond Keyword Search with IBM Watson Explorer Webinar Deck
MC+A
 
Bring IBM Watson to your telephone
Brian Pulito
 
IBM Watson Work Services Development
Van Staub, MBA
 
IBM Watson - Cognitive Robots
Jouko Poutanen
 
Putting IBM Watson to Work.. Saxena
Manoj Saxena
 
Agile and continuous delivery – How IBM Watson Workspace is built
Vincent Burckhardt
 
Modelo completo de contrato de compra e venda de veículo alienado
Dieison Moraes
 
IBM Watson in Healthcare
Anders Quitzau
 
Introduction to the IBM Watson Data Platform
Margriet Groenendijk
 
Solving Design Problem in 2.5 Hours with Google Design Sprint
Borrys Hasian
 
IBM Watson & Cognitive Computing - Tech In Asia 2016
Nugroho Gito
 
IBM Watson Question-Answering System and Cognitive Computing
Rakuten Group, Inc.
 
Ad

Similar to UX in Action: IBM Watson (20)

PPTX
Making Faster UX in an Agile World - HOAPitt 2017
Carol Smith
 
PPT
How to Integrate UX and Agile
UserZoom
 
PPTX
Winning hearts and minds: how to embed UX from scratch in a large organisation
Michele Ide-Smith
 
PPTX
User experience@TechSmith
Barb Hernandez
 
PPTX
Easy & Effective Usability Testing at CodeMash 2012
Carol Smith
 
PPT
Interaction Design ch9 notes
Noel Kehoe
 
PPTX
The Importance of UX
Bart Van Hecke
 
PPTX
Mobile Best Practices for UX
The Wisdom Daily
 
PPTX
Faster Usability Testing in an Agile World presented at Agile2011
Carol Smith
 
PPTX
UX & Design Thinking for BI Applications
ibi
 
PPTX
Designing Better Experiences
Matt Gibson
 
PDF
Remote Moderated Usability Testing & Tools
Susan Price
 
PDF
UXPABOS2013_FABRIZI
Jennifer Fabrizi
 
PDF
2012 Usability Conference
atlanticbtinc
 
PDF
Introduction to User Experience - Mike Biggs
Thoughtworks
 
PDF
Demystifying User Experience - General Assembly
Mike Biggs GAICD
 
PPTX
Integrating User Centered Design with Agile Development
Julia Borkenhagen
 
PDF
UX (User Experience) Process, May 2017
Gary Coker
 
PPTX
User Experience: Research, Design, Process, and Workflow
sollitaire
 
PDF
UXI Matrix (Jon Innes)
Balanced Team
 
Making Faster UX in an Agile World - HOAPitt 2017
Carol Smith
 
How to Integrate UX and Agile
UserZoom
 
Winning hearts and minds: how to embed UX from scratch in a large organisation
Michele Ide-Smith
 
User experience@TechSmith
Barb Hernandez
 
Easy & Effective Usability Testing at CodeMash 2012
Carol Smith
 
Interaction Design ch9 notes
Noel Kehoe
 
The Importance of UX
Bart Van Hecke
 
Mobile Best Practices for UX
The Wisdom Daily
 
Faster Usability Testing in an Agile World presented at Agile2011
Carol Smith
 
UX & Design Thinking for BI Applications
ibi
 
Designing Better Experiences
Matt Gibson
 
Remote Moderated Usability Testing & Tools
Susan Price
 
UXPABOS2013_FABRIZI
Jennifer Fabrizi
 
2012 Usability Conference
atlanticbtinc
 
Introduction to User Experience - Mike Biggs
Thoughtworks
 
Demystifying User Experience - General Assembly
Mike Biggs GAICD
 
Integrating User Centered Design with Agile Development
Julia Borkenhagen
 
UX (User Experience) Process, May 2017
Gary Coker
 
User Experience: Research, Design, Process, and Workflow
sollitaire
 
UXI Matrix (Jon Innes)
Balanced Team
 
Ad

More from UserTesting (20)

PPTX
Live Webinar: Healthcare Chatbot Diagnosis
UserTesting
 
PPTX
CX Survival Guide for 2019
UserTesting
 
PPTX
Marketing Edition: How we leverage UserTesting
UserTesting
 
PPTX
Product Edition: How we leverage UserTesting
UserTesting
 
PPTX
A simple method to make better CX decisions
UserTesting
 
PPTX
Live Conversation: Connecting with customers in real time
UserTesting
 
PPTX
What's with Apple's latest iPhone ads?
UserTesting
 
PPTX
Actionable Results from UX Research
UserTesting
 
PPTX
A Webinar with UserTesting: Orchestrating Experiences
UserTesting
 
PPTX
Creating great customer journeys through customer interviews: Real-world advi...
UserTesting
 
PPTX
The Streaming Media CX Index: What customers expect from SVOD experiences
UserTesting
 
PPTX
Three Ways Fast Human Insight is Revolutionizing Marketing
UserTesting
 
PPTX
3 Digital Transformation Strategies Driving CX
UserTesting
 
PPTX
CX goes mainstream: Five trends driving the future of CX
UserTesting
 
PPTX
How human insights focused organizations become CX leaders
UserTesting
 
PPTX
The Banking Mobile CX Index: Insights to improve the mobile banking experience
UserTesting
 
PPTX
Nordstrom Rack | Hautelook: Building a Customer-Centered Culture
UserTesting
 
PPTX
Insights on 2017 cx trends and 2018 predictions webinar
UserTesting
 
PPTX
Live Conversation: Cut your customer interview costs by up to 90%
UserTesting
 
PDF
Introducing Live Conversation | Human Insight on Demand
UserTesting
 
Live Webinar: Healthcare Chatbot Diagnosis
UserTesting
 
CX Survival Guide for 2019
UserTesting
 
Marketing Edition: How we leverage UserTesting
UserTesting
 
Product Edition: How we leverage UserTesting
UserTesting
 
A simple method to make better CX decisions
UserTesting
 
Live Conversation: Connecting with customers in real time
UserTesting
 
What's with Apple's latest iPhone ads?
UserTesting
 
Actionable Results from UX Research
UserTesting
 
A Webinar with UserTesting: Orchestrating Experiences
UserTesting
 
Creating great customer journeys through customer interviews: Real-world advi...
UserTesting
 
The Streaming Media CX Index: What customers expect from SVOD experiences
UserTesting
 
Three Ways Fast Human Insight is Revolutionizing Marketing
UserTesting
 
3 Digital Transformation Strategies Driving CX
UserTesting
 
CX goes mainstream: Five trends driving the future of CX
UserTesting
 
How human insights focused organizations become CX leaders
UserTesting
 
The Banking Mobile CX Index: Insights to improve the mobile banking experience
UserTesting
 
Nordstrom Rack | Hautelook: Building a Customer-Centered Culture
UserTesting
 
Insights on 2017 cx trends and 2018 predictions webinar
UserTesting
 
Live Conversation: Cut your customer interview costs by up to 90%
UserTesting
 
Introducing Live Conversation | Human Insight on Demand
UserTesting
 

Recently uploaded (20)

PPTX
How tech helps people in the modern era.
upadhyayaryan154
 
PDF
The Internet of Things (IoT) refers to a vast network of interconnected devic...
chethana8182
 
PPTX
The Latest Scam Shocking the USA in 2025.pptx
onlinescamreport4
 
PDF
GEO Strategy 2025: Complete Presentation Deck for AI-Powered Customer Acquisi...
Zam Man
 
PDF
Data Protection & Resilience in Focus.pdf
AmyPoblete3
 
PPTX
Perkembangan Perangkat jaringan komputer dan telekomunikasi 3.pptx
Prayudha3
 
PPTX
dns domain name system history work.pptx
MUHAMMADKAVISHSHABAN
 
PPTX
B2B_Ecommerce_Internship_Simranpreet.pptx
LipakshiJindal
 
PDF
Cybersecurity Awareness Presentation ppt.
banodhaharshita
 
PDF
The AI Trust Gap: Consumer Attitudes to AI-Generated Content
Exploding Topics
 
PPT
Introduction to dns domain name syst.ppt
MUHAMMADKAVISHSHABAN
 
PDF
How Much GB RAM Do You Need for Coding? 5 Powerful Reasons 8GB Is More Than E...
freeshopbudget
 
PPT
1965 INDO PAK WAR which Pak will never forget.ppt
sanjaychief112
 
PPTX
办理方法西班牙假毕业证蒙德拉贡大学成绩单MULetter文凭样本
xxxihn4u
 
PPTX
MSadfadsfafdadfccadradfT_Presentation.pptx
pahalaedward2
 
PPTX
Pengenalan perangkat Jaringan komputer pada teknik jaringan komputer dan tele...
Prayudha3
 
PDF
LOGENVIDAD DANNYFGRETRRTTRRRTRRRRRRRRR.pdf
juan456ytpro
 
PDF
UI/UX Developer Guide: Tools, Trends, and Tips for 2025
Penguin peak
 
DOCX
An_Operating_System by chidi kingsley wo
kingsleywokocha4
 
PPTX
原版北不列颠哥伦比亚大学毕业证文凭UNBC成绩单2025年新版在线制作学位证书
e7nw4o4
 
How tech helps people in the modern era.
upadhyayaryan154
 
The Internet of Things (IoT) refers to a vast network of interconnected devic...
chethana8182
 
The Latest Scam Shocking the USA in 2025.pptx
onlinescamreport4
 
GEO Strategy 2025: Complete Presentation Deck for AI-Powered Customer Acquisi...
Zam Man
 
Data Protection & Resilience in Focus.pdf
AmyPoblete3
 
Perkembangan Perangkat jaringan komputer dan telekomunikasi 3.pptx
Prayudha3
 
dns domain name system history work.pptx
MUHAMMADKAVISHSHABAN
 
B2B_Ecommerce_Internship_Simranpreet.pptx
LipakshiJindal
 
Cybersecurity Awareness Presentation ppt.
banodhaharshita
 
The AI Trust Gap: Consumer Attitudes to AI-Generated Content
Exploding Topics
 
Introduction to dns domain name syst.ppt
MUHAMMADKAVISHSHABAN
 
How Much GB RAM Do You Need for Coding? 5 Powerful Reasons 8GB Is More Than E...
freeshopbudget
 
1965 INDO PAK WAR which Pak will never forget.ppt
sanjaychief112
 
办理方法西班牙假毕业证蒙德拉贡大学成绩单MULetter文凭样本
xxxihn4u
 
MSadfadsfafdadfccadradfT_Presentation.pptx
pahalaedward2
 
Pengenalan perangkat Jaringan komputer pada teknik jaringan komputer dan tele...
Prayudha3
 
LOGENVIDAD DANNYFGRETRRTTRRRTRRRRRRRRR.pdf
juan456ytpro
 
UI/UX Developer Guide: Tools, Trends, and Tips for 2025
Penguin peak
 
An_Operating_System by chidi kingsley wo
kingsleywokocha4
 
原版北不列颠哥伦比亚大学毕业证文凭UNBC成绩单2025年新版在线制作学位证书
e7nw4o4
 

UX in Action: IBM Watson

  • 1. UX in Action: IBM Watson #Utwebinar @UserTesting Carol Smith Sr Design Manager, IBM Watson
  • 2. #UTwebinar UserTesting’s on demand customer insights platform enables companies to create powerful products and exceptional customer experiences.
  • 3. We Partner With Thousands Of Customers #UTwebinar
  • 4. Across All Platforms, At All Stages Of Development Watch customers interact with: DESKTOP SMARTPHONE TABLET WEBSITES APPS & PROGRAMS PROTOTYPES & WIREFRAMES SURVEYS PHYSICAL PRODUCTS Anywhere people interact with your brand: AT HOME AT A STORE ON THE GO On any pla6orm or device: #UTwebinar
  • 5. Traditional Testing 2 to 3 Weeks 2 to 3 Weeks1 Week Results In Hours, Not Weeks. Save Time. Save Money. #UTwebinar Plan & Recruit Run Study Analyze Data & Write Report 1 Day1 Day Analyze Data & Iterate Plan, Recruit & Run Study Within Sprint 2 Days 4-7 Weeks
  • 6. Agenda •  Welcome •  IBM Design Thinking •  Watson Design Team •  Agile •  Usability Testing •  Q&A Adam Rector Webinar Manager, UserTesting @iamadamrector Carol Smith Sr Design Manager, IBM Watson #UTwebinar @carologic
  • 7. 8 Carol Smith Sr Design Manager, IBM Watson #UTwebinar Carol Smith conducted her first usability study in 2000 and has been researching to understand user needs and then advocating for improved designs on their behalf since then. She has led projects for the US government, non-profits, and corporations in a wide variety of industries. She currently manages a team of designers at IBM, creating and improving Watson software products. She has presented over 70 talks and workshops around the world and has collaborated with luminaries in the field on a variety of topics. @carologic
  • 8. Carol Smith •  User Experience •  Mix of consulting and in-house •  Government, non-profits and corps in many industries •  Usability testing, research, interaction design •  Web, software, mobile •  MS, Human-Computer Interaction, DePaul University, 2002 •  Sr. Design Manager at IBM Watson, Pittsburgh, PA •  My thoughts – not those of my employer •  Twitter: @carologic
  • 9. IBM •  105 year old company “Big Blue” •  Continuous reinvention •  Patents: ATM, floppy disk, relational database, SQL, UPC barcode •  Over 170 countries •  Nearly 380,000 employees •  100’s of Designers https://en.wikipedia.org/wiki/IBM and https://en.wikipedia.org/wiki/IBM_Selectric_typewriter
  • 11. Understand people’s needs. IBMDesignThinking © 2016 IBM Corporation | IBM Confidential
  • 12. Form intent. IBMDesignThinking © 2016 IBM Corporation | IBM Confidential
  • 13. Deliver outcomes at speed and scale. IBMDesignThinking © 2016 IBM Corporation | IBM Confidential
  • 14. IBMDesignThinking © 2016 IBM Corporation | IBM Confidential TheLoop TheKeys Principles
  • 15. IBMDesignThinking © 2016 IBM Corporation | IBM Confidential TheLoop Principles TheKeys
  • 16. A focus on user
 outcomes Multidisciplinary
 teams Restless
 reinvention IBMDesignThinking © 2016 IBM Corporation | IBM Confidential Principles
  • 17. Principles IBMDesignThinking © 2016 IBM Corporation | IBM Confidential TheLoop TheKeys
  • 18. IBMDesignThinking TheLoop Observe Reflect Make © 2016 IBM Corporation | IBM Confidential
  • 19. TheLoop Principles IBMDesignThinking © 2016 IBM Corporation | IBM Confidential TheKeys
  • 20. Hills Playbacks Sponsor users IBMDesignThinking © 2016 IBM Corporation | IBM Confidential TheKeys
  • 21. IBMDesignThinking © 2016 IBM Corporation | IBM Confidential TheLoop TheKeys Principles
  • 23. Watson Design Team (~60 Designers) Explore & Discover (8) Management & Guild Leads Director Watson Design Director Carol Researchers Ix Designers Viz Designers
  • 24. Staffing •  Balance of skills and experience •  Provide mentoring and guidance to less experienced folks •  Best designers working on toughest problems •  Focus on skill set •  Enable team to do what they enjoy and practice skill •  Encourage shadowing of others to learn new skills •  Encourage depth and T shape
  • 25. The Work •  Individuals focus on one primary product/problem at a time •  Secondary short projects •  10% time is available •  Tuesdays are ‘No Meeting’ day •  Management facing constant change •  Multiple projects •  Customers around the world •  Big challenges
  • 26. Remote Research •  Interviews •  Observations •  Limited, but helpful •  Usability testing •  Prototypes and working software •  Moderated through screen shares •  Unmoderated with usertesting.com •  Card Sorting and/or co-creation activities (limited)
  • 28. In Person Research •  At customer’s location •  With sales or product owners (Offering Managers) •  Initial meetings are typically to gain trust •  Return to do additional work •  Interviews •  Observations •  Usability Testing •  Co-Creation activities •  Limitations •  Recording can be very difficult •  Access only to whom they approve Photo: http://koin.com/2016/07/12/how-to-protect-yourself-from-computer-camera-hackers/ Security
  • 29. Conferences and Events •  World of Watson •  Design Studio Sessions •  Listed as Feedback Sessions •  Use social media and signage to attract attendees •  Primary Goal: Meet many customers and build relationships for future work. •  Secondary Goal: Learn as much as possible through research activities.
  • 30. Workshops •  Start with Design and product owners (Offering Management) •  Incubator phase •  Create Concept Cars and determine value •  Review with Development teams •  Collaboration sessions •  Design, Development and Offering Management (QA and Content as well)
  • 31. Remote Work and Collaboration •  Screen sharing •  SmartCloud, WebEx, GoToMeeting, etc. •  Sharing ideas •  Virtual post-its - Mural.ly •  Real-time co-editing (Google Docs, Box) •  Document cameras (higher quality) This is only a small sample of available tools. Endorsement is not implied by inclusion here.
  • 32. Co-located and international teams •  Communication •  Skype, Google Hangout, Appear.in •  Chatting •  Slack •  WhatsApp desktop version This is only a small sample of available tools. Endorsement is not implied by inclusion here.
  • 33. IBM Design Thinking and Agile •  Emphasize focus on: •  Team collaboration •  Continuous learning •  Outcomes •  Sprints or Kanban •  3 week cadence •  Demos near end •  Daily squad standups •  Weekly cross-team reviews •  Usability test should take no more than 2 sprints to complete
  • 34. Agile Manifesto Full version: http://agilemanifesto.org/ Individuals and Interactions Customer Collaboration over processes and tools over contract negotiation Working Software Responding to Change over comprehensive documentation over following a plan We have come to value the items in the dark boxes more.
  • 35. Agile and Prioritizing Work •  Things move very quickly •  Constant re-prioritization of work •  Offering Management sets overall priority (fairly stable) •  Teams pick what to work on •  Dependencies •  What we learn through research •  What we learn through dev process •  Time available (if no time to implement, don’t do the research) •  Staff availability (other projects, personal leave, etc.) •  Customer availability
  • 36. Agile Frustrations via Spotify Scaling Agile at Spotify via Slideshare of Vlad Mysla http://www.slideshare.net/vmysla/scrum-at-spotify?qid=2345c3ad-7e68-4383-9673-9e715ff47a75&v=default&b=&from_search=14 squad member
  • 37. Agile UX: The Good •  Most important functionality is done first •  Working together – not “over the wall” •  Keep up with technology and environmental changes •  Enables iteration of requirements •  Less “design drift” and less wasted design
  • 38. Agile UX: The Good (continued) •  Issues get fixed •  “Done” includes design •  Satisfying to see designs in real use •  Learn from actual product use •  User data has effect on current release
  • 39. Be Part of the Team •  They’re not going to “stop the train” for you •  Make UX processes Agile •  Build trust by providing value of work •  Attend daily standups/scrums
  • 40. Design Adapts to Agile •  Falling short of end goals is a constant •  Focus on meeting user’s primary needs •  Constant Improvement is key •  Widen the Design family •  Distribute the work •  Empower entire squad to inform design •  Agile is reactive – no time for predictive work •  Prepare to react Adapted from Jim Laing’s presentation at UX Pittsburgh, May 2014
  • 41. Moderated Usability Testing •  Everyone is invited to observe •  Separate viewing room and project to a screen •  Development, offering management, QA, content, etc. •  Box note to track progress within Design team •  Slack channel for larger group discussion and to ask questions •  Discussions start during the study •  Team gets excited – especially those that are new to usability testing •  I have mixed emotions – sometimes wrong emphasis - but at least they are communicating!
  • 42. Moderated Usability Testing •  Raise and/or discover significant issues through usability testing •  Hard to argue with what you see •  Have to more carefully vet our participants when testing high visibility functionality •  Have had situations with intense scrutiny – which we welcome! •  Create highlight reels of most important topics •  Optional editing – frequently we tell them at what time marker to watch
  • 43. Unmoderated Usability testing •  UserTesting.com •  Clickable prototypes and working software •  IBM Bluemix for hosting •  One participant completed and watched before inviting more •  Find issues with study design •  Determine if getting desired results •  If not, edit and re-start •  Small studies •  Up to 5 participants •  Review and repeat as needed
  • 44. Remote Tool Considerations •  Prototype •  Must be online (impossible to test if not online) •  Document or clickable site/prototype •  How is recruiting handled? •  How selective can I be? •  Can I provide my own list? •  Limited number of observers? •  Limited number of participants? 44
  • 45. Remote Tool Considerations (cont) •  Typically only one facilitator •  Logging and video editing needs met? •  Time on task, highlight video creation •  Ease of use •  Able to provide surveys before or after? 45
  • 46. Towards Building Teams •  Hire great people •  Balance skills and experience •  Enable to work to their strengths •  Encourage to grow, attend webinars, conferences, etc. •  Make time for 1:1’s
  • 49. Shared Documents Aren’t Shared Understanding Cartoon by Luke BarreH © Jeff PaHon, all rights reserved, www.AgileProductDesign.com
  • 50. IBM Health Corps •  Three weeks, completed entire software project •  Partnered with American Cancer Society •  Built a tool to help Ministries of Health in low-income countries get access to more complete and cheaper therapies for cancer. Blog: https://ibmhealthcorpsblog.wordpress.com/2016/09/29/delivering-high-value-in-a-short-time-colocation-agile-and-design-thinking/ Video: https://www.instagram.com/p/BK9N5K_jEN_/?taken-by=ibmhealthcorps
  • 51. World Community Grid – unused computing Web Site: https://www.worldcommunitygrid.org/discover.action
  • 52. More about Watson •  CBS News, 60 Minutes on AI Video: http://www.cbsnews.com/news/60-minutes-artificial-intelligence-charlie-rose-robot-sofia/