Usability modeling and measurement
Download as PPTX, PDF11 likes5,818 views
This document discusses usability modeling and measurement. It defines usability and explains why it is important, especially for web and mobile applications. It describes various usability models and standards. The document outlines how to develop a usability model for an organization by defining measurable attributes and characteristics. It also discusses different usability evaluation methods like surveys, heuristic evaluation, and logging. The key steps are to define a model, identify attributes to measure, collect data, analyze the results, and refine measurements. Taking these steps allows an organization to systematically evaluate and improve usability.
![Usability Model Comparisons
Seffah [33]](https://image.slidesharecdn.com/usabilitymodelingandmeasurement-120114172351-phpapp01/85/Usability-modeling-and-measurement-11-320.jpg)
Usability modeling and measurement
- 1. Usability Modeling and Measurement Philip Lew www.xbosoft.com
- 2. Agenda • Introduction – The importance of usability – Specifics for the web / mobile • What is Usability? • Usability Modeling and Measurements • Case study • Conclusion • Next Steps
- 4. Why is Usability Important? • Usability is important especially for web applications (SaaS, and websites) – And now especially mobile applications • Most prevalent development delivery model today • Without good usability: – Users will leave the applications – For mobile, if they can’t learn in 30 seconds, they won’t come back
- 5. Web and Mobile have Changed the Terrain • Business models have changed – Instead of paying upfront and ‘owning’ the software – Pay as you go, pay by subscription • Behavior and expectations have changed
- 6. Many Alternatives • Depending on the type of application – Users could leave and go elsewhere – Simply not use your mobile app (if only providing alternative access to a main application)
- 7. Thin Client Deployment Most Prevalent Development Delivery Model Today • Mobile and Cloud are singing the same song • Service delivery versus product delivery
- 8. What is Usability In relation to quality In relation to user experience
- 9. General Thoughts on Usability • Understandability • Learnability • Operability • Attractiveness • Navigation • Responsiveness-performance • Efficiency
- 10. Usability in Quality Standards • How is usability Standard Definition defined? ISO 9126-1 The capability of the software product to be – Standard (2000) understood, learned, used, definitions and attractive to the user, when used under specified – Many others conditions. who say ISO 9241-11 The extent to which a (1998) product can be used by similar specified users to achieve specified goals with things effectiveness, efficiency, • Usability and satisfaction in a specified context of use. – Component IEEE 600.12 The ease with which a user can learn to operate, of quality as (1990) prepare inputs for, and listed in interpret outputs of a system or component. many standards
- 11. Usability Model Comparisons Seffah [33]
- 12. Current Research / Usability Standards Kappel et al., The Discipline of Systematic Development of Web Applications, 2003, John Wiley and Sons.
- 13. Current Research – Usability Standards Quality Satisfaction Usability ISO 9241-11 ISO 9126-1 IEEE 610 ISO IEEE XXXY 25010
- 14. Bigger Picture Quality From ISO point of view Usability Usability CMMI ISO 25010
- 15. Usability as a Key Characteristic of Product Quality Source: ISO 25010
- 16. What is Usability-Effect of the Software Product Degree to which specified users can achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use. Source: ISO 25010
- 17. What is Usability-Effect In Actual Usage • Effectiveness – The degree to which specified users can achieve specified goals with accuracy and completeness in a specified context of use. • Efficiency – The degree to which specified users expend appropriate amounts of resources in relation to the effectiveness achieved in a specified context of use. – NOTE Relevant resources can include time to complete the task, materials, or the financial cost of usage. • Satisfaction – The degree to which users are satisfied in a specified context of use. Satisfaction is further subdivided into sub-characteristics: • Likability (cognitive satisfaction) • Pleasure (emotional satisfaction) • Comfort (physical satisfaction) • Trust Source: ISO 25010
- 18. Usability in Actual Usage • User role specified users • Objective specified goals • Task • Environment specified context of use • Domain What else can you think •… of?
- 19. Defining Usability For Your Organization Quality Usability Characteristic 1 Characteristic 2 Characteristic n Subcharacteristic 1 Subcharacteristic 2 Subcharacteristic n Attribute 1 Attribute 2 Attribute 3 Attribute 1 • Attributes expressed hierarchically • Any number of sub-levels is OK • Achieving the sub-attributes=achieving the high level attribute -measurable
- 20. Let’s Define Usability From the Product Viewpoint Usability Characteristic 1 Navigation Understandable Characteristic 2 Characteristic n Subcharacteristic 1 Control Stability Subcharacteristic 2 Previous-Next Subcharacteristic n Attribute 1 Home Location Attribute 2 Save Location Ease of finding Attribute n
- 21. Bigger Picture on Usability Usability Usability • Can be measured from the design point of view or of the product • Can be measured ‘in-use’ with real users
- 22. Defining Usability from an Effect-Real usage Point of View Usability Quality Effectiveness Efficiency Satisfaction Characteristic n Accuracy Completeness Subcharacteristic n Errors Attribute 2 Attribute 3 Attribute 1
- 23. Other Potential Attributes Measurements for ‘in use’ Usability • Effectiveness – Completion rates – Error rate – Help usage • Efficiency – Task time – Backtracking • Learnability – Learning rate – Task time deviation
- 24. Developing a Combined Usability Model Measured ‘in-use’ Usability Measurable and quantifiable factors User Behavior and Activity Application Design and Environment
- 25. Example of Product Quality Model with Attributes External Quality Requirements (for Shopping Cart Entity) 1 Usability (Operability in ISO 25010) 1.1 Understandability 1.1.1 Icon/label ease to be recognized 1.1.2 Information grouping cohesiveness 1.2 Learnability 1.2.1 ……………………………………………………….. 1.3 Operability 1.3.1 Control permanence 1.3.2 Expected behaviour of Controls 2 Content Quality 2.1 Content Suitability 2.1.1 Basic Information Coverage 2.1.1.1 Line item information completeness 2.1.1.2 Product description appropriateness 2.1.2 Coverage of other Contextual Information 2.1.2.1 …………………………………………………………
- 26. Measurable Attributes • Attribute name • Description and purpose Once you have a model (what you • How to measure are going to • What is measured measure), then • Measurement/Calculation you start doing IT! • Range (min, max) • Objective • Current
- 27. Usability Measurement Attribute Scale How Measure or Objective Current Calculation Help Percent of Compare % 90% 40% completeness Menu items menus and with help items help Ease of Keystrokes Sample 50 % 3 12 access to items find/use a feature/func tion/informa tion Consistency Number Examine integer 1 5 locations for menus and same button doc. Accuracy Number Collect from Integer or % <5 10 reported log files errors 27
- 28. Usability Measurement Methods Focus Walk groups Throughs Satisfaction Surveys Labs Heuristic Evaluation Logging Let’s get started
- 29. Current Research-Summary of Usability Evaluation Methods
- 30. Usability Logging Measurement and Data Collection • Identify users by using session ID to identify a unique user. • Iteratively insert code into the application • Collect data • Analyze the data for each attribute in different dimensions and aggregations • Determine the need for further calculations and what attributes to measure further • Revise the data we are collecting, adding or decreasing granularity
- 31. Satisfaction Surveys The process: 1. Calculate the usability score (satisfaction) of version X 2. Do the survey 3. Change to version X.1 – make changes to the software directly correlated to the usability factors to either increase or decrease the usability score 4. Do the survey again 5. See if differences made change impact the survey results
- 32. Notes on Satisfaction and Usability don’t have what I want I’m unsatisfied • Satisfaction is a subjective feeling Highly usable dependent on many software things other than usability: My password doesn’t work – A user can be highly satisfied but the application with low Nice weather I’m usability. today satisfied! – An application can be ! highly usable (high low usability software usability) but the user is not satisfied! Finished my work today
- 33. Sample Evaluation External Quality Requirements Measure EI value P/GI value Global Quality Indicator 61.97% 1 Usability 60.88% 1.1 Understandability 83% 1.1.1 Icon/label ease to be recognized 100% 1.1.2 Information grouping cohesiveness 66% 1.2 Learnability 51.97% 1.2.1 ……………………………………………… … 1.3 Operability 49.50% 1.3.1 Control permanence 100% 1.3.2 Expected behaviour 50% 2 Content Quality 63.05% 2.1 Content Suitability 63.05% 2.1.1 Basic Information Coverage 50% 2.1.1.1 Line item information completeness 2 50% 2.1.1.2 Product description appropriateness 50% 2.1.2 Coverage of other Contextual Information 76.89% 2.1.2.1 ……………………………………………….. … 2.1.2.2 Return policy information completeness 33%
- 35. Conclusion • Usability is an abstract concept • Defining usability is different for each organization • Need a model for your organization • The model is the foundation of what to measure • Once you can measure, then you can evaluate and improve
- 36. Next Steps • Produce an action plan – What usability attributes are important to your organization? • Develop a model – What data can you collect/Which technique can you use • Maybe some elements of the model drop out-can’t be measured that easily – Start collecting and developing benchmarks • Discuss with your manager/team