Are your words working? Creating and sustaining a content-focused research practice
- 1. Erica Jorgensen Senior content designer & manager, Microsoft
- 2. 4 years at Microsoft; currently on the Microsoft 365 content experience team Formerly at Amazon, Rover, Expedia & Premera Blue Cross Taught at the University of Washington’s digital communications program @JorgensenErica | linkedin.com/in/ejorgensen
- 3. • Why & how to do content-focused research (CFR) • How it can help you identify the exact words that are clearest and resonate the most with your audience 3
- 4. • How it helps you create more business impact with content! • How to inspire your UX writers & content designers AND build respect for them 4
- 5. What is content-focused research? 5
- 7. It helps you find out how clear—or not—specific words and phrases are to your audience
- 8. It also helps you optimize content before you launch it
- 9. It’s quick. It can be done with current, potential, or past customers. It helps check bias. You don’t know what you don’t know! 9 Why I’m a fan:
- 10. Provide you with valuable insights Help you speak your customers’ language Build customer trust & loyalty Help your business! Content research can:
- 11. • Messaging frameworks • Product & feature names • Calls to action • Any & all extra-important words in your UX Use content research to validate:
- 12. Which words should you research? Jargon Style guide terms New feature & product names Frequently used words & terms assumed to be clear
- 13. Validate the preferred words in your style guide
- 14. What is content research NOT? 14
- 15. Content testing is not “regular” UX research! It decouples the words from the visual designs
- 16. It’s not statistically significant —and that’s perfectly ok! 16 #CSSeattle Content testing is not the same as A/B experimentation
- 17. 17
- 18. What to know before you start with content research 18
- 19. Tools for content testing • User Zoom • User Testing • Survey Monkey • Microsoft Forms or • Get scrappy—just your phone, laptop, or even a clipboard & paper
- 20. Share your test plan! Stakeholders!
- 27. Test goal: Find out which is clearer: “seat” or “license”? Test audience: 20 small business owners in the US, UK, Canada & Australia
- 28. Pro tip: It’s ok to test with as few as 5 participants!
- 29. QUESTION 1: Multiple-choice question Which word would you use to describe the thing that gives you permission to use software? a) Seat b) License c) Something else (fill in the blank)
- 30. QUESTION 1: Multiple-choice question Which word would you use to describe the thing that gives you permission to use software? a) Seat b) License c) Something else (fill in the blank)
- 31. QUESTION 2: Open-ended question Tell me a bit more about why you responded the way you did to Question 1. Provide as much detail as you’d like.
- 33. Quantitative question Ex: Which word would you use to describe the thing that gives you permission to use software? THE “WHAT”
- 34. Quantitative question Ex: Which word would you use to describe the thing that gives you permission to use software? + Qualitative question Tell me a bit more about why you answered the way you did to the previous question. THE “WHAT” THE “WHY”
- 36. What we learned from content- testing “seat” and “license” 36
- 37. Quantitative test results: The WHAT “License” was preferred by 17 of 20 participants (!)
- 38. #CSSeattle
- 39. Open-ended question: ”Tell me why, tell me more” Direct quote from a participant: ” ‘Licenses’ means how many people/computers are allowed to use the program. So, for example, with Microsoft Office you’re licensed to use it on one PC or device per download & if you need Microsoft Office on an additional computer, you’d need to purchase an additional license.”
- 40. Open-ended question: “Tell me why” Direct quote from a participant: ”Licenses means how many people/computers are allowed to use the program. So, for example, with Microsoft Office you’re licensed to use it on one PC or device per download & if you need Microsoft Office on an additional computer, you’d need to purchase an additional license.”
- 41. The answers to the “tell me why” question showed many people didn’t know how many licenses are needed.
- 42. Customers’ confidence is affected when they don’t truly understand the words in the UX.
- 43. To prevent customers from under- or over-buying, UX content needed to be clarified!
- 48. How to set up a content research study in UserTesting 48
- 49. 49
- 50. 50
- 51. Pro tip: Use content testing to make sure localized content is clear!
- 52. 52
- 54. 54 Create your test questions Keep in mind that the ideal test should take at most 10 to 15 minutes for your participants to complete. The most common types of quantitative questions you’ll use will likely be: • Multiple choice • Rating scale
- 55. 55
- 56. Sample multiple-choice question result Experienced admins Inexperienced admins
- 58. 58
- 59. 59
- 60. How to build a content research program! 60
- 62. Create test plan templates THE “WHAT” THE “WHY” THE “WHAT” THE “WHY”
Editor's Notes
- #3: Marvel team = Modern Assistance Responsive/Research Voice/Video Engagement and Learning Also: Former journalist Digital analytics
- #4: **including how it has many benefits over A/B testing.
- #5: Provides moral support Brings more authority and empathy
- #7: Many ways to do this evaluation.
- #8: Helps you find out how clear—or unclear—specific words & terms are to your customers de
- #9: Though it’s perfectly fine to do this on content that’s already live, If you have a high-profile or high-stakes project, content research can help you make you and your managers more confident that what you launch with will be successful. you don’t necessarily need to wait for A/B tests or statistical significance to find out what calls to action, taglines, or other super-important UX content will resonate with your audience.
- #10: Why do it: --Improve customer experience --improve customer loyalty --build customer trust via clarity --support your brand --help your business do what they do!
- #11: Why do it: --Improve customer experience --improve customer loyalty --build customer trust via clarity --support your brand --support localization/translation --help your business do what they do!
- #12: Microsoft’s style guide MENTION BRAND
- #13: “assume” = you know what that means
- #14: Microsoft’s style guide is available online! BRAND, MARKETING
- #16: Content focused research is at its most powerful when you focus ONLY on the words prototype research because you separate out the words from the UX visual design as much as possible. When you have user research, you probably think of prototypes, showing design drafts to user research participants. CFR focuses on the words, often WITHOUT any design. Sometimes, you will need screen grabs or protoytpes to show your words in context. But it’s often super powerful and helpful for the UX process to ask your audience what words mean to them, what words they would choose to describe something—WITHOUT any visuals.
- #17: We’re not creating a Covid vaccine, we’re creating content Stat sig is FINE for high-risk content but in my experience, it has a tendency to really slow down progress and business impact. Here’s a picture of a startup, Timbuk 2—they need to move fast. Content testing is a way to help the company succeed.
- #20: Anonymity of online testing is a plus Make sure you get permission to contact customers! Privacy policies
- #21: Anonymity of online testing is a plus Make sure you get permission to contact customers! Privacy policies
- #22: **Be sensitive when testing content that’s already live—consider the feelings of those who created that content**
- #23: 10-15 minutes max to keep your audience’s interest!
- #25: Current marketing page
- #34: Qualitative has nothing to do w quality!
- #35: Qualitative has nothing to do w quality!
- #38: Quantitative vs qualitative
- #39: SUPER surprising But validated that the content in the currently-live customer experience was on the right track
- #40: Open-ended questions Like a journalist: Yes or no answers only get you so far. This gets you food for thought
- #41: Open-ended questions Like a journalist: Yes or no answers only get you so far. This gets you food for thought
- #42: Big implications for LTV
- #43: Big implications for LTV
- #44: Big implications for LTV
- #47: Hits.Microsoft.com Hundreds of colleagues have read this study! REPOSITORY
- #48: Hits.Microsoft.com Hundreds of colleagues have read this study! REPOSITORY
- #54: Can also target your tests to previous test particpants
- #62: UserTesting has UserTesting University Hundreds of colleagues have read this study! REPOSITORY
- #63: TestTemplate_forUserTestingStudies.docx (sharepoint.com)
- #64: Hits.Microsoft.com Hundreds of colleagues have read this study! REPOSITORY
- #65: Hits.Microsoft.com Hundreds of colleagues have read this study! REPOSITORY
- #66: When you’re learning so much about your customers via content, content has a greater chance of influencing the roadmap
- #67: Sarah Hendrickson When coworkers hear about tests—especially product managers—they want to learn more This creates momentum for your testing program!
- #70: NN/G: Test with 5 or fewer people https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
- #71: You can read me here:
- #72: H
- #73: H
- #82: Need new screen grab
- #84: TestTemplate_forUserTestingStudies.docx (sharepoint.com)