Fast or Furious - Global Retail Benchmarks Webinar

Editor's Notes

• #2: Good morning everyone in the Europe and good evening to anyone else in the world….especially maybe my good friends and family in Australia – who are likely struggling to make a connection for this webinar. My name is dave Anderson, I’m VP of Marketing for Europe and Asia and I’m super excited to be sharing some performance insights with you. The only house keeping for this webinar that I’m going to share is – if you are twitter ping me @daveando and I’m also one of the voices behind @Dynatrace twitter account so say hi to me sometimes. Usual stuff phones on mute, feel free to ask questions using that chat window or on social media, and I’ll answer some questions at the end. But if it’s in your head ask away because….
• #3: Our attention spans are lower than ever, just shy of a goldfish apparently. Probably because we live in social media feeds, we switch tabs, and apps, and skim read emails. We pick up phones, change channels, skip songs…. So having content delivered in a timely fashion is increasingly important these days…especaily for this millennials.
• #4: So we’ve been talking about the importance of web speed for a long time. I hope when you hear this topic you don’t feel like this? So I what I wanted to do today is represent the evidence on why speed is critical, what might be preventing us from focusing on response times, get some key lessons from global retails like Amazon, Google, Facebook, Apple and Nordstrom, and then share some simple ways you can get started.
• #5: So let’s get a few things straight off the bat. I’m going to be using the word DATA. That’s right. Data. Not Dartar or Dataaa. I apologise in advance for this and hope we can move past it. 2) Data is cool. Big data is even cooler. And that’s where the idea for a global performance report started. We’ve been testing sites for years, and I wanted to see if any trends were emerging from year on year and also country vs country.
• #6: It started when I was in Melbourne and I pulled up our Australian retail benchmarks with a famous melbourne coffee in hand. At Dynatrace we are continually testing the top sites…every 15 minutes, and reporting back on critical metrics like response times, page size, hosts, availability and more. So having access to the data, as a bit of a data nerd is really fun…and what I saw really surprised me.
• #7: This is what the data looked like …… in fact this is last week average response time of the top retail websites for Australia, UK and US which is about 130 websites in total. . No points for guessing which is the blue line. You can see Australia is still beyond 9 seconds, the US is 4.8 seconds and the UK is 5.8 seconds. So without giving away the results to the report…it looks like this year’s story could be even bigger as these results are considerably slower for the US and the UK than last year’s averages… When I looked at this data I set I wondered if it was just australia that was bad, or maybe everyone was getting slower. And why is Australia so bad….what are they doing wrong? Is it the network? Is it maturity of the online space? So just before xmas I gathered up all the data and compiled it into an Australia retail benchmarks report that attempted to answer the questions I had. The report was really popular with nearly thousands of downloads and we fielded some enquires from media outlets. But I remember when our PR lead Frances called saying ‘how would you feel about a live TV interview with Sky News”. I remember because I went from feeling really excited about the data to a bundle of nerves!
• #8: So despite what you my think live TV didn’t include an auto queue, I had no questions given to me and I basically sat in a studio facing a big camera with an ear piece in. My cunning plan to rehearse answers to my microwave was a masterstroke! If you want to watch it you can using that link….try not to laugh at my awkward start ok? Its funny acutally because Frances our PR lead told me after she was so nervous she couldn’t eat all morning…ha.. how’s that for a vote of confidence! If you stop by the blog send me a positive comment or maybe a tweet
• #9: SO after all this momentum we thought let’s take the Australian report and make it a global report as we already had the data. …I even printed a copy recently into a nice book which people really liked…maybe physical is back. If you don’t’ have a copy of this report you can grab it from that URL, and if you want a physical copy then just @ reply me on twitter or email me [email protected] and I’ll see what we can do. So let’s take a moment to recap on why response times are so critical, and is it really still relevant today?
• #10: Well if Amazon says it is, then it is. This is amazon’s app update on the 27 of May this year. They are talking milliseconds. The reason they are talking in milliseconds is because milliseconds mater. In fact apparently we get frustrated with a delay after click with no action, at 200 milliseconds or more. It takes 300 milliseconds to blink so we are ‘fast or furious!’ Oh and it just so happens that Facebook said it as well a day later. So if they think it’s important so should you.
• #11: But of course Google has always preached the importance of page speed. In 2010 they started to factor in page speed as part of a ranking factor in your organic search results. Ok now you’re listening right because speed impacts your page rank, and page rank impacts the number of visitors you get, and that ultimately impacts, conversions, brand awareness and revenue.
• #12: We’ve written quite a bit on this topic if you head over to blog.Dynatrace.com and search for SEO you will see some great posts from our team that talk about how you can optimise your website being seeing the Google Search Bots as users, what pages they go to, and what the user experience is like. Yep the user experience of a bot. Sounds odd I know. We also have a great case study from BVA auctions that did just this and saw an improvement in page rank! Cool huh. Maybe don’t tell Google about this ok?
• #13: So desktop was 2010 and now more than ever we have to factor in mobile. Instead of waiting for us to play catch up and improve the performance of our mobile pages, Google pushed the AMP project which is set of coding standards that you need to adhere to in order to get priority ranking. It’s supposedly 4 times faster than normal response times (something I’m keen to monitor and evaluate). But despite all this complexity as if our lives weren’t hard enough…end users lift their expectatations…again
• #14: This guy is a millennial I’m told. I don’t think I’m one of them…as usual I fell just aside the threshold. The data we have gathered though over recent years is no laughing matter. The kids are now the mainstream and they are demanding. Call it the Netflix, uber, google effect… The reality is – it needs to be quick. If it’s not quick 46% will abandon beyond 3 seconds. It’s even more critical in mobile apps. In fact mobile apps is probably what has forced us to be even more impatient than we have before. What this means for websites is …speed is a massive factor.
• #15: So let’s getting into the testing and the data we gathered. Our testing included nearly 300+ sites – testing mostly homepages every 15 mins, using the chrome browser across 8 countries. We’ve picked the top players in the industry to be part of this testing. For this webinar and this report I’m focusing purely on retail but we at Dynatrace do this for a host of industries. The testing is run from our massive synthetic testing networks. For those that aren’t sure synthetic testing is like a robot hitting the site and reporting back on key performance metrics (which I’ll cover on the next slide). Dynatrace can test from global backbone locations, but uniquely we can also test on last mile PC’s…yes we pay people for that we don’t hijack machines, private last mile (so on your network locations) or mobile. For these benchmarks because the size of the data set I focused purely on backbone testing in order to get a clean and consistent set of data across countries.
• #16: As part of our testing we can cover a huge depth of insight but specifically for this report I narrowed it down to a few key performance metrics. What is the response time of the homepage . i.e how long does it take for the whole page to load? What is the technical make up of the homepage…i.e how many 3rd party hosts like google, facebook, twitter, analaytics, chat etc, and also how many objects in total are required to load. Lastly how heavy is the site. Putting all this into context we compared these numbers across each country, for a year and compared the start of the year to the end of the year. We wanted to answer the question – Are sites meeting user expectations? Are we getting faster or slower? If so, why? And what should you do about?
• #17: The results came in and we saw: Global Retail Response times across 10 major countries increased from 4.2 seconds to 4.5 seconds. Remebering our golden rule was 3 seconds or below we are moving in the wrong direction. Leading factors behind this were: 18% increase in 3rd parties 7% increase in the number of objects Average page size increased by 8%. I saw on social media actually that the average page weight of a website is now the size of the original Doom game.
• #18: When we take a look at the countries, let’s start with the elephant in the room. Australia. 5.4 seconds out to 8.2 seconds. Shocking really. The top 5 for Australia performed well but really they were let down by the majority… Every country in the world got slower except for China and France.
• #19: We cut the data globally so you can see the top 40 performers for each of the key kpi’s we were tracking. This makes it really easy for you to compare yourself against the best of the best. For instance the top 40 have an average response time of 2.2 seconds / below that 3 second golden rule. The top 40 have only 13 hosts vs the average which is 32. We’ll dive deeper in 3rd party performance shortly as this has the biggest impact response times. The average object count is only 74 against an average of 144 and page size of the top 40 is 1.4MB.
• #20: In the report we dive into each country so you can get a better picture of what is happening in your local market./ Remembering though in retail it’s likely you are competiting not just on the domestic front but also against the internationals big sites. So let’s do that. Let’s take a look at Apple!
• #21: You can see Apple has stellar performance in any country it operates in. Their average response time is unbelievable between 1-2 seconds. We know they don’t shirk on page design either…they are rich images, and really engaging….and you can see the page size in MB is about average. The big difference with apple is the number of 3rd parties. They keep these to an absolute minimum. For those that don’t know 3rd parties are chat functions, analytics, social media, basically rich components that are served from external servers… These external connections can cause a site to slow considerably and you also lose control over the content…we’ll deep dive into 3rd party performance in a bit. The only slow site is China, but China is very difficult to deliver fast performance for more reasons than we have time to go into in this webinar. If you do want more info on this head to our blogs to read more.
• #22: So what about the impact of speed on revenue and conversions? Put simply aside from speed impacting SEO, user expectations and abandonment, naturally it’s going to impact conversions. You are probably familiar with the stats from amazon and Walmart. 100ms impacting revenue by 1%! This is why Amazon is obsessed by milliseconds because they know the impact it has on user experience and revenue. But I was a tiny bit sceptical so I asked one our customers, a performance engineer from Nordstrom, if he could validate the impact of speed on revenue. The findings were consistent with what we see globally.
• #23: In this slide, at our event in Australia, Nordstrom presented that their peak conversion rate was at 2.5 seconds. A 0.5 second increase in response time impacted conversions by 11% and this was during one of the key sale periods. For a billion dollar retailer that’s a lot of money for a very small change in performance. So best practice is keep an eye on the impact of response time on conversions and optimise your site not just for speed, but for conversions.
• #24: Gopal went on to state that they now make feature and function decisions based on the SLA’s set by the customers. Not by guesswork, and not by internal teams. The data gives precise answers.
• #25: From t-Mobile our data scientists worked with t-mobile and compiled a really interesting report. They found a 2-3% reduction in conversions for every 2 second delay in time to interact. The faster the site performance the most amount of conversions. You can grab this report using the URL there. As a reminder or the URL’s I’m sharing in this webinar I’ll send in email and I’ll also publish to the CX Blog so you can grab them then. |This report is really interesting as it covers more than just homepage conversion rates but also stipulates that a buyer is content to wait the longer they are invested in the journey. So best practice there is – built your heavy process and response time resources at the end of the journey not on first engagement. If your keen on doing a report like this with us, then make sure you get in touch!
• #26: So many of you might have jumped onto the amazon prime sale this year. I certainly did! Amazon, as I’ve used through this webinar is really a best practice case so our team has been tracking their website performance in the lead up to this peak event. What you are viewing here is a multi step transaction and how amazon has shaved time leading up to the sale to optimise the site for speed. Transaction monitoring is a great way to replicate a user journey and find ways to optimise. They’ve likely done this for two reasons: Increase likelihood of people completing a purchase because speed is critical Reduce the weight and complexity of their pages so it can more easily withstand the significant load they are about to get.
• #27: In this graph here you can see they have reduced the number of objects on the page, which would reduce the load on the server to deliver these objects. They also minimise page weight and complexity. For more information on this you should jump on the blog.Dynatrace.com and read David Jone’s great insight into how Amazon prepared for their sale period.
• #28: Prepping your site for peak sale periods or peak events – like a launch or a massive media buy is best practice. This is Go Daddy just before the superbowl a few years ago. They basically stripped their page down to almost pure HTML in order to ensure the site could cope with a surge in traffic. You can see how the response time dropped from 2.4 seconds down to 0.49 seconds which was during the promotion period. Great example of being adaptive to the environment.
• #29: At the same time, and the same superbowl and major car brand didn’t adapt, and this is the avaibility of the site during the ad series. From 100% to offline. Not a great use of a $12 million dollar ad campaign really.
• #30: Which led me to this quote as a best practice for website performance. We must be response and adapt to change. You need to be continually monitoring and adjusting as the use of more 3rd parties increase and as the business surges with marketing campaigns. I love this quote from Charles Darwin….but turns out after someone used this…I googled it and maybe he never said it…but it sounds good and it fitted in nicely.
• #31: So let’s return to what typically impacts website performance aside from peak traffic …3rd parties. What I found interesting when I looked at Australia was they had almost exactly the same number of 3rd parties as the rest of the world, and yet their performance is so much worse. Our research shows that Australian sites suffered more than other countries from this increase, possibly due to Australia’s remote geographical location causing slower connections to third-party sources. Obviously It can also be the type of 3rd parties Australian retailers opted for…i.e more live chat, or more adserving…maybe Nordstrom for instance, because of sheer revenue size doesn’t have to supplement income with aderving units like Australian retailers do? So let’s spot check and take Nordstrom and compare it to an average Australian retailer who’m will remain nameless.
• #32: Synthetic monitoring is cool because you can quickly pick up another competitor or best in class retailer like Nordstrom and then check how you compare. What’s that saying – can’t beat them join them? It really is a 2 minute job to run the test, and then compare the data that you see. In this view I have grouped 3rd party host performance by response time to see which hosts are costing us precious response time. On the left is Nordstrom. On the right is a retailer in Australia who’m I leave nameless. See how poor the response time is from their 1st part host serving their assets / content, but also the google, adroll, bazaarvoice. There are 9 hosts beyond 700ms where as Nordstrom have none…not even above 500ms. The key lesson from this is you have to keep a close eye on the number of hosts and how they perform as these add up and can slow the site down considerably.
• #33: So I went a little deeper again and for those that aren’t massively technical I’ll try not to spend to long on this slide and I’ll try an dkeep it simple. here I have extracted the connection time to gather all the resources required for the site to load. I then compared it to the number of connection. The connect time takes into account network latency, number of connections, packet loss, and server side TCP connectivity. Even if you didn’t want to get into technical analysis by benchmarking yourself you can see whether or not your performing adequately. We can start to see a trend where by Australia and China have much higher connect time, likely because of the latency in the connections, as well as the speed of the network. The main outlier I found in this was Japan. See how many connections they have ---nearly 90--- and yet they had the smallest connection time? My best guess on this is that Japan is very unique and the 3rd parties they are serving are typically Japanese…so they are loading close to the origin and consequently they load quickly. Also they have a high speed network Interesting right?
• #34: The best thing about synthetic monitoring, apart from the simplicity is to benchmark your performance. Keep tab on how you competitors performance stacks up. We recommend running an extended trial to do this, instead of a simple snapshot, but if you are game, then go to our benchmarks and run a performance test. You can see the URL on the screen bit.ly/DT-test. We can also run a demo or an extended trial for you, just follow that bit.ly link. Again I’ll send all these links to you post webinar.
• #35: At the same time there is nothing more sweeter than viewing this in real time with the Dynatrace real user monitoring. Here is my dashboard that I use to monitor Dynatrace.com and my marketo performance. I can see user experience acorss the globe of all my users and see any issues that might be impacting their journeys. Of course each of these tiles are really simple to click into and get more insights and this whole dashboard is fully customisable so you can see whatever data you prefer.
• #36: Like seeing 1st party performance …ie what laoded resources are we serving and how is this impacting performance Here we are splitting out images, scripts, css and other resources that are loading. If we see any specific spikes, we can drill into what is causing the issue.
• #37: I can quicly delve into 3rd party performance and see what I have on my site, what was recently added by our web team, and whether this is impacting performance. And if an issue occurs like this.
• #38: When facebook went down in Australia and the impact it had on Australian retail benchmarks (which I was monitoring)….it shows a Huge spike in response time which would have rendered a lot of sites unusable…you can quickly pick it up and remove that component from the site….
• #39: And if that‘s too deep come back to 300,000 feet and see user experience and performance across the globe in real time.
• #40: So in summary what I’ve been covering is our synthetic monitoring solution which is the basis that has run all the testing for the benchmarks. It’s simple and aswesome for comparing your performance from multiple regions and importantly it’s great benchmarking your performance. Remember it was all about comparing your site to the best in the industry and remembering the importantce of tieing response times back to revenue. I also covered real user monitoring which will give you insights into every user as they interact with your site and how you can best optimise the site for improved performance ultimately to maximise those conversions. But if you are still feeling a bit like this:
• #41: Or maybe you internal web or IT team is feeling like this then we should talk about how we can deliver all this as a service for you and just give you exactly what you need.
• #42: Our managed insights is just that managed…so you don’t have to manage it. Get ongoing measurements, daily email reports, ongoing feedback and support and expertise on demand. It’s like having a personal trainer….but of course you get access to the gym (our platform) when ever you need.
• #43: So by now you should have a fairly decent grasp of who Dynatrace is and here you can see just some of the people that we work with. We are doing our part to make the internet a happy and prosperous place, and at the same time doing our best to minimise time spent trying to fix website and applications by giving you exactly what you need…. \we work with 17 out of the top 20 retailers, but also the majority of leading banks, airlines, insurance companies, media outlets and government agencies.
• #44: Because whilst I made it all sound a bit simple..we can’t all be apple, Nordstrom, google or amazon…and the reality is, because of peak demand, because of the complexity, you are going to run into problems, and when you do, that’s what we are here for. Here are 2 very popular sites that had issues just last weekend. Pokemon go….well the world has gone nuts over this so keeping that up was always going to be a lesson in advanced availiablity and likewise prisma…which also had issues last weekend.
• #45: Yep even the monster Netflix that delivers a seemless experience nearly every single time had a minor issue (for me anyway) last weekend.
• #46: Which is kind of funny because it trigged a HSBC fraud alert….yep that’s how rarely they have issues I guess.
• #47: But whilst it was damn annoying at the time, Netflix gave me a free week….but being on hold to my bank for 30 mins was not my idea of a great Saturday….
• #48: But it’s not just these unicorns that suffer performance issues. We see it everyday. We get it nobody is perfect…if it can happen to these guys, it can happen to anyone…which is why you need the monitoring software that gives you answers…not more data…. And expertise that comes with working with Dynatrace.
• #49: We get it. Just let us know what we can do to help you. We will send you an email with contact details, free reports, and how you can get started. Thanks for listening everyone, I hope you enjoyed it, I hope you learnt something, and I hope we can help you deliver awesome everything. Look forward to connecting with you. We’ll go through any questions you might have now.