
By Alan Gordon
In this report we take a look at how selected Swiss retailers and watch companies performed over the Black Friday weekend, and what it means for their businesses.
Importance of Black Friday
Is Black Friday still important, or just an excuse for companies to get rid of their old stock while convincing people they are getting unbelievable bargains? Well, clearly that’s a matter of opinion for each of us. But studies showed that 95% of 18 to 29-year-olds did intend to look for discounts over the weekend. As year-on-year sales continue to rise, there is$281 billion of global online spending at stake. Early feedback shows that 2023 will again breakrecords, just as 2022 did.
As we pointed outlast year, having a fast response and a good user experience can boost your site, increasing conversions and encouraging customers to browse for longer on your site. Sadly each year some retailers don’t prepare properly and are leftregretting lost sales.
In what has become a tradition, each year we choose a sample of sites and monitor them from Black Friday to Cyber Monday to see how well they perform. It’s a game of high stakes! Swiss consumers generally have very fast internet speeds, both for home internet and mobile, so expectations are high – if a website is responding slowly it will be painfully exposed.
This year’s targets
This year we decided to monitor two groups of companies. (As before, we are keeping the companies anonymous, as we feel it would be unfair to name and shame – but feel free to contact us to find out who they are. If you think you recognise your own company in the description, we’re happy to talk!)
Retailers
Perhaps the most interesting for Black Friday, this was the area we focused on last year. Some, but not all, of the same companies are included in this year’s list.
-
“Retailer 1” – This retailer, founded in the 2010s, still relatively small but highly rated by its customers. As an online-only store, it will live or die by its website’s performance!
-
“Retailer 2” – A small privately owned company which started out as a physical store and moved into online sales. The internet can be a great leveller, allowing small players to compete against major-league competition, but customer loyalties can disappear if response times go up…
-
“Retailer 3” – A major electronics retail chain with annual sales in the hundreds of millions, it has a large market share to defend.
-
“Retailer 4” – Another major player in Switzerland online shopping, rated in the top 3 in the electronics market.
-
“Retailer 5” – Last year we saw rather poor performance from this big name compared to its peers. Will it be able to sharpen up this time around?
Watch industry
One of Switzerland’s most famous industries, and vital to the country’s economy – exports of Swiss watches totalled 25 billion Swiss francs in 2022 and continue to grow. Swiss watches are known for their quality – but will the websites of the watchmakers display the same quality level?
We chose four representative companies to keep a close “watch” on:
-
“Watchmaker 1” – A privately owned company with revenues of $25-50m per year and a high rate of international visitors, especially from from the USA.
-
“Watchmaker 2” – A watch and jewellery retailer, focused on the Swiss market but with branches all over Europe. As with many chains, online sales continue to increase but analytics suggest that they suffer from a very high bounce rate (the number of customers who leave after viewing just one page).
-
“Watchmaker 3” – One of the leading producers of Swiss watches by volume.
-
“Watchmaker 4” – A leading luxury watchmaker with a long, proud history. This watchmaker has the highest monthly visits and highest annual revenue of our four competitors.
Monitoring approach
As always, we go for a combination of automated scripts and manual deep-dive analysis to give the best possible coverage. If you want to know more about the technical details of how we set up the monitoring, this section is for you. Skip to the Findings section further down if you just want to see the results!
Synthetic monitoring
This year we decided to use an open source framework, sitespeed.io, to run scripts against each target site every 30 minutes. Sitespeed.io is one of the most flexible tools out there, and certainly has some of the most fun logos! It’s been around for years without many apparent updates, so we wanted to see how it compares to the many commercial tools on the market.
- Sitespeed is very easy to get up and running in a Docker container.
-
Data is stored in a Graphite database and it’s naturally easy to make dashboards in Grafana, which is perfect for our needs.
-
You can create scripts using a built-in format or run your own Selenium scripts.
-
Each time Sitespeed runs our scripts, in addition to having the data in Grafana, Sitespeed also stores the detailed web report with lots of useful information, including all the standard web metrics, HAR file, Lighthouse scan details and more.
Manual analysis
With the automated scripts covering the basics, a few of our experts looked in more detail at the sites to check whether they are following good performance design practices and to drill down into any issues.
The best tool for hands-on web page analysis is already installed on your machine –Chrome Dev Tools. It really is (appropriately) the Swiss army knife of web page analysis. There are too many useful features to even list them all, but the main ones we use are:
-
Network analysis – showing the breakdown of how each page is loaded and rendered, we can check to see what causes slow responses.
-
Performance analysis – we can profile the performance of browsers including CPU and memory usage and time spent running JavaScript.
-
Google Lighthouse – this tool analyses pages and rates them against industry standards, giving recommendations that can generate many useful optimisation ideas.
There are a variety of other tools that we use from time to time. Among the best of them is WebPageTestwhich has nowadays added a great look & feel to its huge range of features. One thing we used it for here was to see how slower connections affect page loading time.
But it’s not all about fancy tools – logical thinking, an analytic approach and an understanding of web front end technologies all help a lot in working through issues.
What does good look like?
How do we decide whether the performance of a given site is good or bad?
User experience = Core Web Vitals
Firstly we refer to the Core Web Vitals, an initiative led by Google to define the “signals that are essential to delivering a great user experience on the web.“ Right now the essential metrics are defined as:
-
Largest Contentful Paint (LCP)– the time that the largest element on the page took to render on screen. To satisfy users it’s recommended the LCP is < 2.5 seconds of when the page first starts loading.
-
First Input Delay (FID) – measures how long it takes for the page to respond after the first user interaction – such as a button press. It has beenknown for decades that a good user interface should respond within 0.1 seconds to make the user feel as though the system is reacting instantaneously.
-
Cumulative Layout Shift (CLS): measures how stable the visual layout of the page is. Have you experienced, and been annoyed by, pages that jump around just when you try to press a button? A poor CLS score leads to a bad user impression. The method of calculation is complex, but all we really need to know is that a good user experience means a CLS score of 0.1 or less.
Taken together, these three metrics cover loading time, interactivity and stability – all of which are needed to make the human user feel happy and in control of their browsing.
Side note: this set of metrics will change very soon. FID will be replaced by Interaction to Next Paint(INP) which will give a better overall measure of the interactivity. Where FID was limited to only measuring the delay between the first user interaction with the page and the browser beginning to process it, the INP takes all user interactions into consideration, and also measures the full time from the user interaction to when the browser paints the next frame. So the Core Web Vitals continue to evolve in this quickly-changing field. But because this change does not take effect until 2024, we stick with the FID for now.
There are a host of other metrics which can also be useful to look at – for example Speed Index, Blocking Time, Time to First Byte… It’s easy to get lost in a whirlwind of numbers. We decided to rely on the three core values first, and use others mainly to drill down when a potential issue is reported.
User experience = a matter of opinion
Sometimes a score looks bad but human users don’t really feel the effects. That’s why it’s important that an experienced web tester verifies any potentially bad scores by loading the page and checking for themselves. There’s always the chance that an automated script has some temporary network glitch or loss of resources and reports a problem. Our experts checked every page numerous times and noted only the consistent patterns.
And Now…. The Findings
Group A – Retailers
There are some winners and some losers in our group of retailers… Looking across the whole 4-day weekend (Friday 24.11 to Monday 28.11), it’s a sad story for Retailer 2.
Retailer 2 lets its customers down…
Check out the graph below which shows the Largest Contentful Paint score over the 4 days. (Quick reminder, the LCP represents the time taken to display the most important element on the screen – often this will be the biggest image – so effectively this shows how long it takes for the page to appear populated on the screen.)
Although it’s hard to make out much detail at this level, once thing is clear – the blue line, for Retailer 2, is unstable and has much higher peaks than the others. But what does this mean for Retailer 2’s bottom line?
Sadly, the proven effect of a bad LCP is that users will be less happy, and spend less time and money on the site. So for Retailer 2 this means losing sales to the competition. And once customers go away, they may never come back.
What is causing the poor performance? Looking in more detail we see that the problem is at the server side – the “Time to First Byte” shows that Retailer 2’s website is simply too slow in responding to requests. (The yellow and red lines here show the recommended target times – to be rated good it should be below the yellow line.)
Interestingly, as the graphs make clear, the performance isn’t consistently bad, but varies up and down. So some requests get a quick response whereas others are kept waiting, without the user knowing why – most frustrating for customers.
Does Retailer 2 really care about their website?
-
The site itself looks a bit old-fashioned on a close inspection. A glance at the technologies used gives the same impression – choices that we might not make if building a website today, like jQuery and Magento.
-
It’s hard to avoid concluding that Retailer 2 just hasn’t invested enough in their site, and now they are reaping the rewards in lost sales. We hope for their sake the real-life stores are busy, but perhaps management needs to realise that in the world today, online sales are gradually swallowing the majority of in-store business.
Happy days for Retailer 1
On the bright side, Retailer 1 can be happy with their performance. For LCP scores, they were the fastest of the retailers.
Recall that Retailer 1 is a relatively small online-only outfit which aims to profit by being more fast-moving and appealing to younger, online-friendly audiences. On this evidence, their strategy is succeeding.
Even in this happy story, it isn’t all perfection. We noticed that a change made on the homepage for Black Friday and the following two days affected the blocking time.
This was due to a new set of images that were added for Black Friday. Once those images changed for Cyber Monday, the blocking delay disappeared. It’s surprising how often companies spoil their own sites’ performance by adding huge images or clumsy JavaScript, just when the site is most under pressure. Maybe they don’t know the importance of sticking to a performance budget?
Overall, we judged that this blocking time thankfully didn’t affect the user experience too much and the site was still rated as good.
Retailer 4 and 5 have room for improvement
Although the speed of response was mostly ok for the bigger companies in our group, Retailers 3, 4 and 5, there were certainly some points where the user experience wasn’t great.
(Reminder – Cumulative Layout Shift represents the extent to which the layout of the page shifts or jerks around during loading and rendering – causing annoyance to us as we try to use the page.)
The CLS value for Retailer 4 is much higher than any of the others.
We don’t have their internal analytics data to see how many customers abandoned the site due to this, butcase studies suggest that improving CLS can reduce your bounce rate.
Looking for the cause of the high CLS, we see many images showing the different Black Friday offers do not have the size defined. This can cause the browser to set the image size itself, resizing as needed which can create some shifts on the webpage as more content is rendered. The end result is a frustrated customer! But as soon as the site was updated on the Monday morning, removing the Black Friday images – the CLS dropped back down.
Meanwhile, Retailer 5 is the one where we saw several issues last year. The good news here is that performance this year was improved overall. Still, we see some areas where it could be better.
The below graph of Total Blocking Time illustrates this.
On Black Friday, the TBT reached up to 8 seconds, which means that a customer could wait 8 seconds for the page to load without being able to perform any actions, surely resulting in some closed browser tabs!
Heavy JavaScript can impact the performance of the website and create some blocking time while downloading it. For example the following resource is 1,1MB:
With a long run time:
In addition, we can see using Google Tag Manager that the use of third party code blocked the main thread execution:
Retailers – our verdict
Overall, performance of most of the retailers group was not that bad. After all, there were no complete outages and apart from Retailer 2, no really bad performance.
Bear in mind that we are testing the sites over a good quality internet connection, as most Swiss consumers experience them. But there are plenty of people who browse with a poor quality connection, whether in the mountains or on the train, and in this case the performance is worse… much worse.
Just to give one example, simulating a “Fast 3G” connection instead of standard internet makes the page load time soar to above 20 seconds. Would you wait that long?
Group B – Watch industry
What makes this sector interesting for us is the “extreme quality” ethos. Few things in life are as high-precision as a top quality Swiss watch. From a marketing point of view, a poor quality online experience may risk tarnishing the entire brand.
The sites of watchmakers show different buying patterns compared with retailers, especially at the luxury end. Overall visitor numbers are smaller and direct purchases even fewer. Consequently they are probably less affected by Black Friday traffic – and indeed this seems to match what our data shows. Overall the performance of the watchmaking companies is comparable and stable during the Black Friday weekend.
“Watchmaker 2” – the worst performer
The “black sheep” among the watchmakers is definitely Watchmaker 2. It appears the company attempted to make some last-minute adjustments or add new marketing materials during Cyber Monday, which backfired in terms of performance. We observe a significant increase in LCP, as well as other metrics like Speed Index and Fully Loaded time. As we can see from the below, the response time changes from “good” (below the yellow line threshold) to “poor” (above the red line threshold)
A separate source of irritation for end users is a high CLS score, which exceeds the score of the competitors by a factor of up to 10!
CLS score measures the unexpected layout shifts, thus a higher value implies that the elements will be jumping around the page, frustrating the end users and complicating the interaction. The effects will be more pronounced if the network connection is poor, on older devices or for users with disabilities. Furthermore, a high CLS score decreases the web page ranking in search engines.
We wouldn’t be surprised if some users felt negative about Watchmaker 2. However, in the grand scheme of things this does not rate as a disaster, merely disappointment.
Watchmaker 3 – Performance hero
Over at the website of Watchmaker 3, minimal changes have been made for Black Friday and they seem to have benefited from this.
The site has the best scores for LCP, CLS and other measures like Time to First Byte:
A Google Lighthouse scan gave a score of 97 out of 100, which is among the best (most of our scans of the other companies have given scores of 50-60)
We have decided that Watchmaker 3 wins our award for best Black Friday performer!
General recommendations for watchmakers
Despite the overall acceptable performance, we saw that several performance good practices have not been followed on their sites. Here are a few of the main ones – remember, the drive to improve quality never ends
-
Improve cache policy – many images are only cached for a few seconds which means they have to be reloaded much more often than needed
-
Reduce JavaScript size – scripts can be optimised – some JS may even be completely unused as the below screenshot from Dev Tools suggests:
-
Define sizes of images and refine CSS to improve the CLS score (as noted for the Retailers above)
-
Reduce DOM size – as picked up by the Lighthouse scan, the larger the DOM, the longer it takes to receive and process in the browser.
Good performance practices
in the interests of fairness, we should also highlight some good performance practices that our target companies have followed – great to see!
-
Using the latest HTTP2 – HTTP3 protocols to speed up connection and transfer times
-
Extensive usage of Priority Hints
-
Resource compression to reduce total page size
-
Image formats – using optimal image formats to reduce their size without impacting quality
Conclusions
One reason we do this annual monitoring is to throw the spotlight on performance of retail sites in Switzerland. As a Swiss company, part of our purpose is to improve the daily experience of Swiss people. As our chosen field is software performance, we want to make sure Swiss consumers continue to have top performing sites with a great user experience. The best way to get a great performance is to regularly check and try to improve it!
We hope this review has been interesting for you – we had fun and also learned a lot.
Any of you out there working on e-commerce websites? Hope you found some ideas above – remember, no matter how mature your setup, there’s always room to improve.
If your company doesn’t look at performance at all, it’s never too late to start!
-
First, work out what good means for you – what is the level of service your customers need?
-
Then put in place some monitoring to give you visibility of how fast and reliable your site is today. Don’t forget to focus on the user experience!
-
Focus on the most important area to improve and develop, test, deploy, measure, tune… in cycles. You will quickly see things start to improve.
0 Comments