Crawling ecommerce sites – Maria Camanes' top tips from Tea-time SEO
Download as PPTX, PDF0 likes141 views
This document provides tips for crawling large ecommerce sites to find issues like out of stock products, empty category pages, and thin product listing pages. It recommends using Screaming Frog's custom search and extraction tools to identify these problems at scale. For out of stock products, the tool searches for identifiable out of stock text. For empty category pages, it searches for "no products" text. And for thin pages, it counts product containers or elements to identify pages with limited selections. Finding and addressing these issues can improve the user experience and organic search performance of ecommerce sites.
![How to do it:
Taking this ASOS category page as an example
• Step 1: fetch the contents of any container using class=“styleCount” by typing this: //*[contains(@class,
'styleCount’)]
Tip #3:
Use the ‘Custom extraction’ tool to find thin PLPs](https://image.slidesharecdn.com/mariacamanes-crawlingecommercesites-teatimeseo-200603141908/85/Crawling-ecommerce-sites-Maria-Camanes-top-tips-from-Tea-time-SEO-10-320.jpg)
![How to do it:
Taking this ASOS category page as an example
• Step 1: fetch the contents of any container using class=“styleCount” by typing this: //*[contains(@class,
'styleCount’)]
• Step 2: when crawl finishes, go to the ‘Custom’ tab at the top of the tool and you’ll see something like this
Tip #3:
Use the ‘Custom extraction’ tool to find thin PLPs](https://image.slidesharecdn.com/mariacamanes-crawlingecommercesites-teatimeseo-200603141908/85/Crawling-ecommerce-sites-Maria-Camanes-top-tips-from-Tea-time-SEO-11-320.jpg)
![How to do it:
Taking this ASOS category page as an example
• Step 1: fetch the contents of any container using class=“styleCount” by typing this: //*[contains(@class,
'styleCount’)]
• Step 2: when crawl finishes, go to the ‘Custom’ tab at the top of the tool and you’ll see something like this
*Note: if your page doesn’t have a container with the number of products available, you can still count the number
of elements on a page: count(//div[@class="offer__content"])
Tip #3:
Use the ‘Custom extraction’ tool to find thin PLPs](https://image.slidesharecdn.com/mariacamanes-crawlingecommercesites-teatimeseo-200603141908/85/Crawling-ecommerce-sites-Maria-Camanes-top-tips-from-Tea-time-SEO-12-320.jpg)
More Related Content
Similar to Crawling ecommerce sites – Maria Camanes' top tips from Tea-time SEO (20)
Crawling ecommerce sites – Maria Camanes' top tips from Tea-time SEO
- 1. Crawling large ecommerce sites
- 2. Maria Camanes, Senior SEO Consultant ● Over 6 years in SEO. Now a Senior SEO Consultant at Builtvisible, where I joined 3 years ago ● Passionate about the technical side of SEO and specialised in site speed optimisation and eCommerce SEO ● Work across a variety of accounts but mostly eCommerce sites ● Occasional speaker and regular trainer at BrightonSEO ● Twitter @mariacamanes
- 3. Common issues: • A missing or wrongly implemented product retirement strategy can – and will – have a negative impact on any ecommerce site’s organic performance • Broken links are harmful for all types of sites but the possibility of broken links in an ecommerce site is higher • Discontinued or temporarily unavailable products can result in large quantities of 404s, broken links and empty category pages (thin content) • Displaying a 404 or empty page to your beloved customers will result in bad UX but also on large quantities of link equity being lost Today we’ll focus on how to find out of stock products as well as thin category pages and - as these often occur in large quantities - how to deal with them at scale. Maria’s tips on crawling large ecommerce sites
- 4. For example: • This product page, has 91 backlinks from 28 different referring domains. The site has a number of pages that return a 4xx status code with a significant number of backlinks • As a result, it’s quite common to find large amounts of out of stock product pages for a single site indexed by search engines Tip #1: Crawl your site to find out of stock products at scale
- 5. How to do it: Use the ‘Custom search’ feature in Screaming Frog to find these when they return a 200 status code (as these won’t be picked up via a standard crawl or GSC) • Step 1: find the identifiable out of stock copy on your product page. Using Amazon as an example, their identifiable out of stock is “currently unavailable” Tip #1: Crawl your site to find out of stock products at scale
- 6. How to do it: Use the ‘Custom search’ feature in Screaming Frog to find these when they return a 200 status code (as these won’t be picked up via a standard crawl or GSC) • Step 1: find the identifiable out of stock copy on your product page. Using Amazon as an example, their identifiable out of stock is “currently unavailable” • Step 2: copy and paste this into Screaming Frog’s ‘Custom Search’ feature and run a crawl Tip #1: Crawl your site to find out of stock products at scale
- 7. How to do it: Use the ‘Custom search’ feature in Screaming Frog to find these when they return a 200 status code (as these won’t be picked up via a standard crawl or GSC) • Step 1: find the identifiable out of stock copy on your product page. Using Amazon as an example, their identifiable out of stock is “currently unavailable” • Step 2: copy and paste this into Screaming Frog’s ‘Custom Search’ feature • Step 3: crawl will return all of the product pages that contain the “out of stock” string. Don’t forget to manually QA for any errors Tip #1: Crawl your site to find out of stock products at scale
- 8. • You can use the same process to find product listing pages that are empty (meaning they have no products) • Just copy the ‘no products’ identifier in Screaming Frog, in the same way we did for ‘out of stock’ products • Here are some examples: Tip #2: Apply this to category pages to find empty PLPs
- 9. Common issues: • Thin category pages with limited stock are also a source of bad UX • They will result in lost sales and when this happens at scale, this can have a significant impact in revenue (not only for SEO) • They put the site at risk of algorithm penalties Tip #3: Use the ‘Custom extraction’ tool to find thin PLPs
- 10. How to do it: Taking this ASOS category page as an example • Step 1: fetch the contents of any container using class=“styleCount” by typing this: //*[contains(@class, 'styleCount’)] Tip #3: Use the ‘Custom extraction’ tool to find thin PLPs
- 11. How to do it: Taking this ASOS category page as an example • Step 1: fetch the contents of any container using class=“styleCount” by typing this: //*[contains(@class, 'styleCount’)] • Step 2: when crawl finishes, go to the ‘Custom’ tab at the top of the tool and you’ll see something like this Tip #3: Use the ‘Custom extraction’ tool to find thin PLPs
- 12. How to do it: Taking this ASOS category page as an example • Step 1: fetch the contents of any container using class=“styleCount” by typing this: //*[contains(@class, 'styleCount’)] • Step 2: when crawl finishes, go to the ‘Custom’ tab at the top of the tool and you’ll see something like this *Note: if your page doesn’t have a container with the number of products available, you can still count the number of elements on a page: count(//div[@class="offer__content"]) Tip #3: Use the ‘Custom extraction’ tool to find thin PLPs