Prevent indexing of duplicate content

Step-by-step Playbooks
, , , ,
1

Contributors

@brandon-leuangpaseuth @andreea-macoveiciuc-content-expert

Business Benefits

Send link equity to the right pages and ensure they are visible and ranking in search engine results pages.

Use tools like Google Search Console or Screaming Frog to find pages with duplicate page titles, URLs, and meta descriptions, and add them to a spreadsheet.

  1. Click on Page Title, URL, or Meta Description in Screaming Frog and filter for Duplicate for a list of duplicate pages.
  2. Navigate to Search Appearance in your Google Webmaster Tools dashboard, click HTML Improvements and Duplicate meta descriptions or Duplicate page titles to find pages with duplicate content.

Use a tool like Small SEO Tools’ Similar Sites Checker to compare pages and determine whether they contain duplicate content.

Enter the page URLs in the relevant fields and click Compare Content.

Use a tool like Moz’s Link Explorer to find out which of the duplicate pages has higher page authority and use it as the main page.

  1. Enter the URL for each page on your spreadsheet into the relevant field, click Analyze, and make a note of the page authority.
  2. Use the page with the highest page authority as the main page for the content.

Add rel=canonical tags to duplicate pages to tell search engines which page to index and rank.

Add between the and `` tags in the HTML code for the duplicate page, replacing [MAIN PAGE URL] with the full URL of the main page.

Use 301 redirects to redirect all old or outdated versions of a page to the new URL to ensure search engines are indexing the right page.

This is particularly useful if you’ve updated or changed a page URL to make it more SEO friendly. Some platforms have plugins you can use to easily create 301 redirects, such as Redirection for WordPress. For most, however, you’ll need to reach out to your web developer or web host to set up 301 redirects on old or outdated versions of a page.

Add noindex meta tags to exclude pages with duplicate content from being indexed by search engines.

  • Add between the and `` tags in the HTML code for a page to instruct search engines not to index it.
  • If you don’t want a page to be indexed but still want search engines to follow links on it and pass value and authority on to the linked pages, such as for a blog category page, add a follow tag after noindex. For example: ``

Last edited by @hesh_fekry 2023-11-14T12:35:26Z