Home / Online Marketing / The first steps of your SEO audit: Indexing issues

The first steps of your SEO audit: Indexing issues

Indexing is in point of fact step one in any search engine optimization audit. Why?

In case your website online isn’t being listed, it’s necessarily unread by means of Google and Bing. And if the major search engines can’t in finding and “learn” it, no quantity of magic or SEO (search engine optimization) will fortify the score of your internet pages.

With a purpose to be ranked, a website online will have to first be listed.

Is your website online being listed?

There are lots of equipment to be had that will help you decide if a website online is being listed.

Indexing is, at its core, a page-level procedure. In different phrases, serps learn pages and deal with them for my part.

A snappy strategy to take a look at if a web page is being listed by means of Google is to make use of the website online: operator with a Google seek. Coming into simply the area, as in my instance underneath, will display you all the pages Google has listed for the area. You’ll be able to additionally input a particular web page URL to peer if that specific web page has been listed.

When a web page isn’t listed

In case your website online or web page isn’t being listed, the commonest wrongdoer is the meta robots tag getting used on a web page or the mistaken use of disallow within the robots.txt report.

Each the meta tag, which is at the web page point, and the robots.txt report supply directions to go looking engine indexing robots on methods to deal with content material for your web page or website online.

The variation is that the robots meta tag seems on a person web page, whilst the robots.txt report supplies directions for the website online as a complete. At the robots.txt report, then again, you’ll unmarried out pages or directories and the way the robots will have to deal with those spaces whilst indexing. Let’s read about methods to use every.


If you happen to’re now not certain in case your website online makes use of a robots.txt report, there’s a very easy strategy to take a look at. Merely input your area in a browser adopted by means of /robots.txt.

This is an instance the usage of Amazon (https://www.amazon.com/robots.txt):

The listing of “disallows” for Amazon is going on for reasonably awhile!

Google Seek Console additionally has a handy robots.txt Tester device, serving to you determine mistakes on your robots report. You’ll be able to additionally check a web page at the website online the usage of the bar on the backside to peer in case your robots report in its present shape is obstructing Googlebot.

If a web page or listing at the website online is disallowed, it is going to seem after Disallow: within the robots report. As my instance above displays, I’ve disallowed my touchdown web page folder (/lp/) from indexing the usage of my robots report. This prevents any pages dwelling in that listing from being listed by means of serps.

There are lots of cool and sophisticated choices the place you’ll make use of the robots report. Google’s Builders website online has a perfect rundown of all the tactics you’ll use the robots.txt report. Listed below are a couple of:

Robots meta tag

The robots meta tag is positioned within the header of a web page. Normally, there is not any want to use each the robots meta tag and the robots.txt to disallow indexing of a specific web page.

Within the Seek Console symbol above, I don’t want to upload the robots meta tag to all of my touchdown pages within the touchdown web page folder (/lp/) to forestall Google from indexing them since I’ve disallowed the folder from indexing the usage of the robots.txt report.

Then again, the robots meta tag does produce other purposes as neatly.

As an example, you’ll inform serps that hyperlinks on all of the web page will have to now not be adopted for SEO functions. That would turn out to be useful in positive eventualities, like on press free up pages.

Most likely the 2 directives used maximum regularly for search engine optimization with this tag are noindex/index and nofollow/practice:

  • Index practice. Implied by means of default. Seek engine indexing robots will have to index the tips in this web page. Seek engine indexing robots will have to practice hyperlinks in this web page.
  • Noindex nofollow. Seek engine indexing robots will have to NOT index the tips in this web page. Seek engine indexing robots will have to NOT practice hyperlinks in this web page.

The Google Developer’s website online additionally has a radical rationalization of makes use of of the robots meta tag.

XML sitemaps

You probably have a brand new web page for your website online, preferably you wish to have serps to search out and index it briefly. One strategy to support in that effort is to make use of an eXtensible markup language (XML) sitemap and sign in it with the major search engines.

XML sitemaps supply serps with a list of pages for your website online. That is particularly useful if you have new content material that most probably doesn’t have many one-way links pointing to it but, making it harder for seek engine robots to practice a hyperlink to search out that content material. Many content material control techniques now have XML sitemap capacity in-built or to be had by means of a plugin, just like the Yoast search engine optimization Plugin for WordPress.

Make sure to have an XML sitemap and that it’s registered with Google Seek Console and Bing Webmaster Equipment. This guarantees that Google and Bing know the place the sitemap is positioned and will regularly come again to index it.

How briefly can new content material be listed the usage of this technique? I as soon as did a check and located my new content material were listed by means of Google in simplest 8 seconds — and that was once the time it took me to modify browser tabs and carry out the website online: operator command. So it’s very fast!


In 2011, Google introduced it was once in a position to execute JavaScript and index positive dynamic parts. Then again, Google isn’t all the time in a position to execute and index all JavaScript. In Google Seek Console, the Fetch and Render device mean you can decide if Google’s robotic, Googlebot, is in reality in a position to peer your content material in JavaScript.

On this instance, the college website online is the usage of asynchronous JavaScript and XML (AJAX), which is a type of JavaScript, to generate a route matter menu that hyperlinks to express spaces of research.

The Fetch and Render device displays us that Googlebot is not able to peer the content material and hyperlinks the similar means people will. Which means that Googlebot can’t practice the hyperlinks within the JavaScript to those deeper route pages at the website online.


All the time bear in mind your website online must be listed with a view to be ranked. If serps can’t in finding or learn your content material, how can they review and rank it? So you should definitely prioritize checking your website online’s indexability while you’re appearing an search engine optimization audit.

Reviews expressed on this article are the ones of the visitor writer and now not essentially Seek Engine Land. Workforce authors are indexed right here.


Check Also

Nokia Is Debuting a New Web Series About the Future of Tech

Nokia is hoping its new internet collection about rising era will lend a hand lift …


  1. I’m truly enjoying the design and layout of your website.
    It’s a very easy on the eyes which makes it much more enjoyable for me to come
    here and visit more often. Did you hire out a developer to
    create your theme? Superb work!

Leave a Reply

Your email address will not be published. Required fields are marked *

Make Money From Home With 30 Day Success Club!