Hey guys, Today I'm going to tell you about the crawl anomaly notification a Google search console and what you could do about it. Google search console is a free tool that helps you diagnose potential issues with your website that Google's running into as it's crawling and indexing your content.
Now, these are things that you want to pay attention to because resolving these issues can lead your website to rank higher and organic search results and potentially help you get more traffic to your site. One of the more common issues that we run into is this crawl anomaly issue and the way that Google defines that is an unspecified anomaly that occurred when fetching this URL. This can mean a 400 or 500 level response code. Now the keyword here is unspecified. Google tried crawling a URL on your site and something weird happened but you didn't actually specify the response code or at least it couldn't see the response code for one reason or another. So, this is something that you need to investigate it could be that the page is simply gone.
You can get to this report by going into Google search console and you're gonna go to your index coverage and then there are four key areas. There's
- Error
- valid with warnings
- valid and;
- excluded
We're looking at the excluded part of the report and as you scroll down you will see this crawl anomaly section. Go ahead and click on that and that's going to give you a list of all the pages on your site where the googles are run into the crawl anomaly and you know clicking on. An individual URL is going to give you this panel where you could diagnose using some of the tools that Google provides to you.
The first thing you want to do is actually copy that URL and open it up in your browser and I did this here and I could see that this page no longer exists it's given me a 404 error page not found. So, the question is why did that actually show up in this report is a crawl anomaly and one thing I'm going to do is I would hit ctrl shift I and go into the network panel and then quickly refresh the page and I want to see what the response header is for this particular page. So, I'll let that load and then I'm gonna find the URL in this list and then you're gonna see several different tabs you want to click on the headers tab and you should see a status code so this is showing me a 404 not found.
Which is what I would expect because this is a broken page. So, going back to the search console I'm probably not going to do anything this is returning the proper status code and I just need to let Google now detect that code and eventually drop it from its index and no longer needs to crawl anymore. But let's say this what page was actually active or it was redirecting to a page that was broken and that was something that I wanted to fix of course need to go back and clean that up. Make sure your redirects are functioning as they should make sure that you know you're not just redirecting to a page that redirects to another page that's a bit like this where Google's trying to get to a page in your website and it's basically just hitting a wall.
Conclusion
So, the key thing here is to make sure the page is working as expected. Once any issues that you run into are resolved go back to the search console and run through the different tools that they give you as part of the validation process fetch as Google. Make sure that Google can crawl and render that page view a search result submit to index and hopefully it will eventually drop from this crawl anomaly list. Hope you find that helpful if you have any questions or examples that you'd like to share let us know in the comments.
