Google considers crawl errors as one of the most well-known features in Webmaster Tools and recently they have rolled out some very significant improvements to make this feature even more helpful for webmasters and professional search engine optimization experts.
Vanessa Fox of Search Engine Land stated that the “update is mostly about removing super useful data, masked by a few user interface changes.”
With the new features, Google now monitors and tracks the failure rates for each type of site-wide error called site errors. Site errors are said to be errors that aren’t specific to a particular URL but they impact your entire site. These would include DNS resolution failures, connectivity issues with your web server, and problems fetching your robots.txt file.
Previously, Google used to report these errors by URL, but because these errors are not specific to individual URLs, they stopped doing it.
Google now also stated that they will now try to send out alerts when these errors become frequent enough that they warrant attention by professional SEO.
Google now also shows URL errors by type with full current and historical counts.
According to Google URL, errors are errors that are specific to a particular page.
Before, when Googlebot tried to crawl the URL, it was able to resolve your DNS, connect to your server, fetch and read your robots.txt file, and then request this URL. But because something went wrong after that, Google had made some changes and now Google has broken down the URL errors into various categories based on what caused the error.
Before, Google used to show at most 100,000 errors of each type. But because it is quite difficult to determine what is important or not, Google has now focused on trying to provide webmasters and search engine optimisation experts with the only most important errors up front. For each category, Google will provide you the 1000 most important and actionable errors as Google would like to think. Then, you can sort and filter these top 100 errors and be able to let Google know when you think you’ve fixed them and view details about them.