|
Therefore, it is extremely important to be able to return stable responses. If the cause of the low crawl frequency is the server, improving the server may improve the crawl frequency. Check whether errors have been returned using crawl statistics or server logs, and if you can predict that the reason for the low frequency is due to the server, take appropriate measures such as server reinforcement, load balancing, or improvements to the server or website program.
Let's take appropriate action. Reduce crawled URLs Most likely, the reason for the low crawl frequency is that there are too many URLs to crawl considering the quality of the content, and Google doesn't consider them worth crawling. Many large-scale sites often have a huge number of URLs to crawl. Bahamas Email List In such cases, by reducing the number of URLs that have low value to crawl, you may be able to increase the frequency of crawling the URLs you want to crawl .

You may want to optimize it in the following way. ・Normalization using redirects and canonica Always normalize duplicate pages to prevent them from being crawled into non-canonical pages. ・Normalization of internal links Suppress crawling to unnecessary URLs by avoiding links to non-canonical URLs that are not indexed as much as possible. - redirects, all URLs will be subject to crawling, so make sure to forward directly to the final redirect destination.
|
|