To stay away from undesirable content within the search indexes, webmasters can instruct spiders to not crawl particular data files or directories with the conventional robots.txt file in the basis directory with the domain. Also, a webpage is often explicitly excluded from a online search engine's database through the use https://news.rafeeg.ae