They discover Sites by “crawling” from URL to URL, and because they transfer via pages, the bots scrutinize the key text and supplementary parts of the web page. The written content on the found site as well as the context on the hyperlinks the crawler adopted from Patagonia on the https://thestargateproject.io