https://medium.com/@shotaishikawa8/the-technology-search-engines-use-to-crawl-and-index-websites-9fabff750a6c
