A web spider acts like a robot surfing the Internet. You can give a web spider a simple instruction for fetching, reading, and handling content being served by any available web server. Most common applications typically use the http & https protocols but any protocol can be used. There are 2 basic kinds of spiders: Indexers and Scrapers.