A web spider is an automated socket application that requests data from  network servers. 

• Based on the presence and context of the collected data, new actions can be taken by the spider. In this way, spiders are autonomous robots surfing the web and collecting data based on pre-defined logic and criteria. 

• A common application is to automate requests to a web server's URLs over http.  Once the data is received by the spider, the data is parsed for additional URLs to follow or any other data that can be logged.  The new URLs are saved to a list that the web spider crawls through.  Each new URL is visited and the process is repeated until all available URLs are visited.