web crawler

internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering)

en-US/ˈwɛb ˈkɹɑlɚ/
en-GB/ˈwɛb ˈkɹɔːlə/

카테고리:

software categoryInternet bot