Here is search logs of 650,000 AOL users. It's very interesting to view search history of particular person and analyze his personality. Let's do it together! Read more about AOL search database scandal or view research papers on web searching.
| Information on the robots.txt Robots Exclusion Standard and other articles about writing well-behaved Web robots. |
URLs from www.robotstxt.org:
Back to all urls
,
.