Skip to main content
Internet Archive's 25th Anniversary Logo

Survey Crawls

Survey crawls are run about twice a year, on average, and attempt to capture the content of the front page of every web host ever seen by the Internet Archive since 1996.



rss RSS

100,903
RESULTS


Show sorted alphabetically

Show sorted alphabetically

SHOW DETAILS
up-solid down-solid
eye
Title
Date Archived
Creator
Survey Crawl Number 8
collection
8,758
ITEMS
1.2B
VIEWS
collection

eye 1.2B

collection

eye 2B

The seed for this crawl was a list of every host in the Wayback Machine This crawl was run at a level 1 (URLs including their embeds, plus the URLs of all outbound links including their embeds) The WARC files associated with this crawl are not currently available to the general public.
collection

eye 1.2B

The seeds for this crawl came from: 251 million Domains that had at least one link from a different domain in the Wayback Machine, across all time ~ 300 million Domains that we had in the Wayback, across all time 55,945,067 Domains from https://archive.org/details/wide00016 This crawl was run with a Heritrix setting of "maxHops=0" (URLs including their embeds) The WARC files associated with this crawl are not currently available to the general public.
collection

eye 1.3B

The seed for this crawl was a list of every host in the Wayback Machine This crawl was run at a level 1 (URLs including their embeds, plus the URLs of all outbound links including their embeds) The WARC files associated with this crawl are not currently available to the general public.
collection

eye 1.1B

The seed for this crawl was a list of every host in the Wayback Machine This crawl was run at a level 1 (URLs including their embeds, plus the URLs of all outbound links including their embeds) The WARC files associated with this crawl are not currently available to the general public.
Survey Crawl Number 7
Survey Crawl Number 7
collection
6,605
ITEMS
858.8M
VIEWS
collection

eye 858.8M

This "Survey" crawl was started on Feb. 24, 2018. This crawl was run with a Heritrix setting of "maxHops=0" (URLs including their embeds) Survey 7 is based on a seed list of 339,249,218 URLs which is all the URLs in the Wayback Machine that we saw a 200 response code from in 2017 based on a query we ran on Feb. 1st, 2018.   The WARC files associated with this crawl are not currently available to the general public.
collection

eye 762.3M

The seed for this crawl was a list of every host in the Wayback Machine This crawl was run at a level 1 (URLs including their embeds, plus the URLs of all outbound links including their embeds) The WARC files associated with this crawl are not currently available to the general public.
collection

eye 655M

The seed for this crawl was a list of every host in the Wayback Machine This crawl was run at a level 1 (URLs including their embeds, plus the URLs of all outbound links including their embeds) The WARC files associated with this crawl are not currently available to the general public.
.com survey started January 2011
.com survey started January 2011
collection
2,535
ITEMS
501.4M
VIEWS
collection

eye 501.4M

Survey crawl of .com domains started January 2011.
Topic: webcrawl
collection

eye 370.7M

The seed for this crawl was a list of every host in the Wayback Machine This crawl was run at a level 1 (URLs including their embeds, plus the URLs of all outbound links including their embeds) The WARC files associated with this crawl are not currently available to the general public.
Survey Crawl Number 9
collection
561
ITEMS
203.2M
VIEWS
collection

eye 203.2M

COM Survey Crawl 2009-2010
COM Survey Crawl 2009-2010
collection
729
ITEMS
77.1M
VIEWS
collection

eye 77.1M

COM survey crawl data collected by Internet Archive in 2009-2010. This data is currently not publicly accessible.
survey_net00000
survey_net00000
collection
300
ITEMS
61.5M
VIEWS
collection

eye 61.5M

Survey crawl of .net domains started December 2010.
Topic: webcrawl
ORG Survey Crawls
ORG Survey Crawls
collection
191
ITEMS
33.7M
VIEWS
collection

eye 33.7M

Survey of .org domains. This data is currently not publicly accessible.
survey_net00001
collection
170
ITEMS
22.5M
VIEWS
collection

eye 22.5M

Survey crawl of .net domains started October 2011.
Topics: webwidecrawl, net
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Sat Sep 30 10:02:40 PDT 2017 to Sat Sep 30 03:34:44 PDT 2017.
Topic: crawldata
.com survey started January 2011
web

eye 219,609

favorite 0

comment 0

Internet Archive crawldata from COM zone survey, captured by crawl318.us.archive.org:com from Sat Feb 5 22:15:07 PST 2011 to Sat Feb 5 15:33:45 PST 2011.
Topic: crawldata
survey_00010
web

eye 703,424

favorite 0

comment 0

Internet Archive crawldata from feed-driven by 1.2 million top ranked domains from "data.domainrank.io/" , captured by crawl423.us.archive.org:survey_00010 from Tue Sep 15 16:20:44 PDT 2020 to Tue Sep 15 20:46:31 PDT 2020.
Topics: survey_00010, crawldata
Survey Crawl Number 8
web

eye 1.2M

favorite 1

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl838.us.archive.org:survey from Fri Jan 18 06:42:06 PST 2019 to Fri Jan 18 00:46:03 PST 2019.
Topic: crawldata
Survey Crawl Number 8
web

eye 1.8M

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl818.us.archive.org:survey from Sat Jan 12 11:20:21 PST 2019 to Sat Jan 12 08:14:51 PST 2019.
Topic: crawldata
Survey Crawl Number 8
web

eye 1.8M

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl339.us.archive.org:survey from Tue Jan 22 09:37:30 PST 2019 to Tue Jan 22 03:52:56 PST 2019.
Topic: crawldata
Survey Crawl Number 8
web

eye 1.8M

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Thu Jan 10 16:19:12 PST 2019 to Thu Jan 10 14:12:18 PST 2019.
Topic: crawldata
Survey Crawl Number 8
web

eye 1.7M

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl835.us.archive.org:survey from Wed Jan 16 13:02:07 PST 2019 to Wed Jan 16 13:30:45 PST 2019.
Topic: crawldata
Survey Crawl Number 8
web

eye 1.3M

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl836.us.archive.org:survey from Fri Feb 22 02:17:40 PST 2019 to Thu Feb 21 23:46:50 PST 2019.
Topic: crawldata
Survey Crawl Number 8
web

eye 535,624

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl824.us.archive.org:survey from Sat Jan 26 15:02:34 PST 2019 to Sat Jan 26 08:47:21 PST 2019.
Topic: crawldata
Survey Crawl Number 7
web

eye 499,870

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl843.us.archive.org:survey from Fri May 11 14:54:48 PDT 2018 to Fri May 11 09:38:10 PDT 2018.
Topic: crawldata
Survey Crawl Number 8
web

eye 535,403

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl818.us.archive.org:survey from Thu Jan 24 00:44:19 PST 2019 to Wed Jan 23 18:36:19 PST 2019.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 457,892

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl339.us.archive.org:survey from Sat Oct 14 11:17:53 PDT 2017 to Sat Oct 14 04:45:39 PDT 2017.
Topic: crawldata
Survey Crawl Number 8
web

eye 482,654

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Thu Nov 29 20:01:41 PST 2018 to Thu Nov 29 17:30:37 PST 2018.
Topic: crawldata
Survey Crawl Number 8
web

eye 530,535

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl824.us.archive.org:survey from Sat Jan 26 10:01:13 PST 2019 to Sat Jan 26 05:36:23 PST 2019.
Topic: crawldata
Survey Crawl Number 8
web

eye 468,146

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl825.us.archive.org:survey from Mon Nov 26 06:50:00 PST 2018 to Mon Nov 26 06:16:37 PST 2018.
Topic: crawldata
Survey Crawl Number 5: Oct 21st, 2016 to Sep 10th, 2017
web

eye 443,422

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl836.us.archive.org:survey from Wed Oct 26 19:56:48 PDT 2016 to Wed Oct 26 15:40:51 PDT 2016.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 437,190

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Mon Oct 16 01:15:40 PDT 2017 to Sun Oct 15 18:45:40 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 437,147

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl824.us.archive.org:survey from Sat Oct 14 11:26:47 PDT 2017 to Sat Oct 14 04:54:28 PDT 2017.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl805.us.archive.org:survey from Fri Feb 19 11:46:14 PST 2016 to Fri Feb 19 12:08:16 PST 2016.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl808.us.archive.org:survey from Sat Dec 27 23:31:46 PST 2014 to Sat Dec 27 21:07:24 PST 2014.
Topic: crawldata
Survey Crawl Number 7
web

eye 620,175

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl849.us.archive.org:survey from Tue Jul 17 16:57:11 PDT 2018 to Tue Jul 17 11:36:35 PDT 2018.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl455.us.archive.org:survey from Sun Dec 28 03:48:06 PST 2014 to Sat Dec 27 23:09:20 PST 2014.
Topic: crawldata
Survey Crawl Number 7
web

eye 708,731

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl841.us.archive.org:survey from Tue Jul 17 16:28:45 PDT 2018 to Tue Jul 17 11:24:40 PDT 2018.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl413.us.archive.org:survey from Tue Jun 3 04:58:57 PDT 2014 to Tue Jun 3 01:09:27 PDT 2014.
Topic: crawldata
Survey Crawl Number 7
web

eye 633,169

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl843.us.archive.org:survey from Tue Jul 17 18:09:36 PDT 2018 to Tue Jul 17 13:08:03 PDT 2018.
Topic: crawldata
Survey Crawl Number 8
web

eye 1.2M

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl835.us.archive.org:survey from Fri Feb 8 21:12:51 PST 2019 to Fri Feb 8 14:44:22 PST 2019.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 336,222

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Wed Oct 11 23:22:24 PDT 2017 to Wed Oct 11 16:29:56 PDT 2017.
Topic: crawldata
Survey Crawl Number 7
web

eye 3.3M

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl841.us.archive.org:survey from Mon May 14 17:51:22 PDT 2018 to Mon May 14 16:01:52 PDT 2018.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 146,683

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl817.us.archive.org:survey from Sat Oct 7 17:09:56 PDT 2017 to Sat Oct 7 11:37:04 PDT 2017.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl455.us.archive.org:survey from Sun Nov 30 05:27:12 PST 2014 to Sat Nov 29 22:49:06 PST 2014.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 135,308

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl818.us.archive.org:survey from Sat Oct 7 17:24:23 PDT 2017 to Sat Oct 7 11:59:24 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 142,718

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl825.us.archive.org:survey from Sat Oct 7 16:31:54 PDT 2017 to Sat Oct 7 10:53:02 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 148,595

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl836.us.archive.org:survey from Sat Oct 7 17:05:46 PDT 2017 to Sat Oct 7 12:31:36 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 143,893

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl835.us.archive.org:survey from Fri Oct 13 10:05:17 PDT 2017 to Fri Oct 13 05:21:33 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 131,327

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl825.us.archive.org:survey from Fri Oct 13 10:35:56 PDT 2017 to Fri Oct 13 04:40:53 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 132,940

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl817.us.archive.org:survey from Fri Oct 13 11:01:37 PDT 2017 to Fri Oct 13 05:19:40 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 138,018

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl825.us.archive.org:survey from Fri Oct 13 09:25:26 PDT 2017 to Fri Oct 13 03:35:56 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 138,660

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl824.us.archive.org:survey from Sat Oct 7 16:27:36 PDT 2017 to Sat Oct 7 11:11:54 PDT 2017.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl825.us.archive.org:survey from Thu Nov 3 05:59:54 PDT 2016 to Thu Nov 3 02:33:09 PDT 2016.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 133,561

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl818.us.archive.org:survey from Fri Oct 13 09:49:42 PDT 2017 to Fri Oct 13 04:24:40 PDT 2017.
Topic: crawldata
Survey Crawl Number 7
web

eye 149,757

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl849.us.archive.org:survey from Tue May 29 06:50:35 PDT 2018 to Tue May 29 10:47:33 PDT 2018.
Topic: crawldata
Survey Crawl Number 7
web

eye 3.3M

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl840.us.archive.org:survey from Tue May 15 20:39:06 PDT 2018 to Tue May 15 21:00:30 PDT 2018.
Topic: crawldata
Survey Crawl Number 8
web

eye 673,428

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Sun Jan 6 21:49:58 PST 2019 to Sun Jan 6 15:13:44 PST 2019.
Topic: crawldata
Survey Crawl Number 8
web

eye 736,605

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl817.us.archive.org:survey from Thu Jan 10 20:56:16 PST 2019 to Thu Jan 10 18:51:35 PST 2019.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 187,409

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl818.us.archive.org:survey from Wed Oct 25 18:49:51 PDT 2017 to Wed Oct 25 12:16:24 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 183,937

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl825.us.archive.org:survey from Wed Oct 25 18:54:38 PDT 2017 to Wed Oct 25 12:25:45 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 133,994

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl838.us.archive.org:survey from Fri Oct 13 10:15:17 PDT 2017 to Fri Oct 13 05:56:36 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 156,191

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl837.us.archive.org:survey from Fri Oct 13 08:03:40 PDT 2017 to Fri Oct 13 03:48:51 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 142,221

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl836.us.archive.org:survey from Fri Oct 13 09:05:49 PDT 2017 to Fri Oct 13 05:02:49 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 133,884

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl837.us.archive.org:survey from Sat Oct 7 03:58:54 PDT 2017 to Fri Oct 6 22:57:34 PDT 2017.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Sat Oct 7 17:39:29 PDT 2017 to Sat Oct 7 10:55:22 PDT 2017.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl429.us.archive.org:survey from Tue Jun 3 14:36:58 PDT 2014 to Tue Jun 3 09:53:11 PDT 2014.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 128,973

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl835.us.archive.org:survey from Wed Oct 25 17:17:26 PDT 2017 to Wed Oct 25 11:42:52 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 158,571

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl339.us.archive.org:survey from Tue Oct 17 18:24:39 PDT 2017 to Tue Oct 17 14:21:33 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 158,964

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl817.us.archive.org:survey from Tue Oct 17 17:51:30 PDT 2017 to Tue Oct 17 12:03:07 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 119,186

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl836.us.archive.org:survey from Fri Oct 27 13:41:55 PDT 2017 to Fri Oct 27 07:09:46 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 123,146

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl817.us.archive.org:survey from Fri Oct 27 13:44:21 PDT 2017 to Fri Oct 27 07:09:30 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 119,102

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Wed Oct 25 18:24:52 PDT 2017 to Wed Oct 25 12:48:09 PDT 2017.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 132,935

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl838.us.archive.org:survey from Wed Oct 25 18:27:35 PDT 2017 to Wed Oct 25 13:50:40 PDT 2017.
Topic: crawldata