Data

Basic Statistics

The following table shows some statistics about the crawled data in the GAW project:

Date WARC count Data in TB Start Time End Time Unique URLs
2012-10 2923 2.67 2012-10-09 10:50:39 2012-10-29 11:03:00 3739921
2013-02 6493 5.90 2013-02-11 16:27:47 2013-03-05 12:16:15 15111570
2013-12 7091 6.46 2013-12-04 15:01:17 2014-01-16 06:59:35 9551341
2014-05 6339 5.77 2014-04-30 06:40:53 2014-05-13 14:51:21 9671164
2014-12 6154 5.60 2014-12-01 08:28:59 2014-12-08 06:01:53 9014438
2015-05 6449 5.87 2015-05-27 08:25:53 2015-06-09 06:04:34 9302972
2015-12 5928 5.40 2015-12-15 09:43:00 2016-01-05 07:40:02 8530349
2016-06 7289 6.64 2016-06-01 07:34:16 2016-06-13 07:01:58 9483131
2016-12 7088 6.46 2016-12-08 08:13:27 2016-12-17 12:15:04 8369778
2017-06 7417 6.76 2017-06-12 07:44:10 2017-06-21 07:09:56 8258000
2017-12 7374 6.72 2017-12-05 12:42:31 2017-12-17 10:17:47 8573969
2018-06 7116 6.47 2018-06-18 07:08:30 2018-06-29 21:31:07 8436781
2018-12 7297 6.65 2018-12-03 09:37:45 2018-12-14 22:32:14 8447226
2019-06 7015 6.39      
2019-12          

MIME-Type-Distribution

MIME-Types

Consistency

Intersection MatSURTs