Data

Basic Statistics

The following table shows some statistics about the crawled data in the GAW project:

Date	Snapshot partitions	Data in TB	Start Time (UTC)	End Time (UTC)	Unique URLs
2012-10	2922	2.67	2012-10-09 10:50:36	2012-10-29 11:03:21	40,077,499
2013-02	6477	5.90	2013-02-11 16:27:43	2013-03-05 12:16:15	170,625,168
2013-12	7089	6.46	2013-12-04 15:01:14	2014-01-16 06:59:35	107,608,630
2014-05	6339	5.77	2014-04-30 06:40:49	2014-05-13 14:51:21	107,969,414
2014-12	6154	5.60	2014-12-01 08:28:56	2014-12-08 06:01:53	100,145,401
2015-05	6449	5.87	2015-05-27 08:25:50	2015-06-09 06:04:34	103,575,494
2015-12	5928	5.40	2015-12-15 09:42:56	2016-01-05 07:40:02	101,279,483
2016-06	7289	6.64	2016-06-01 07:34:12	2016-06-13 07:01:58	112,824,260
2016-12	7088	6.46	2016-12-08 08:13:23	2016-12-17 12:15:04	99,952,250
2017-06	7417	6.76	2017-06-12 07:44:06	2017-06-21 07:09:56	100,847,532
2017-12	7374	6.72	2017-12-05 12:42:27	2017-12-17 10:17:47	102,133,168
2018-06	7106	6.47	2018-06-18 07:08:26	2018-06-29 21:31:07	101,022,822
2018-12	7297	6.65	2018-12-03 09:37:41	2018-12-14 22:32:14	100,768,903
2019-06	7012	6.39	2019-06-13 06:48:30	2019-06-27 11:07:40	99,797,675
2019-12	7035	6.41	2019-12-03 10:08:00	2019-12-10 11:32:52	100,976,809
2020-06	6953	6.41	2020-06-03 07:44:24	2020-06-11 11:44:55	99,999,403
2020-12	6998	6.37	2020-12-01 09:19:54	2020-12-10 07:58:47	100,269,671
2021-06	7207	6.57	2021-06-01 11:21:10	2021-06-14 06:33:25	105,563,149
2021-12	7202	6.56	2021-12-07 07:20:59	2021-12-17 07:07:08	101,919,655
2022-06	7855	7.20	2022-06-09 12:06:53	2022-06-20 10:25:27
2022-12	8122	7.40	2022-12-06 08:02:33	2022-12-17 20:06:41
2023-06	8614	7.84	2023-06-08 15:14:04	2023-06-20 08:17:33
2023-12	8675	7.90	2023-12-11 11:22:42	2023-12-25 10:54:45