The full download is 568GB large. The data is segmented and organized into folders and subfolders, so you may just choose the files you need and don't have to download everything. The data is tab separated, ordered by IP and timestamp. The torrent also contains an offline version of this website and tab separated lists of the data which can be browsed in the service probe overview section, in the Hilbert Browser and in the reverse DNS overview.
The data is compressed using ZPAQ 1.10, which is default in Debian and Ubuntu. It was found to have the smallest filesize, although it comes at the cost of very high cpu usage. Python code to distribute decompression workload across LAN computers is part of the code pack. Decompressing all data results in 9TB of raw logfiles, but this code can also be used to recompress the data into gzip files. The gziped dataset should be ~1.5TB.
Torrent MAGNET LINK
Download torrent file (6MB)
The tables below show a small sample of every dataset as well as the number of records and the compressed and decompressed size.
ICMP Ping
52 Billion records
81.8GB ZPAQ > 1.8 TB unpacked
Result times in 10uS
| IP | Timestamp | Result |
|---|---|---|
| 108.0.11.254 | 1355672700 | alive, 24205 |
| 108.0.11.255 | 1335190500 | unreachable |
| 108.0.18.118 | 1345646700 | Icmp Type: 11,ICMP Time Exceeded, from 59.185.211.246 |
| 108.0.18.182 | 1335248100 | Icmp Error: 13,ICMP Unreachable (Communication Administratively Prohibited), from 10.23.41.2 |
Reverse DNS
10.5 Billion records
16.3GB ZPAQ > 366GB unpacked
Error codes as in libevent evdns.h
| IP | Timestamp | Result |
|---|---|---|
| 108.0.140.255 | 1336954500 | (3) |
| 108.0.141.0 | 1336886100 | (2) |
| 108.0.141.1 | 1336914900 | L100.LSANCA-VFTTP-165.verizon-gni.net |
| 108.0.141.2 | 1336752900 | pool-108-0-141-2.lsanca.fios.verizon.net |
Serviceprobes
180 Billion records
374.5GB ZPAQ > 5.5TB unpacked
States: 1:Open ; 2:Open/Reset ; 3:Open/Timeout ; 4:Closed/Reset ; 5:Timeout
All results are encoded as quoted printable
For details on this states see service probe overview
| IP | Timestamp | State | Result |
|---|---|---|---|
| 108.0.170.164 | 1343223900 | 3 | |
| 108.0.170.228 | 1355611500 | 4 | |
| 108.0.170.232 | 1355582700 | 5 | |
| 108.0.170.250 | 1355526900 | 1 | HTTP/1.1=20200=20OK=0D=0AServer:Cross=20Web=20Ser... |
| 108.0.185.34 | 1343198700 | 2 |
Hostprobes
19.5 Billion records 27.9GB ZPAQ > 771GB unpacked
Reason was only reported for some IPs.
| IP | Timestamp | State | Reason |
|---|---|---|---|
| 108.0.0.14 | 1346307300 | up | echo-reply (0.32s latency). |
| 108.0.0.15 | 1335696300 | down | no-response |
| 108.0.0.15 | 1337433300 | up | unknown |
Syncscans
2.8 Billion records for 660 Million IPs and 71 Billion ports tested.
63.5GB ZPAQ > 435GB unpacked
| IP | Timestamp | State | Reason | Tcp/Udp | Ports |
|---|---|---|---|---|---|
| 108.0.0.214 | 1337465700 | closed | port-unreachable | udp | 60917 |
| 108.0.12.10 | 1335845700 | filtered | no-response | tcp | 20,21,22,23,25,53,80,110,111,135,139,143,993,995,1723,3306,3389,5900,8080 |
| 108.0.12.252 | 1346822100 | open | syn-ack | tcp | 80 |
| 108.0.13.1 | 1335809700 | closed | reset | tcp | 20,21,22,23,25,53,80,110,111,143,443,993,995,1723,3306,3389,5900,8080 |
TCP IP Fingerprints
80 Million records
1.3GB ZPAQ > 50GB unpacked
These fingerprints use the original Nmap format, except that newlines were replaced by ",".
| IP | Timestamp | Result |
|---|---|---|
| 108.1.69.19 | 1346413500 | SCAN(V=6.01%E=4%D=8/31%OT=80%CT=22%CU=30953%PV=N%DS=11%DC=I%G=N%TM=5040A3D6%P=mipsel-openwrt-linux-gnu),SEQ(CI=I),ECN(R=N),T1(R=N)T2(R=N),T3(R=N),T4(R=N),T5(R=Y%DF=N%T=FA%W=0%S=Z%A=S+%F=AR%O=%RD=0%Q=),T6(R=Y%DF=N%T=FA%W=0%S=A%A=Z%F=R%O=%RD=0%Q=),T7(R=Y%DF=N%T=FA%W=0%S=Z%A=S%F=AR%O=%RD=0%Q=),U1(R=Y%DF=N%T=FA%IPL=38%UN=0%RIPL=G%RID=G%RIPCK=G%RUCK=G%RUD=G),IE(R=Y%DFI=N%T=FA%CD=S) |
IP ID Sequence
75 Million records
155MB ZPAQ > 2.7GB unpacked
| IP | Timestamp | Result |
|---|---|---|
| 1.0.16.2 | 1346710500 | All zeros |
| 1.0.16.6 | 1340090100 | Incremental |
| 1.0.16.8 | 1340081100 | Busy server or unknown class |
| 1.0.16.8 | 1346670900 | Incremental |
Traceroute
68 Million records
1.9GB ZPAQ > 18GB unpacked
Results are formated as ID:IP:Time,Time,Time;ID:IP:..
| Timestamp | Source IP | Target IP | ICMP/UDP | Result |
|---|---|---|---|---|
| 1340158500 | 189.81.25.249 | 74.46.163.136 | icmp | 1:200.164.176.130:20ms,30ms,*;2:200.164.28.109:20ms,30ms,30ms; 3:200.223.44.233:30ms,30ms,30ms;4:200.223.254.121:40ms,50ms,40ms; 5:200.223.45.173:70ms,60ms,70ms;6:200.223.46.138:230ms,160ms,*; 7:209.58.26.105:160ms,190ms,170ms;8:216.6.81.37:170ms,180ms,170ms; 9:66.198.111.97:*,190ms,170ms;10:216.6.87.9:200ms,200ms,160ms; 11:206.82.139.110:*,180ms,210ms;12:74.40.2.173:200ms,190ms,190ms; 13:74.40.2.193:*,200ms,200ms;14:74.40.1.54:220ms,230ms,220ms; |
| 1340158500 | 189.81.25.249 | 29.103.138.84 | icmp | 1:200.164.176.130:20ms,30ms,30ms;2:200.164.28.109:30ms,30ms,30ms; 3:200.223.44.233:20ms,30ms,30ms;4::*,*,*; 5::*,*,*; |
Statistics on how many probes were stored per hour. This gives an overview of the timeframe in which the data was collected. Scale is records/hour
Requires Javascript