Login
Login

Automated ways to query bgp.tools


Overview

bgp.tools attempts to ensure that data is as up to date as possible, this means that the website has to be designed in a special way that makes page generation rather expensive server side.

If at all possible, we prefer that robots do not crawl the website. To help on this request there are alternatives to using the website.

whois

bgp.tools also listens on whois, giving basic info on who owns what IP address. For the most part is this all most people need.

Assuming you are on a bash shell, you can easily add this as a macro on the cli by adding this to the .bashrc of your user:

function cwhois { whois -h bgp.tools " -v $*"; }
export -f cwhois

You can use this to query ASNs and IP addresses, or MAC addresses:

$ cwhois as206924
AS      | IP     | BGP Prefix| CC | Registry | Allocated  | AS Name
206924  |        |           | GB | RIPE     | 2017-04-11 | Ben Cartwright-Cox

$ cwhois 90:e2:ba:61:c3:88
Vendor:	Intel Corporate

whois bulkmode

Sometimes you need to lookup a large list at once. To add on to the whois interface, is a bulk interface.

You can activate the bulk mode by connecting to bgp.tools on TCP port 43, and starting the first message with “begin”

After that you can feed items into the engine, and then finish with “end”. The whois interface will spit out all of the prefixes + asn’s at once for you to check over, This is useful for scripts that output large amounts of IP addresses.

Example:

# cat test
begin
as206924
as174
1.1.1.1
2a00::
end

# cat test | nc bgp.tools 43
206924  |        |           | GB | RIPE     | 2017-04-11 | Ben Cartwright-Cox
174     |        |           | US | ARIN     | 1996-05-15 | Cogent Communications
13335   | 1.1.1.1| 1.1.1.0/24| US | ARIN     | 2010-07-14 | Cloudflare, Inc.
3209    | 2a00:: | 2a00::/22 | DE | RIPE     | 2002-09-11 | Vodafone GmbH

Counting mode

There is also a mode where the verbose output can print line numbers too, This allows for easier comparision.

Example:

# cat test
begin
verbose
count
as206924
as174
1.1.1.1
end

# cat test | nc bgp.tools 43
3     | 206924  |                  |                     | GB | RIPE     | 2017-04-11 | Ben Cartwright-Cox
4     | 174     |                  |                     | US | ARIN     | 1996-05-15 | Cogent Communications
5     | 13335   | 1.1.1.1          | 1.1.1.0/24          | US | ARIN     | 2010-07-14 | Cloudflare, Inc.

HTTP API Notes

You must set a HTTP User-Agent to identify yourself on these requests. Default user agents are not supported as we have no contact method to reach you if something goes wrong.

Systems who use a default or generic user agents to fetch bgp.tools data constantly may be blocked.

The preferred user agent format is:

acmeco bgp.tools - contact@acme.co

Users who scrape the HTML pages from bgp.tools may be banned with no notice. bgp.tools’s HTML output is a site for humans and their respective usage patterns, not robots. If you need data there may be solution if you email admin@bgp.tools

Tag export

Tags can be exported by putting a .csv or .txt on the end of the URL for a tag list. For example: https://bgp.tools/tags/cdn.csv

Table visibility export

If you wish to search for prefixes, or pull a full table without having a full table BGP session. We have a table dump in two formats:

https://bgp.tools/table.txt

Example (searching for all prefixes originated by AS206924)

$ curl https://bgp.tools/table.txt -s | grep 206924
2a0c:2f07:d::/48 206924
...
185.230.223.0/24 206924

To aid machine parsing, the table is also available as json objects split by lines https://bgp.tools/table.jsonl

$ curl https://bgp.tools/table.jsonl -s | grep 206924
{"CIDR":"2a0c:2f07:d::/48","ASN":206924,"Hits":509}
{"CIDR":"2a0c:2f07:f::/48","ASN":206924,"Hits":508}
{"CIDR":"2a0c:2f07:29::/48","ASN":206924,"Hits":513}

This table export also gives you hints on how many times bgp.tools has seen that originator, this helps filter out low visibility prefixes.

Both of these tables only update every ~30 mins at the earliest. Please do not download these files more often than 30 mins, Likely you are best off caching it for 2~ hours depending on your requirements.

ASN Name export

You can dump a CSV file of all of the ASN numbers to name mappings that bgp.tools knows about, This should save you time yourself from needing to bother pulling whois yourself:

$ curl -s https://bgp.tools/asns.csv | head -n 5
asn,name,class
AS1,"Level 3 Parent, LLC",Unknown
AS10,CSNET Coordination and Information Center,Unknown
AS100,FMC Central Engineering Laboratories,Unknown
AS10000,Nagasaki Cable Media Inc.,Eyeball

The suggested time to cache this file is 24 hours. Changes are unlikely to take place sooner than that. This file also applies all of the custom name change edits that happen on bgp.tools.