Skill instructions
analyze-database2.36 KB
View saved version β
---
name: analyze-database
description: Explore what is inside an InternetData database and analyze it - its columns and what a row means, a first look from its sample rows, and in Codex, counts and lookups over a downloaded copy. Use when someone asks what a database contains, wants example rows, or wants numbers from the data such as counts by provider, how much of a network is covered, or whether an address or range is in it.
---
# Analyze an InternetData database
1. Find the database with `list_databases` and use its `versions[].id` (`vpn_ip_v1`, not `vpn_ip`). Every family it lists can be described, whatever its `standing`.
2. Call `database_metadata`. It gives the columns per format with their types and descriptions, including the allowed values of enum columns, a few sample rows, the row count, the build date and each file's size.
3. Explain the columns in plain words and show the sample rows as a table. Say what one row means for this database, for example one address range and the provider it belongs to.
4. The sample is a handful of rows picked at random when the database is built. It shows the shape of the data and is never enough to count from, so do not draw totals, shares or rankings from it. The row count is the only whole-dataset figure the metadata gives: `sample_entries` and `sample_size` describe the separate evaluation sample file, not the database or these rows.
5. For real numbers, work on a full copy, which needs a `licensed` family. The `open` ones are licensed to every organization from the start and free under CC BY-SA 4.0. In Codex, with the `internetdata` CLI installed and signed in (`internetdata login`), tell the person the file size from step 2, then download and query it locally:
```console
internetdata db download vpn_ip_v1 vpn_ip_v1.csv.gz
duckdb -c "SELECT provider, count(*) AS ranges FROM 'vpn_ip_v1.csv.gz' GROUP BY 1 ORDER BY 2 DESC LIMIT 20"
```
The CLI verifies the checksum before it finishes. Use whatever the person has for the query, whether that is duckdb, sqlite or pandas, and base column names on the metadata rather than guessing.
6. Outside Codex there is no terminal to download into. Answer what the metadata and samples can, and say that counting needs a downloaded copy.
To check single addresses rather than the database as a whole, `lookup_ip` and `lookup_ips` answer from the same data without a download.
check-ips2.58 KB
View saved version β
---
name: check-ips
description: "Look up IP addresses with InternetData: where each one is, whose network it's on and who uses it, its abuse contact and reverse DNS name, whether it's a VPN, a residential, datacenter, mobile or public proxy, a Tor node, a privacy relay, a hosting provider or a CDN, and which services answer on it. Use when someone gives you one or more IP addresses, or text that contains them, and asks where they are, who owns or operates them, whether they're anonymized or risky, or where to report abuse from them."
---
# Look up IP addresses with InternetData
1. Collect every IPv4 and IPv6 address in what you were given and drop duplicates. If there are none, ask for them.
2. For one address call `lookup_ip`. For more than one, call `lookup_ips` once with the whole list: it batches long lists itself, so never split it or loop over `lookup_ip`. A batch needs a key on a paid plan. On any other key, a list of up to 10 is still answered, one address at a time, and a longer one comes back refused on every address: say a list that long needs a paid plan, and offer to look up the 10 the person cares about most.
3. Read `coverage` before you conclude anything. A field listed in `coverage.not_included` was not returned because of the plan behind the API key, which is not the same as "no". Say "not covered by this key's plan" for it, never "not a VPN". If the person needs those fields, `my_entitlement` shows the plan and what it has used.
4. A private or reserved address comes back with `is_bogon: true`. Report it as an internal address, not as a clean one.
5. Report what came back:
- One address: where it is, the network and the company behind it, then each positive classification with its provider name when the result has one, then the abuse contact if the person asked who to report to.
- Several: a table of address, country, network or company, what it is (VPN, residential proxy, Tor, hosting and so on, or nothing found), provider, and notes, with flagged addresses first, then one line counting each category.
- Quote raw JSON only when asked.
6. If `lookup_ips` returns an error in place of a result for an address, show it in that row and keep the rest. If a call is refused with `unauthorized`, the connection needs signing in again. If it is refused for going over the plan's allowance, check `my_entitlement` and say when it resets.
A positive answer describes the network the address belongs to, not the intent of the person using it. Leave judgement about what to do with a flagged user to the person asking, unless they ask for it; the `review-traffic` skill covers that case.
database-files2.9 KB
View saved version β
---
name: database-files
description: Look into the InternetData IP databases - what exists and which ones an organization holds, what is inside one, how big it is, whether a downloaded copy is intact, and why a download failed. Use when someone asks about the downloadable databases, their columns, sample rows, sizes, build dates, formats (CSV or MMDB), checksums, license standing or download history.
---
# InternetData databases
1. Start with `list_databases`. Every published family is listed with its `standing`: `licensed` means the organization holds it today, `expired` means the term ended, `unlicensed` means it has never been bought. Never read an unlicensed or expired family as one they hold. A family marked `open` is an Open database, free under CC BY-SA 4.0, which asks them to credit InternetData and share what they build from it under the same license. Every organization holds a `standard` license to it from the start, which its `standing` reports like any other. A version's `sample_formats` lists the formats of its free evaluation sample, a small file cut from the real database that the person can download from the Databases page at https://app.internetdata.io/database/datasets, license or not.
2. Each family has a `base` ID, which is what a license names, and `versions[].id`, which is what every other database tool takes: pass `vpn_ip_v1`, never `vpn_ip`. If a tool answers `UNKNOWN_DATASET`, you probably passed the base.
3. To describe one, licensed or not, call `database_metadata` with the versioned ID. It answers the columns per format with their types, a few sample rows, the row count, the build date and the file size of each format. Show the columns as a table and the sample rows as they came, and use the sizes to say what a transfer will cost before anyone starts one.
4. To check a copy someone already has, call `database_checksum` with the versioned ID and the format, then compare against the file's own digest. In a terminal that is `sha256sum <file>` (or `shasum -a 256 <file>` on macOS). A mismatch means the file is truncated, corrupted or from a different build.
5. To explain a download that failed or stopped working, call `list_downloads`. It lists recent attempts newest first, refusals included, each with its `outcome` and `http_status`. Its `sample` flag is true for an evaluation sample rather than the database itself, and its `open` flag is true for a download of an Open database. It is a bounded window, so a database missing from it has not necessarily never been downloaded.
6. There is no download tool, on purpose: the files run to several GB, which is not something to pull into a conversation. Point people to the CLI, which verifies the checksum for them:
```console
internetdata db download vpn_ip_v1
internetdata db download vpn_ip_v1 --format mmdb vpn_ip_v1.mmdb
```
or to the client libraries at https://github.com/internetdata, or the API at https://docs.internetdata.io.
investigate-domain-or-as2.74 KB
View saved version β
---
name: investigate-domain-or-as
description: "Look into a domain or an autonomous system with InternetData: whether a domain is registered, which DNS records it has and what they point to, the IP ranges its organization holds and whether it's a disposable email domain; an AS's operator, country, registry, size, Whois record and the prefixes it announces with their RPKI status. Use when someone asks about a domain, the domain of an email address, an AS number such as AS15169, or the network operator behind one."
---
# Look into a domain or an AS with InternetData
1. Work out what you were given. An AS is a number, with or without `AS` in front: call `lookup_asn` with the number. A domain, or an email address whose domain is what matters: call `lookup_domain` with the domain alone, lower-case, without a scheme or a path. An IP address belongs to the `check-ips` skill.
2. Read `coverage` before you conclude anything. A field listed in `coverage.not_included` was not returned because of the plan behind the API key, which is not the same as "no". Without a paid plan, a domain answers only whether it is registered and which record types it has, and an AS its operator, country, registry, allocation date and domain. `my_entitlement` shows the plan.
3. For a domain, report:
- Whether it is registered, and which of A, AAAA, MX, NS and CNAME it has. No MX means it receives no mail.
- Where the plan covers them, the records themselves with when each was last seen, the IP ranges its organization holds, and `is_disposable` with the services behind it. A disposable domain is the strongest sign an email address was made to be thrown away.
- The answer comes from InternetData's latest snapshots, not a live DNS query, so say so if the person needs the state this minute.
4. For an AS, report its operator, country, registry and allocation date, and where the plan covers them, its size (IPv4 addresses and prefixes announced), its Whois record and its RPKI counts. Take every count from those fields, never by counting the prefix list: a big AS's answer carries only its first 100 prefixes, with `truncated` giving how many it announces. Summarize the prefixes (the largest few) rather than listing them, and if the person asks for every one of a truncated answer, give the 100 it has and say that the API's `GET /asn/<number>` returns them all.
5. An AS no database knows comes back as its number with empty values: say it is unknown, not that it is clean. If a call is refused with `unauthorized`, the connection needs signing in again; if it is refused for going over the plan's allowance, check `my_entitlement` and say when it resets.
Keep to what the data shows: a domain's records and an AS's prefixes describe infrastructure, not who is behind a message or what they intend.
review-traffic2.92 KB
View saved version β
---
name: review-traffic
description: Screen a batch of signups, logins, orders, reviews or log lines for anonymized or automated traffic with InternetData, keeping each result next to the record it came from. Use when someone shares an export, a CSV, a table or a log file and asks which entries came from VPNs, proxies, Tor or hosting, which countries or networks they came from, or asks for help spotting fraud, abuse or account sharing by network.
---
# Review traffic with InternetData
1. Find the IP address in each record, and keep whatever identifies the record (user, email, order ID, timestamp) beside it. If a record has more than one address, use the client address, not a server or proxy hop, and say which column you used.
2. Look up every distinct address with a single `lookup_ips` call. A batch needs a key on a paid plan: on any other key, a list of more than 10 addresses comes back refused on every address, so say a review that size needs a paid plan and offer to check the 10 records the person cares about most. Read `coverage` first: a category the key's plan does not include was not checked, so never report those records as clean on that category.
3. Join the results back onto the records and sort them into what they are:
- `is_resproxy`, `is_mobproxy`: residential or mobile proxy. The address looks like a home or phone connection but is resold, which is the strongest signal of the set.
- `is_vpn`, `is_tor`: a VPN or a Tor node. Deliberate anonymization, common among ordinary privacy-minded users too.
- `is_dcproxy`, `is_hosting`: a datacenter proxy or a hosting range. Usually automation, scripts or servers rather than a person at a device.
- `is_pubproxy`: a public proxy, one anyone can use for free, so read it like a datacenter proxy.
- `is_relay`: a privacy relay such as iCloud Private Relay: ordinary people with a privacy setting on. Weigh it lightly.
- `is_cdn`: a CDN range. Usually a service fetching on someone's behalf.
Every result also carries `location` (the country, its continent and timezone), `asn` (the network: its number and name) and `company` (who uses the address), with a key or without one. None of them is a verdict, but they group the records: a country that doesn't match the account's, one network or one company behind many accounts.
4. Report a count per category, then a table of the flagged records with the identifying fields, the address, the category, the provider where known, and the country and network. Then call out patterns that only show up across records: one provider, one network or one address behind many accounts, bursts in time, or several accounts sharing an address.
5. Offer the full joined table as CSV if the person wants to act on it.
Keep the conclusions to what the data shows. A flag says the network is anonymized, not that the person is acting in bad faith, so present the evidence and let the person asking make the call unless they ask you to recommend one.