diff --git a/CHANGELOG.md b/CHANGELOG.md index 2a463b8..005dec6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -49,6 +49,10 @@ replacement, and each removed path, column or flag. `number` and its `name`, selectable by either; `misc.port` renders a TCP port number, with the IANA `service` name beside it, keeping the first service the registry describes for a port so a number selects one row. +- `misc.tld` is the IANA root zone as a table: every delegated TLD keyed with its + leading dot, `.se` and `.xn--p1ai`, the register's `type` beside it, and a `unicode` + column holding the form the register displays, so `misc.tld[.рф]` selects the row + `.xn--p1ai` renders. A delegation the register marks "Not assigned" does not ship. - `geo.SE` and `geo.US`: five linked tables per country, `region`, `municipality`, `locality`, `postal-code` and `street`, weighted by population and address counts and built from SCB, GeoNames, Trafikverket NVDB and the US Census Bureau, and an diff --git a/DATA-LICENSES.md b/DATA-LICENSES.md index 704e83b..033cbb1 100644 --- a/DATA-LICENSES.md +++ b/DATA-LICENSES.md @@ -24,6 +24,7 @@ Every shipped dataset, its source, its licence and the attribution it asks for. | `misc/protocol.tsv` | [IANA Protocol Numbers](https://www.iana.org/assignments/protocol-numbers/) | [public domain](https://www.iana.org/help/licensing-terms) | none required | `data-import/protocol.py` | | `misc/territory.tsv` | [datasets/country-codes](https://github.com/datasets/country-codes) | [PDDL 1.0](https://opendatacommons.org/licenses/pddl/1-0/) | none required | `data-import/territory.py` | | `misc/timezone.tsv` | [IANA tzdb](https://www.iana.org/time-zones) 2026d `zone.tab` and the standard offset of each zone; weights from [GeoNames](https://www.geonames.org/) `cities15000` populations | [public domain](https://data.iana.org/time-zones/tzdb/LICENSE); [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) | "Populations from GeoNames, www.geonames.org" | `data-import/timezone.py` | +| `misc/tld.tsv` | [IANA Root Zone Database](https://www.iana.org/domains/root/db), checked against the root zone's [TLD list](https://data.iana.org/TLD/tlds-alpha-by-domain.txt) | [public domain](https://www.iana.org/help/licensing-terms) | none required | `data-import/tld.py` | | `misc/useragent.tsv` | [top-user-agents](https://github.com/microlinkhq/top-user-agents) desktop and mobile lists | [MIT](https://github.com/microlinkhq/top-user-agents/blob/master/LICENSE.md) | "Copyright © 2020 Kiko Beats" | `data-import/useragent.py` | Every other category is hand-written under [`data/`](data), MIT like the code. diff --git a/README.md b/README.md index a50fb2c..b0756fc 100644 --- a/README.md +++ b/README.md @@ -164,13 +164,13 @@ Each locale carries `address`, `color`, `company`, `date`, `email`, `first-name` carries `car`, `coordinate`, `creditcard` (Luhn-valid), `currency` (ISO 4217), `datetime` (RFC 3339), `emoji`, `httpmethod`, `httpstatus`, `language` (ISO 639-1, with its 639-2/T code), `mac`, `mimetype`, `objectid`, `port`, `protocol`, -`territory` (ISO 3166-1), `timezone` (IANA), `useragent` and `uuid` (v4). Many carry -sub-fields — `misc.currency.symbol`, `misc.territory.alpha2`, `misc.httpstatus.code` — -which `--list` shows. `car`, `currency`, `httpmethod`, `httpstatus`, `language`, -`mimetype`, `port`, `protocol`, `territory`, `timezone` and `useragent` are -[tables](#table), so `misc.territory[SE].capital` and `misc.currency[Euro].symbol` -select a row; `car` and `useragent` carry no key or name, so they are drawn from rather -than selected in. +`territory` (ISO 3166-1), `timezone` (IANA), `tld` (IANA root zone), `useragent` and +`uuid` (v4). Many carry sub-fields — `misc.currency.symbol`, `misc.territory.alpha2`, +`misc.httpstatus.code` — which `--list` shows. `car`, `currency`, `httpmethod`, +`httpstatus`, `language`, `mimetype`, `port`, `protocol`, `territory`, `timezone`, +`tld` and `useragent` are [tables](#table), so `misc.territory[SE].capital` and +`misc.currency[Euro].symbol` select a row; `car` and `useragent` carry no key or name, +so they are drawn from rather than selected in. `misc.httpmethod`, `misc.protocol` and `misc.port` are IANA's registries. `misc.httpmethod` is the eight methods RFC 9110 defines and PATCH; the register's @@ -184,6 +184,13 @@ IANA `service` beside it, and selecting by number alone. A draw spans the whole register, so pin `misc.port[443]` where a fixture needs a port a reader recognises; it stops at 49150, IANA assigning nothing above, so an ephemeral source port is `{int(49152,65535)}`. +`misc.tld` is the root zone's delegated TLDs, keyed with the leading dot +`misc.territory.tld` already carries, `.se`. Its `type` is the register's — +`country-code`, `generic`, `generic-restricted`, `infrastructure` or `sponsored` — and +its `unicode` the form the register displays, `.рф` for `.xn--p1ai` and the key itself +for an ASCII one, which selects the row too, so `misc.tld[.рф]` renders `.xn--p1ai`. A +draw spans the whole zone, `.arpa` and `.zuerich` alike, so pin `misc.tld[.com]` where +a fixture needs one a reader recognises. [`DATA-LICENSES.md`](DATA-LICENSES.md) names each table's source and licence. `misc.timezone` is every zone tzdb gives a shipped territory, from one apiece for most @@ -1199,11 +1206,11 @@ the Development section below, and who ships a register the four above then draw stays a column. Layer your own `misc.territory` over the shipped one and you must layer `misc.timezone` too, or the link fails at load naming the row. - **A table whose register publishes no frequency draws evenly.** `misc.httpmethod`, - `misc.port`, `misc.httpstatus` and `misc.mimetype` weigh every row alike, so GET is - a ninth of the methods drawn. Goal 10 keeps an authored fact out of a sourced table, - and no register publishes how often a method or a port is used, so a weight here - would be invented. Where one exists it is read, as `misc.timezone` reads GeoNames - populations and `sv_SE.first-name` SCB bearers. + `misc.port`, `misc.httpstatus`, `misc.mimetype` and `misc.tld` weigh every row alike, + so GET is a ninth of the methods drawn. Goal 10 keeps an authored fact out of a + sourced table, and no register publishes how often a method, a port or a TLD is used, + so a weight here would be invented. Where one exists it is read, as `misc.timezone` + reads GeoNames populations and `sv_SE.first-name` SCB bearers. - **`misc.port` selects by number, and carries no `name`.** 29 of its services sit on more than one port, `http-alt` on three, so `misc.port[http-alt]` could name no one row. The number is what a port field holds anyway, and `misc.port[443].service` @@ -1223,6 +1230,15 @@ the Development section below, and who ships a register the four above then draw every value names a row instead. A territory the register records no state for stands alone, which is `EH` alone, and naming one for it would be a claim fejkdata has no business making. +- **`misc.tld` keys carry the leading dot, where other tables key on a bare code.** + The register spells a TLD `.se` and `misc.territory.tld` already ships it so, which a + bare key would make two spellings of one fact; `{/misc.tld}` also composes onto a + host with no separator. `misc.tld[se]` misses for it, which `todo.md` carries. +- **`misc.tld` is a table of its own, and `misc.territory.tld` stays a column.** A + `parent` demands a child for every parent row, so linking them would drop every root + zone row naming no territory, which is most of them, and goal 10 holds a sourced + table whole. The loader refuses the link outright anyway: `tld` is a column of + `misc.territory`, and a table may not be named like a column of its ancestor. - **`misc.territory` carries a currency code, it does not link to `misc.currency`.** A `parent` demands a child for every parent row, and ISO 4217 registers codes no country's row can name: the funds codes (Mvdol, WIR Euro, US Dollar (Next day)), @@ -1330,6 +1346,7 @@ docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/port. docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/protocol.py docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/territory.py docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/timezone.py +docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/tld.py docker compose run --rm --user "$(id -u):$(id -g)" data-import data-import/useragent.py ``` diff --git a/data-import/tld.py b/data-import/tld.py new file mode 100644 index 0000000..4ff8820 --- /dev/null +++ b/data-import/tld.py @@ -0,0 +1,68 @@ +#!/usr/bin/env python3 +"""Rebuild data/misc/tld.tsv from the IANA Root Zone Database. + + data-import/tld.py [--source URL_OR_FILE] [--zone URL_OR_FILE] [--cache DIR] [--out FILE] + +A TLD ships when the database names a manager for it; one it marks "Not assigned" is a +delegation the root zone no longer holds, and the zone file is fetched to prove the two +agree. The key is the A-label the root zone holds; `unicode` is the form the database +displays, less the bidi marks it wraps a right-to-left label in, and proved to punycode +back to the key. +""" +import argparse +import html +import re +import sys +from pathlib import Path + +import source +import tsv + +SOURCE = "https://www.iana.org/domains/root/db" +ZONE = "https://data.iana.org/TLD/tlds-alpha-by-domain.txt" +OUT = Path(__file__).resolve().parent.parent / "data" / "misc" / "tld.tsv" +CACHE = Path(__file__).resolve().parent / "cache" +COLUMNS = ["tld", "type", "unicode"] +TYPES = ("country-code", "generic", "generic-restricted", "infrastructure", "sponsored") +BIDI = {0x200E: None, 0x200F: None} +ROW = re.compile(r'([^<]+).*?\s*([^<]*)\s*(.*?)', re.S) + + +def rows(page): + for a_label, shown, kind, manager in ROW.findall(page): + if "Not assigned" in manager: + continue + kind = kind.strip() + if kind not in TYPES: + sys.exit(f"{a_label}: the database types it {kind!r}, which is none of {TYPES}") + yield {"tld": "." + a_label, "type": kind, "unicode": html.unescape(shown).strip().translate(BIDI)} + + +def prove_each_unicode_form_is_its_key(table): + """The displayed form is the key's own spelling, which punycode encodes back to.""" + for r in table: + label = r["unicode"][1:] + encoded = label if label.isascii() else "xn--" + label.encode("punycode").decode("ascii") + if "." + encoded != r["tld"]: + sys.exit(f"{r['tld']}: the database displays {r['unicode']!r}, which is no spelling of it") + + +def main(): + p = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + p.add_argument("--cache", default=str(CACHE)) + p.add_argument("--source", default=SOURCE) + p.add_argument("--zone", default=ZONE) + p.add_argument("--out", default=str(OUT)) + a = p.parse_args() + table = sorted(rows(source.fetch(a.source, a.cache, "root-db.html").decode("utf-8")), key=lambda r: r["tld"]) + prove_each_unicode_form_is_its_key(table) + listing = source.fetch(a.zone, a.cache, "tlds-alpha-by-domain.txt").decode("utf-8") + delegated = {"." + line.strip().lower() for line in listing.splitlines() if line.strip() and not line.startswith("#")} + shipped = {r["tld"] for r in table} + if len(table) != len(delegated) or shipped != delegated: + sys.exit(f"the database assigns {len(table)} rows where the root zone holds {len(delegated)} TLDs: {sorted(shipped ^ delegated)[:10]}") + tsv.write(a.out, COLUMNS, table) + + +if __name__ == "__main__": + main() diff --git a/data/misc/tld.json b/data/misc/tld.json new file mode 100644 index 0000000..18e382b --- /dev/null +++ b/data/misc/tld.json @@ -0,0 +1 @@ +{ "format": "{tld}", "rows": "tld.tsv", "key": "tld", "name": "unicode" } diff --git a/data/misc/tld.tsv b/data/misc/tld.tsv new file mode 100644 index 0000000..9162bf3 --- /dev/null +++ b/data/misc/tld.tsv @@ -0,0 +1,1439 @@ +tld type unicode +.aaa generic .aaa +.aarp generic .aarp +.abb generic .abb +.abbott generic .abbott +.abbvie generic .abbvie +.abc generic .abc +.able generic .able +.abogado generic .abogado +.abudhabi generic .abudhabi +.ac country-code .ac +.academy generic .academy +.accenture generic .accenture +.accountant generic .accountant +.accountants generic .accountants +.aco generic .aco +.actor generic .actor +.ad country-code .ad +.ads generic .ads +.adult generic .adult +.ae country-code .ae +.aeg generic .aeg +.aero sponsored .aero +.aetna generic .aetna +.af country-code .af +.afl generic .afl +.africa generic .africa +.ag country-code .ag +.agakhan generic .agakhan +.agency generic .agency +.ai country-code .ai +.aig generic .aig +.airbus generic .airbus +.airforce generic .airforce +.airtel generic .airtel +.akdn generic .akdn +.al country-code .al +.alibaba generic .alibaba +.alipay generic .alipay +.allfinanz generic .allfinanz +.allstate generic .allstate +.ally generic .ally +.alsace generic .alsace +.alstom generic .alstom +.am country-code .am +.amazon generic .amazon +.americanexpress generic .americanexpress +.americanfamily generic .americanfamily +.amex generic .amex +.amfam generic .amfam +.amica generic .amica +.amsterdam generic .amsterdam +.analytics generic .analytics +.android generic .android +.anquan generic .anquan +.anz generic .anz +.ao country-code .ao +.aol generic .aol +.apartments generic .apartments +.app generic .app +.apple generic .apple +.aq country-code .aq +.aquarelle generic .aquarelle +.ar country-code .ar +.arab generic .arab +.aramco generic .aramco +.archi generic .archi +.army generic .army +.arpa infrastructure .arpa +.art generic .art +.arte generic .arte +.as country-code .as +.asda generic .asda +.asia sponsored .asia +.associates generic .associates +.at country-code .at +.athleta generic .athleta +.attorney generic .attorney +.au country-code .au +.auction generic .auction +.audi generic .audi +.audible generic .audible +.audio generic .audio +.auspost generic .auspost +.author generic .author +.auto generic .auto +.autos generic .autos +.aw country-code .aw +.aws generic .aws +.ax country-code .ax +.axa generic .axa +.az country-code .az +.azure generic .azure +.ba country-code .ba +.baby generic .baby +.baidu generic .baidu +.banamex generic .banamex +.band generic .band +.bank generic .bank +.bar generic .bar +.barcelona generic .barcelona +.barclaycard generic .barclaycard +.barclays generic .barclays +.barefoot generic .barefoot +.bargains generic .bargains +.baseball generic .baseball +.basketball generic .basketball +.bauhaus generic .bauhaus +.bayern generic .bayern +.bb country-code .bb +.bbc generic .bbc +.bbt generic .bbt +.bbva generic .bbva +.bcg generic .bcg +.bcn generic .bcn +.bd country-code .bd +.be country-code .be +.beats generic .beats +.beauty generic .beauty +.beer generic .beer +.berlin generic .berlin +.best generic .best +.bestbuy generic .bestbuy +.bet generic .bet +.bf country-code .bf +.bg country-code .bg +.bh country-code .bh +.bharti generic .bharti +.bi country-code .bi +.bible generic .bible +.bid generic .bid +.bike generic .bike +.bing generic .bing +.bingo generic .bingo +.bio generic .bio +.biz generic-restricted .biz +.bj country-code .bj +.black generic .black +.blackfriday generic .blackfriday +.blockbuster generic .blockbuster +.blog generic .blog +.bloomberg generic .bloomberg +.blue generic .blue +.bm country-code .bm +.bms generic .bms +.bmw generic .bmw +.bn country-code .bn +.bnpparibas generic .bnpparibas +.bo country-code .bo +.boats generic .boats +.boehringer generic .boehringer +.bofa generic .bofa +.bom generic .bom +.bond generic .bond +.boo generic .boo +.book generic .book +.booking generic .booking +.bosch generic .bosch +.bostik generic .bostik +.boston generic .boston +.bot generic .bot +.boutique generic .boutique +.box generic .box +.br country-code .br +.bradesco generic .bradesco +.bridgestone generic .bridgestone +.broadway generic .broadway +.broker generic .broker +.brother generic .brother +.brussels generic .brussels +.bs country-code .bs +.bt country-code .bt +.build generic .build +.builders generic .builders +.business generic .business +.buy generic .buy +.buzz generic .buzz +.bv country-code .bv +.bw country-code .bw +.by country-code .by +.bz country-code .bz +.bzh generic .bzh +.ca country-code .ca +.cab generic .cab +.cafe generic .cafe +.cal generic .cal +.call generic .call +.calvinklein generic .calvinklein +.cam generic .cam +.camera generic .camera +.camp generic .camp +.canon generic .canon +.capetown generic .capetown +.capital generic .capital +.capitalone generic .capitalone +.car generic .car +.caravan generic .caravan +.cards generic .cards +.care generic .care +.career generic .career +.careers generic .careers +.cars generic .cars +.casa generic .casa +.case generic .case +.cash generic .cash +.casino generic .casino +.cat sponsored .cat +.catering generic .catering +.catholic generic .catholic +.cba generic .cba +.cbn generic .cbn +.cbre generic .cbre +.cc country-code .cc +.cd country-code .cd +.center generic .center +.ceo generic .ceo +.cern generic .cern +.cf country-code .cf +.cfa generic .cfa +.cfd generic .cfd +.cg country-code .cg +.ch country-code .ch +.chanel generic .chanel +.channel generic .channel +.charity generic .charity +.chase generic .chase +.chat generic .chat +.cheap generic .cheap +.chintai generic .chintai +.christmas generic .christmas +.chrome generic .chrome +.church generic .church +.ci country-code .ci +.cipriani generic .cipriani +.circle generic .circle +.cisco generic .cisco +.citadel generic .citadel +.citi generic .citi +.citic generic .citic +.city generic .city +.ck country-code .ck +.cl country-code .cl +.claims generic .claims +.cleaning generic .cleaning +.click generic .click +.clinic generic .clinic +.clinique generic .clinique +.clothing generic .clothing +.cloud generic .cloud +.club generic .club +.clubmed generic .clubmed +.cm country-code .cm +.cn country-code .cn +.co country-code .co +.coach generic .coach +.codes generic .codes +.coffee generic .coffee +.college generic .college +.cologne generic .cologne +.com generic .com +.commbank generic .commbank +.community generic .community +.company generic .company +.compare generic .compare +.computer generic .computer +.comsec generic .comsec +.condos generic .condos +.construction generic .construction +.consulting generic .consulting +.contact generic .contact +.contractors generic .contractors +.cooking generic .cooking +.cool generic .cool +.coop sponsored .coop +.corsica generic .corsica +.country generic .country +.coupon generic .coupon +.coupons generic .coupons +.courses generic .courses +.cpa generic .cpa +.cr country-code .cr +.credit generic .credit +.creditcard generic .creditcard +.creditunion generic .creditunion +.cricket generic .cricket +.crown generic .crown +.crs generic .crs +.cruise generic .cruise +.cruises generic .cruises +.cu country-code .cu +.cuisinella generic .cuisinella +.cv country-code .cv +.cw country-code .cw +.cx country-code .cx +.cy country-code .cy +.cymru generic .cymru +.cyou generic .cyou +.cz country-code .cz +.dad generic .dad +.dance generic .dance +.data generic .data +.date generic .date +.dating generic .dating +.datsun generic .datsun +.day generic .day +.dclk generic .dclk +.dds generic .dds +.de country-code .de +.deal generic .deal +.dealer generic .dealer +.deals generic .deals +.degree generic .degree +.delivery generic .delivery +.dell generic .dell +.deloitte generic .deloitte +.delta generic .delta +.democrat generic .democrat +.dental generic .dental +.dentist generic .dentist +.desi generic .desi +.design generic .design +.dev generic .dev +.dhl generic .dhl +.diamonds generic .diamonds +.diet generic .diet +.digital generic .digital +.direct generic .direct +.directory generic .directory +.discount generic .discount +.discover generic .discover +.dish generic .dish +.diy generic .diy +.dj country-code .dj +.dk country-code .dk +.dm country-code .dm +.dnp generic .dnp +.do country-code .do +.docs generic .docs +.doctor generic .doctor +.dog generic .dog +.domains generic .domains +.dot generic .dot +.download generic .download +.drive generic .drive +.dtv generic .dtv +.dubai generic .dubai +.dupont generic .dupont +.durban generic .durban +.dvag generic .dvag +.dvr generic .dvr +.dz country-code .dz +.earth generic .earth +.eat generic .eat +.ec country-code .ec +.eco generic .eco +.edeka generic .edeka +.edu sponsored .edu +.education generic .education +.ee country-code .ee +.eg country-code .eg +.email generic .email +.emerck generic .emerck +.energy generic .energy +.engineer generic .engineer +.engineering generic .engineering +.enterprises generic .enterprises +.epson generic .epson +.equipment generic .equipment +.er country-code .er +.ericsson generic .ericsson +.erni generic .erni +.es country-code .es +.esq generic .esq +.estate generic .estate +.et country-code .et +.eu country-code .eu +.eurovision generic .eurovision +.eus generic .eus +.events generic .events +.exchange generic .exchange +.expert generic .expert +.exposed generic .exposed +.express generic .express +.extraspace generic .extraspace +.fage generic .fage +.fail generic .fail +.fairwinds generic .fairwinds +.faith generic .faith +.family generic .family +.fan generic .fan +.fans generic .fans +.farm generic .farm +.farmers generic .farmers +.fashion generic .fashion +.fast generic .fast +.fedex generic .fedex +.feedback generic .feedback +.ferrari generic .ferrari +.ferrero generic .ferrero +.fi country-code .fi +.fidelity generic .fidelity +.fido generic .fido +.film generic .film +.final generic .final +.finance generic .finance +.financial generic .financial +.fire generic .fire +.firestone generic .firestone +.firmdale generic .firmdale +.fish generic .fish +.fishing generic .fishing +.fit generic .fit +.fitness generic .fitness +.fj country-code .fj +.fk country-code .fk +.flickr generic .flickr +.flights generic .flights +.flir generic .flir +.florist generic .florist +.flowers generic .flowers +.fly generic .fly +.fm country-code .fm +.fo country-code .fo +.foo generic .foo +.food generic .food +.football generic .football +.ford generic .ford +.forex generic .forex +.forsale generic .forsale +.forum generic .forum +.foundation generic .foundation +.fox generic .fox +.fr country-code .fr +.free generic .free +.fresenius generic .fresenius +.frl generic .frl +.frogans generic .frogans +.frontier generic .frontier +.ftr generic .ftr +.fujitsu generic .fujitsu +.fun generic .fun +.fund generic .fund +.furniture generic .furniture +.futbol generic .futbol +.fyi generic .fyi +.ga country-code .ga +.gal generic .gal +.gallery generic .gallery +.gallo generic .gallo +.gallup generic .gallup +.game generic .game +.games generic .games +.gap generic .gap +.garden generic .garden +.gay generic .gay +.gb country-code .gb +.gbiz generic .gbiz +.gd country-code .gd +.gdn generic .gdn +.ge country-code .ge +.gea generic .gea +.gent generic .gent +.genting generic .genting +.george generic .george +.gf country-code .gf +.gg country-code .gg +.ggee generic .ggee +.gh country-code .gh +.gi country-code .gi +.gift generic .gift +.gifts generic .gifts +.gives generic .gives +.giving generic .giving +.gl country-code .gl +.glass generic .glass +.gle generic .gle +.global generic .global +.globo generic .globo +.gm country-code .gm +.gmail generic .gmail +.gmbh generic .gmbh +.gmo generic .gmo +.gmx generic .gmx +.gn country-code .gn +.godaddy generic .godaddy +.gold generic .gold +.goldpoint generic .goldpoint +.golf generic .golf +.goodyear generic .goodyear +.goog generic .goog +.google generic .google +.gop generic .gop +.got generic .got +.gov sponsored .gov +.gp country-code .gp +.gq country-code .gq +.gr country-code .gr +.grainger generic .grainger +.graphics generic .graphics +.gratis generic .gratis +.green generic .green +.gripe generic .gripe +.grocery generic .grocery +.group generic .group +.gs country-code .gs +.gt country-code .gt +.gu country-code .gu +.gucci generic .gucci +.guge generic .guge +.guide generic .guide +.guitars generic .guitars +.guru generic .guru +.gw country-code .gw +.gy country-code .gy +.hair generic .hair +.hamburg generic .hamburg +.hangout generic .hangout +.haus generic .haus +.hbo generic .hbo +.hdfc generic .hdfc +.hdfcbank generic .hdfcbank +.health generic .health +.healthcare generic .healthcare +.help generic .help +.helsinki generic .helsinki +.here generic .here +.hermes generic .hermes +.hiphop generic .hiphop +.hisamitsu generic .hisamitsu +.hitachi generic .hitachi +.hiv generic .hiv +.hk country-code .hk +.hkt generic .hkt +.hm country-code .hm +.hn country-code .hn +.hockey generic .hockey +.holdings generic .holdings +.holiday generic .holiday +.homedepot generic .homedepot +.homegoods generic .homegoods +.homes generic .homes +.homesense generic .homesense +.honda generic .honda +.horse generic .horse +.hospital generic .hospital +.host generic .host +.hosting generic .hosting +.hot generic .hot +.hotels generic .hotels +.hotmail generic .hotmail +.house generic .house +.how generic .how +.hr country-code .hr +.hsbc generic .hsbc +.ht country-code .ht +.hu country-code .hu +.hughes generic .hughes +.hyatt generic .hyatt +.hyundai generic .hyundai +.ibm generic .ibm +.icbc generic .icbc +.ice generic .ice +.icu generic .icu +.id country-code .id +.ie country-code .ie +.ieee generic .ieee +.ifm generic .ifm +.ikano generic .ikano +.il country-code .il +.im country-code .im +.imamat generic .imamat +.imdb generic .imdb +.immo generic .immo +.immobilien generic .immobilien +.in country-code .in +.inc generic .inc +.industries generic .industries +.infiniti generic .infiniti +.info generic .info +.ing generic .ing +.ink generic .ink +.institute generic .institute +.insurance generic .insurance +.insure generic .insure +.int sponsored .int +.international generic .international +.intuit generic .intuit +.investments generic .investments +.io country-code .io +.ipiranga generic .ipiranga +.iq country-code .iq +.ir country-code .ir +.irish generic .irish +.is country-code .is +.ismaili generic .ismaili +.ist generic .ist +.istanbul generic .istanbul +.it country-code .it +.itau generic .itau +.itv generic .itv +.jaguar generic .jaguar +.java generic .java +.jcb generic .jcb +.je country-code .je +.jeep generic .jeep +.jetzt generic .jetzt +.jewelry generic .jewelry +.jio generic .jio +.jll generic .jll +.jm country-code .jm +.jmp generic .jmp +.jnj generic .jnj +.jo country-code .jo +.jobs sponsored .jobs +.joburg generic .joburg +.jot generic .jot +.joy generic .joy +.jp country-code .jp +.jpmorgan generic .jpmorgan +.jprs generic .jprs +.juegos generic .juegos +.juniper generic .juniper +.kaufen generic .kaufen +.kddi generic .kddi +.ke country-code .ke +.kerryhotels generic .kerryhotels +.kerryproperties generic .kerryproperties +.kfh generic .kfh +.kg country-code .kg +.kh country-code .kh +.ki country-code .ki +.kia generic .kia +.kids generic .kids +.kim generic .kim +.kindle generic .kindle +.kitchen generic .kitchen +.kiwi generic .kiwi +.km country-code .km +.kn country-code .kn +.koeln generic .koeln +.komatsu generic .komatsu +.kosher generic .kosher +.kp country-code .kp +.kpmg generic .kpmg +.kpn generic .kpn +.kr country-code .kr +.krd generic .krd +.kred generic .kred +.kuokgroup generic .kuokgroup +.kw country-code .kw +.ky country-code .ky +.kyoto generic .kyoto +.kz country-code .kz +.la country-code .la +.lacaixa generic .lacaixa +.lamborghini generic .lamborghini +.lamer generic .lamer +.land generic .land +.landrover generic .landrover +.lanxess generic .lanxess +.lasalle generic .lasalle +.lat generic .lat +.latino generic .latino +.latrobe generic .latrobe +.law generic .law +.lawyer generic .lawyer +.lb country-code .lb +.lc country-code .lc +.lds generic .lds +.lease generic .lease +.leclerc generic .leclerc +.lefrak generic .lefrak +.legal generic .legal +.lego generic .lego +.lexus generic .lexus +.lgbt generic .lgbt +.li country-code .li +.lidl generic .lidl +.life generic .life +.lifeinsurance generic .lifeinsurance +.lifestyle generic .lifestyle +.lighting generic .lighting +.like generic .like +.lilly generic .lilly +.limited generic .limited +.limo generic .limo +.lincoln generic .lincoln +.link generic .link +.live generic .live +.living generic .living +.lk country-code .lk +.llc generic .llc +.llp generic .llp +.loan generic .loan +.loans generic .loans +.locker generic .locker +.locus generic .locus +.lol generic .lol +.london generic .london +.lotte generic .lotte +.lotto generic .lotto +.love generic .love +.lpl generic .lpl +.lplfinancial generic .lplfinancial +.lr country-code .lr +.ls country-code .ls +.lt country-code .lt +.ltd generic .ltd +.ltda generic .ltda +.lu country-code .lu +.lundbeck generic .lundbeck +.luxe generic .luxe +.luxury generic .luxury +.lv country-code .lv +.ly country-code .ly +.ma country-code .ma +.madrid generic .madrid +.maif generic .maif +.maison generic .maison +.makeup generic .makeup +.man generic .man +.management generic .management +.mango generic .mango +.map generic .map +.market generic .market +.marketing generic .marketing +.markets generic .markets +.marriott generic .marriott +.marshalls generic .marshalls +.mattel generic .mattel +.mba generic .mba +.mc country-code .mc +.mckinsey generic .mckinsey +.md country-code .md +.me country-code .me +.med generic .med +.media generic .media +.meet generic .meet +.melbourne generic .melbourne +.meme generic .meme +.memorial generic .memorial +.men generic .men +.menu generic .menu +.merck generic .merck +.merckmsd generic .merckmsd +.mg country-code .mg +.mh country-code .mh +.miami generic .miami +.microsoft generic .microsoft +.mil sponsored .mil +.mini generic .mini +.mint generic .mint +.mit generic .mit +.mitsubishi generic .mitsubishi +.mk country-code .mk +.ml country-code .ml +.mlb generic .mlb +.mls generic .mls +.mm country-code .mm +.mma generic .mma +.mn country-code .mn +.mo country-code .mo +.mobi generic .mobi +.mobile generic .mobile +.moda generic .moda +.moe generic .moe +.moi generic .moi +.mom generic .mom +.monash generic .monash +.money generic .money +.monster generic .monster +.mormon generic .mormon +.mortgage generic .mortgage +.moscow generic .moscow +.moto generic .moto +.motorcycles generic .motorcycles +.mov generic .mov +.movie generic .movie +.mp country-code .mp +.mq country-code .mq +.mr country-code .mr +.ms country-code .ms +.msd generic .msd +.mt country-code .mt +.mtn generic .mtn +.mtr generic .mtr +.mu country-code .mu +.museum sponsored .museum +.music generic .music +.mv country-code .mv +.mw country-code .mw +.mx country-code .mx +.my country-code .my +.mz country-code .mz +.na country-code .na +.nab generic .nab +.nagoya generic .nagoya +.name generic-restricted .name +.navy generic .navy +.nba generic .nba +.nc country-code .nc +.ne country-code .ne +.nec generic .nec +.net generic .net +.netbank generic .netbank +.netflix generic .netflix +.network generic .network +.neustar generic .neustar +.new generic .new +.news generic .news +.next generic .next +.nextdirect generic .nextdirect +.nexus generic .nexus +.nf country-code .nf +.nfl generic .nfl +.ng country-code .ng +.ngo generic .ngo +.nhk generic .nhk +.ni country-code .ni +.nico generic .nico +.nike generic .nike +.nikon generic .nikon +.ninja generic .ninja +.nissan generic .nissan +.nissay generic .nissay +.nl country-code .nl +.no country-code .no +.nokia generic .nokia +.norton generic .norton +.now generic .now +.nowruz generic .nowruz +.nowtv generic .nowtv +.np country-code .np +.nr country-code .nr +.nra generic .nra +.nrw generic .nrw +.ntt generic .ntt +.nu country-code .nu +.nyc generic .nyc +.nz country-code .nz +.obi generic .obi +.observer generic .observer +.office generic .office +.okinawa generic .okinawa +.olayan generic .olayan +.olayangroup generic .olayangroup +.ollo generic .ollo +.om country-code .om +.omega generic .omega +.one generic .one +.ong generic .ong +.onl generic .onl +.online generic .online +.ooo generic .ooo +.open generic .open +.oracle generic .oracle +.orange generic .orange +.org generic .org +.organic generic .organic +.origins generic .origins +.osaka generic .osaka +.otsuka generic .otsuka +.ott generic .ott +.ovh generic .ovh +.pa country-code .pa +.page generic .page +.panasonic generic .panasonic +.paris generic .paris +.pars generic .pars +.partners generic .partners +.parts generic .parts +.party generic .party +.pay generic .pay +.pccw generic .pccw +.pe country-code .pe +.pet generic .pet +.pf country-code .pf +.pfizer generic .pfizer +.pg country-code .pg +.ph country-code .ph +.pharmacy generic .pharmacy +.phd generic .phd +.philips generic .philips +.phone generic .phone +.photo generic .photo +.photography generic .photography +.photos generic .photos +.physio generic .physio +.pics generic .pics +.pictet generic .pictet +.pictures generic .pictures +.pid generic .pid +.pin generic .pin +.ping generic .ping +.pink generic .pink +.pioneer generic .pioneer +.pizza generic .pizza +.pk country-code .pk +.pl country-code .pl +.place generic .place +.play generic .play +.playstation generic .playstation +.plumbing generic .plumbing +.plus generic .plus +.pm country-code .pm +.pn country-code .pn +.pnc generic .pnc +.pohl generic .pohl +.poker generic .poker +.politie generic .politie +.porn generic .porn +.post sponsored .post +.pr country-code .pr +.praxi generic .praxi +.press generic .press +.prime generic .prime +.pro generic-restricted .pro +.prod generic .prod +.productions generic .productions +.prof generic .prof +.progressive generic .progressive +.promo generic .promo +.properties generic .properties +.property generic .property +.protection generic .protection +.pru generic .pru +.prudential generic .prudential +.ps country-code .ps +.pt country-code .pt +.pub generic .pub +.pw country-code .pw +.pwc generic .pwc +.py country-code .py +.qa country-code .qa +.qpon generic .qpon +.quebec generic .quebec +.quest generic .quest +.racing generic .racing +.radio generic .radio +.re country-code .re +.read generic .read +.realestate generic .realestate +.realtor generic .realtor +.realty generic .realty +.recipes generic .recipes +.red generic .red +.redumbrella generic .redumbrella +.rehab generic .rehab +.reise generic .reise +.reisen generic .reisen +.reit generic .reit +.reliance generic .reliance +.ren generic .ren +.rent generic .rent +.rentals generic .rentals +.repair generic .repair +.report generic .report +.republican generic .republican +.rest generic .rest +.restaurant generic .restaurant +.review generic .review +.reviews generic .reviews +.rexroth generic .rexroth +.rich generic .rich +.richardli generic .richardli +.ricoh generic .ricoh +.ril generic .ril +.rio generic .rio +.rip generic .rip +.ro country-code .ro +.rocks generic .rocks +.rodeo generic .rodeo +.rogers generic .rogers +.room generic .room +.rs country-code .rs +.rsvp generic .rsvp +.ru country-code .ru +.rugby generic .rugby +.ruhr generic .ruhr +.run generic .run +.rw country-code .rw +.rwe generic .rwe +.ryukyu generic .ryukyu +.sa country-code .sa +.saarland generic .saarland +.safe generic .safe +.safety generic .safety +.sakura generic .sakura +.sale generic .sale +.salon generic .salon +.samsclub generic .samsclub +.samsung generic .samsung +.sandvik generic .sandvik +.sandvikcoromant generic .sandvikcoromant +.sanofi generic .sanofi +.sap generic .sap +.sarl generic .sarl +.sas generic .sas +.save generic .save +.saxo generic .saxo +.sb country-code .sb +.sbi generic .sbi +.sbs generic .sbs +.sc country-code .sc +.scb generic .scb +.schaeffler generic .schaeffler +.schmidt generic .schmidt +.scholarships generic .scholarships +.school generic .school +.schule generic .schule +.schwarz generic .schwarz +.science generic .science +.scot generic .scot +.sd country-code .sd +.se country-code .se +.search generic .search +.seat generic .seat +.secure generic .secure +.security generic .security +.seek generic .seek +.select generic .select +.sener generic .sener +.services generic .services +.seven generic .seven +.sew generic .sew +.sex generic .sex +.sexy generic .sexy +.sfr generic .sfr +.sg country-code .sg +.sh country-code .sh +.shangrila generic .shangrila +.sharp generic .sharp +.shell generic .shell +.shia generic .shia +.shiksha generic .shiksha +.shoes generic .shoes +.shop generic .shop +.shopping generic .shopping +.shouji generic .shouji +.show generic .show +.si country-code .si +.silk generic .silk +.sina generic .sina +.singles generic .singles +.site generic .site +.sj country-code .sj +.sk country-code .sk +.ski generic .ski +.skin generic .skin +.sky generic .sky +.skype generic .skype +.sl country-code .sl +.sling generic .sling +.sm country-code .sm +.smart generic .smart +.smile generic .smile +.sn country-code .sn +.sncf generic .sncf +.so country-code .so +.soccer generic .soccer +.social generic .social +.softbank generic .softbank +.software generic .software +.sohu generic .sohu +.solar generic .solar +.solutions generic .solutions +.song generic .song +.sony generic .sony +.soy generic .soy +.spa generic .spa +.space generic .space +.sport generic .sport +.spot generic .spot +.sr country-code .sr +.srl generic .srl +.ss country-code .ss +.st country-code .st +.stada generic .stada +.staples generic .staples +.star generic .star +.statebank generic .statebank +.statefarm generic .statefarm +.stc generic .stc +.stcgroup generic .stcgroup +.stockholm generic .stockholm +.storage generic .storage +.store generic .store +.stream generic .stream +.studio generic .studio +.study generic .study +.style generic .style +.su country-code .su +.sucks generic .sucks +.supplies generic .supplies +.supply generic .supply +.support generic .support +.surf generic .surf +.surgery generic .surgery +.suzuki generic .suzuki +.sv country-code .sv +.swatch generic .swatch +.swiss generic .swiss +.sx country-code .sx +.sy country-code .sy +.sydney generic .sydney +.systems generic .systems +.sz country-code .sz +.tab generic .tab +.taipei generic .taipei +.talk generic .talk +.taobao generic .taobao +.target generic .target +.tatamotors generic .tatamotors +.tatar generic .tatar +.tattoo generic .tattoo +.tax generic .tax +.taxi generic .taxi +.tc country-code .tc +.tci generic .tci +.td country-code .td +.tdk generic .tdk +.team generic .team +.tech generic .tech +.technology generic .technology +.tel sponsored .tel +.temasek generic .temasek +.tennis generic .tennis +.teva generic .teva +.tf country-code .tf +.tg country-code .tg +.th country-code .th +.thd generic .thd +.theater generic .theater +.theatre generic .theatre +.tiaa generic .tiaa +.tickets generic .tickets +.tienda generic .tienda +.tips generic .tips +.tires generic .tires +.tirol generic .tirol +.tj country-code .tj +.tjmaxx generic .tjmaxx +.tjx generic .tjx +.tk country-code .tk +.tkmaxx generic .tkmaxx +.tl country-code .tl +.tm country-code .tm +.tmall generic .tmall +.tn country-code .tn +.to country-code .to +.today generic .today +.tokyo generic .tokyo +.tools generic .tools +.top generic .top +.toray generic .toray +.toshiba generic .toshiba +.total generic .total +.tours generic .tours +.town generic .town +.toyota generic .toyota +.toys generic .toys +.tr country-code .tr +.trade generic .trade +.trading generic .trading +.training generic .training +.travel sponsored .travel +.travelers generic .travelers +.travelersinsurance generic .travelersinsurance +.trust generic .trust +.trv generic .trv +.tt country-code .tt +.tube generic .tube +.tui generic .tui +.tunes generic .tunes +.tushu generic .tushu +.tv country-code .tv +.tvs generic .tvs +.tw country-code .tw +.tz country-code .tz +.ua country-code .ua +.ubank generic .ubank +.ubs generic .ubs +.ug country-code .ug +.uk country-code .uk +.unicom generic .unicom +.university generic .university +.uno generic .uno +.uol generic .uol +.ups generic .ups +.us country-code .us +.uy country-code .uy +.uz country-code .uz +.va country-code .va +.vacations generic .vacations +.vana generic .vana +.vanguard generic .vanguard +.vc country-code .vc +.ve country-code .ve +.vegas generic .vegas +.ventures generic .ventures +.verisign generic .verisign +.versicherung generic .versicherung +.vet generic .vet +.vg country-code .vg +.vi country-code .vi +.viajes generic .viajes +.video generic .video +.vig generic .vig +.viking generic .viking +.villas generic .villas +.vin generic .vin +.vip generic .vip +.virgin generic .virgin +.visa generic .visa +.vision generic .vision +.viva generic .viva +.vivo generic .vivo +.vlaanderen generic .vlaanderen +.vn country-code .vn +.vodka generic .vodka +.volvo generic .volvo +.vote generic .vote +.voting generic .voting +.voto generic .voto +.voyage generic .voyage +.vu country-code .vu +.wales generic .wales +.walmart generic .walmart +.walter generic .walter +.wang generic .wang +.wanggou generic .wanggou +.watch generic .watch +.watches generic .watches +.weather generic .weather +.weatherchannel generic .weatherchannel +.web generic .web +.webcam generic .webcam +.weber generic .weber +.website generic .website +.wed generic .wed +.wedding generic .wedding +.weibo generic .weibo +.weir generic .weir +.wf country-code .wf +.whoswho generic .whoswho +.wien generic .wien +.wiki generic .wiki +.williamhill generic .williamhill +.win generic .win +.windows generic .windows +.wine generic .wine +.winners generic .winners +.wme generic .wme +.woodside generic .woodside +.work generic .work +.works generic .works +.world generic .world +.wow generic .wow +.ws country-code .ws +.wtc generic .wtc +.wtf generic .wtf +.xbox generic .xbox +.xerox generic .xerox +.xihuan generic .xihuan +.xin generic .xin +.xn--11b4c3d generic .कॉम +.xn--1ck2e1b generic .セール +.xn--1qqw23a generic .佛山 +.xn--2scrj9c country-code .ಭಾರತ +.xn--30rr7y generic .慈善 +.xn--3bst00m generic .集团 +.xn--3ds443g generic .在线 +.xn--3e0b707e country-code .한국 +.xn--3hcrj9c country-code .ଭାରତ +.xn--3pxu8k generic .点看 +.xn--42c2d9a generic .คอม +.xn--45br5cyl country-code .ভাৰত +.xn--45brj9c country-code .ভারত +.xn--45q11c generic .八卦 +.xn--4dbrk0ce country-code .ישראל +.xn--4gbrim generic .موقع +.xn--54b7fta0cc country-code .বাংলা +.xn--55qw42g generic .公益 +.xn--55qx5d generic .公司 +.xn--5su34j936bgsg generic .香格里拉 +.xn--5tzm5g generic .网站 +.xn--6frz82g generic .移动 +.xn--6qq986b3xl generic .我爱你 +.xn--80adxhks generic .москва +.xn--80ao21a country-code .қаз +.xn--80aqecdr1a generic .католик +.xn--80asehdb generic .онлайн +.xn--80aswg generic .сайт +.xn--8y0a063a generic .联通 +.xn--90a3ac country-code .срб +.xn--90ae country-code .бг +.xn--90ais country-code .бел +.xn--9dbq2a generic .קום +.xn--9et52u generic .时尚 +.xn--9krt00a generic .微博 +.xn--b4w605ferd generic .淡马锡 +.xn--bck1b9a5dre4c generic .ファッション +.xn--c1avg generic .орг +.xn--c2br7g generic .नेट +.xn--cck2b3b generic .ストア +.xn--cckwcxetd generic .アマゾン +.xn--cg4bki generic .삼성 +.xn--clchc0ea0b2g2a9gcd country-code .சிங்கப்பூர் +.xn--czr694b generic .商标 +.xn--czrs0t generic .商店 +.xn--czru2d generic .商城 +.xn--d1acj3b generic .дети +.xn--d1alf country-code .мкд +.xn--e1a4c country-code .ею +.xn--eckvdtc9d generic .ポイント +.xn--efvy88h generic .新闻 +.xn--fct429k generic .家電 +.xn--fhbei generic .كوم +.xn--fiq228c5hs generic .中文网 +.xn--fiq64b generic .中信 +.xn--fiqs8s country-code .中国 +.xn--fiqz9s country-code .中國 +.xn--fjq720a generic .娱乐 +.xn--flw351e generic .谷歌 +.xn--fpcrj9c3d country-code .భారత్ +.xn--fzc2c9e2c country-code .ලංකා +.xn--fzys8d69uvgm generic .電訊盈科 +.xn--g2xx48c generic .购物 +.xn--gckr3f0f generic .クラウド +.xn--gecrj9c country-code .ભારત +.xn--gk3at1e generic .通販 +.xn--h2breg3eve country-code .भारतम् +.xn--h2brj9c country-code .भारत +.xn--h2brj9c8c country-code .भारोत +.xn--hxt814e generic .网店 +.xn--i1b6b1a6a2e generic .संगठन +.xn--imr513n generic .餐厅 +.xn--io0a7i generic .网络 +.xn--j1aef generic .ком +.xn--j1amh country-code .укр +.xn--j6w193g country-code .香港 +.xn--jlq480n2rg generic .亚马逊 +.xn--jvr189m generic .食品 +.xn--kcrx77d1x4a generic .飞利浦 +.xn--kprw13d country-code .台湾 +.xn--kpry57d country-code .台灣 +.xn--kput3i generic .手机 +.xn--l1acc country-code .мон +.xn--lgbbat1ad8j country-code .الجزائر +.xn--mgb9awbf country-code .عمان +.xn--mgba3a3ejt generic .ارامكو +.xn--mgba3a4f16a country-code .ایران +.xn--mgba7c0bbn0a generic .العليان +.xn--mgbaam7a8h country-code .امارات +.xn--mgbab2bd generic .بازار +.xn--mgbah1a3hjkrd country-code .موريتانيا +.xn--mgbai9azgqp6j country-code .پاکستان +.xn--mgbayh7gpa country-code .الاردن +.xn--mgbbh1a country-code .بارت +.xn--mgbbh1a71e country-code .بھارت +.xn--mgbc0a9azcg country-code .المغرب +.xn--mgbca7dzdo generic .ابوظبي +.xn--mgbcpq6gpa1a country-code .البحرين +.xn--mgberp4a5d4ar country-code .السعودية +.xn--mgbgu82a country-code .ڀارت +.xn--mgbi4ecexp generic .كاثوليك +.xn--mgbpl2fh country-code .سودان +.xn--mgbt3dhd generic .همراه +.xn--mgbtx2b country-code .عراق +.xn--mgbx4cd0ab country-code .مليسيا +.xn--mix891f country-code .澳門 +.xn--mk1bu44c generic .닷컴 +.xn--mxtq1m generic .政府 +.xn--ngbc5azd generic .شبكة +.xn--ngbe9e0a generic .بيتك +.xn--ngbrx generic .عرب +.xn--node country-code .გე +.xn--nqv7f generic .机构 +.xn--nqv7fs00ema generic .组织机构 +.xn--nyqy26a generic .健康 +.xn--o3cw4h country-code .ไทย +.xn--ogbpf8fl country-code .سورية +.xn--otu796d generic .招聘 +.xn--p1acf generic .рус +.xn--p1ai country-code .рф +.xn--pgbs0dh country-code .تونس +.xn--pssy2u generic .大拿 +.xn--q7ce6a country-code .ລາວ +.xn--q9jyb4c generic .みんな +.xn--qcka1pmc generic .グーグル +.xn--qxa6a country-code .ευ +.xn--qxam country-code .ελ +.xn--rhqv96g generic .世界 +.xn--rovu88b generic .書籍 +.xn--rvc1e0am3e country-code .ഭാരതം +.xn--s9brj9c country-code .ਭਾਰਤ +.xn--ses554g generic .网址 +.xn--t60b56a generic .닷넷 +.xn--tckwe generic .コム +.xn--tiq49xqyj generic .天主教 +.xn--unup4y generic .游戏 +.xn--vermgensberater-ctb generic .vermögensberater +.xn--vermgensberatung-pwb generic .vermögensberatung +.xn--vhquv generic .企业 +.xn--vuq861b generic .信息 +.xn--w4r85el8fhu5dnra generic .嘉里大酒店 +.xn--w4rs40l generic .嘉里 +.xn--wgbh1c country-code .مصر +.xn--wgbl6a country-code .قطر +.xn--xhq521b generic .广东 +.xn--xkc2al3hye2a country-code .இலங்கை +.xn--xkc2dl3a5ee0h country-code .இந்தியா +.xn--y9a3aq country-code .հայ +.xn--yfro4i67o country-code .新加坡 +.xn--ygbi2ammx country-code .فلسطين +.xn--zfr164b generic .政务 +.xxx sponsored .xxx +.xyz generic .xyz +.yachts generic .yachts +.yahoo generic .yahoo +.yamaxun generic .yamaxun +.yandex generic .yandex +.ye country-code .ye +.yodobashi generic .yodobashi +.yoga generic .yoga +.yokohama generic .yokohama +.you generic .you +.youtube generic .youtube +.yt country-code .yt +.yun generic .yun +.za country-code .za +.zappos generic .zappos +.zara generic .zara +.zero generic .zero +.zip generic .zip +.zm country-code .zm +.zone generic .zone +.zuerich generic .zuerich +.zw country-code .zw diff --git a/data_test.go b/data_test.go index a0d465f..7e1b0b2 100644 --- a/data_test.go +++ b/data_test.go @@ -119,6 +119,9 @@ func TestShippedMiscReferenceData(t *testing.T) { "protocol": regexp.MustCompile(`^\S+( \S+)*$`), "protocol.name": regexp.MustCompile(`\S`), "protocol.number": regexp.MustCompile(`^\d{1,3}$`), + "tld": regexp.MustCompile(`^\.[a-z0-9]([a-z0-9-]*[a-z0-9])?$`), + "tld.type": regexp.MustCompile(`^(country-code|generic|generic-restricted|infrastructure|sponsored)$`), + "tld.unicode": regexp.MustCompile(`^\.\S+$`), "httpstatus": regexp.MustCompile(`^[1-5]\d{2} \S.*$`), "httpstatus.code": regexp.MustCompile(`^[1-5]\d{2}$`), "useragent": regexp.MustCompile(`^Mozilla/5\.0 .+`), diff --git a/table_test.go b/table_test.go index ff998c9..6e0e348 100644 --- a/table_test.go +++ b/table_test.go @@ -9,6 +9,7 @@ import ( "strconv" "strings" "testing" + "unicode" ) // writeFiles writes a data directory from a map of relative file name, extension @@ -716,6 +717,13 @@ func TestShippedTables(t *testing.T) { "port[465].service": "urd", "port[2049].service": "nfs", "protocol[Transmission Control].number": "6", + "tld[.se]": ".se", + "tld[.se].type": "country-code", + "tld[.com].type": "generic", + "tld[.museum].type": "sponsored", + "tld[.arpa].type": "infrastructure", + "tld[.рф]": ".xn--p1ai", + "tld[.xn--p1ai].unicode": ".рф", "territory[NP].timezone.offset": "+05:45", } { if got := fake(t, f, path); got != want { @@ -830,6 +838,30 @@ func TestEveryTimezoneRowIsWellFormed(t *testing.T) { } } +// TestEveryTldRowSpellsItsKey scans the file rather than drawing: 100 draws leave +// most of 1438 rows unrendered, and the register wraps a right-to-left label in bidi +// marks that belong to its display, not to the label. +func TestEveryTldRowSpellsItsKey(t *testing.T) { + head, rows := shippedRows(t, "tld.tsv", "tld", "type", "unicode") + types := map[string]bool{"country-code": true, "generic": true, "generic-restricted": true, "infrastructure": true, "sponsored": true} + for i, r := range rows { + key, kind, shown := r[head["tld"]], r[head["type"]], r[head["unicode"]] + if !types[kind] { + t.Errorf("tld.tsv line %d: %s is typed %q, which the register assigns no TLD", i+2, key, kind) + } + for _, c := range shown { + if unicode.Is(unicode.Cf, c) { + t.Errorf("tld.tsv line %d: %s displays %q, carrying the format character %U", i+2, key, shown, c) + } + } + if idn := strings.HasPrefix(key, ".xn--"); idn && shown == key { + t.Errorf("tld.tsv line %d: the IDN %s displays its A-label, not a Unicode form", i+2, key) + } else if !idn && shown != key { + t.Errorf("tld.tsv line %d: the ASCII TLD %s displays %q", i+2, key, shown) + } + } +} + // shippedRows reads a shipped misc TSV as the loader does, proving it names the // columns asked for and that every row fills them. func shippedRows(t *testing.T, file string, want ...string) (map[string]int, [][]string) { diff --git a/testdata/shipped_shape.txt b/testdata/shipped_shape.txt index f089db8..d14f80c 100644 --- a/testdata/shipped_shape.txt +++ b/testdata/shipped_shape.txt @@ -274,6 +274,10 @@ misc.timezone.offset string misc.timezone.territory string misc.timezone.weight string misc.timezone.zone string +misc.tld format "{tld}" key tld name unicode +misc.tld.tld string +misc.tld.type string +misc.tld.unicode string misc.useragent format "{ua}" misc.useragent.browser string misc.useragent.device string diff --git a/todo.md b/todo.md index 31865bf..514a1f8 100644 --- a/todo.md +++ b/todo.md @@ -27,6 +27,9 @@ by a `data-import/` script, as README goal 10 asks. - Audit the rest of the shipped set against goal 12 and give each a never-reaching path where it lacks one: `phone` draws live PTS and NANP ranges, `bankgiro`, `plusgiro` and `routing` draw live prefixes. +- Stop `misc.territory[EH].tld` rendering `.eh`, the one shipped TLD `misc.tld` does + not hold: ISO 3166 reserves it for Western Sahara and the root zone has never been + delegated it, so no resolver answers for it and goal 1 promises otherwise. - Accept a middle name, and draw a shipped `personnummer` inside a *selected* sex; both want a draw group sharing its family's pins. Stop the conflict error naming a rewrite that returns a different value where the read it conflicts with sits inside @@ -46,7 +49,6 @@ by a `data-import/` script, as README goal 10 asks. | Category | Shape | Source | Licence | |---|---|---|---| -| `tld` | T | IANA root zone | CC0 | | `loglevel` | c | RFC 5424 severities | facts | | `ip` v4 documentation ranges, private, CIDR; `ipv6` `2001:db8::/32`; `mac` locally administered | t | RFC 5737, 1918, 9637 | facts | | `creditcard` per network: IIN, length, CVV, expiry | T+t | network rules, `{luhn()}` | facts | @@ -108,7 +110,9 @@ by a `data-import/` script, as README goal 10 asks. - Name a spelling that works when a row selector misses: `misc.protocol[tcp]`, `misc.httpmethod[get]` and `misc.territory[se]` all answer "no row … has key or name" and stop there, where a case-insensitive match could name the row that exists, and a - table with no `name` column could say it selects by key alone. + table with no `name` column could say it selects by key alone. `misc.tld[se]` misses + on the leading dot its keys carry rather than on case, so the near miss is worth + naming whatever shape it takes. - Let a table column carry a `datatype`, so `--format json` writes `"safe": true` and `--format sql` a boolean rather than the text `'true'`. Today only a JSON field takes one, so `misc.httpmethod`'s booleans are typed in Go and text