Profiling a country before screening its cities

Distilled from Canada, the first country profiled this way (2026-09-21), which
took 13 candidate cities down to 6 viable ones. Read add-city first: this
sits above its Step 0, not inside it.

Why country-level first. A city screen re-derives the same national facts
every time - the licence template, the privacy statute, whether a geocoder
exists, whether business registration is municipal at all. Those are per
country, and some of them disqualify every city at once. Canada's screen
cost roughly a day and covered six cities; the per-city marginal cost
afterwards is small because the licences rhyme, the privacy regime is shared,
and the portal software repeats.

This argues for depth per country over breadth across countries. Five
cities in one country is cheaper than five countries with one city each.

Run the questions in this order - cheapest disqualifier first

Each one can end the profile.

Which one is cheapest depends on the region, so choose the order rather than
inheriting it. In Canada, rail-first removed 6 of 13 cities for the cost of
reading routes.txt. In Europe and East Asia it removes almost nothing,
because every candidate city has rail - there, question 2 is the cheapest
disqualifier and should run first, and rail becomes a confirmation step. The
2026-09-21 global screen ran rail first out of habit and wasted the step.

Evidence discipline - three rules that cost a day to learn

Build the exhaustive base before filtering. Start from the full universe -
the 2026-09-21 screen began with every country in the Mobility Database
catalogue, 87 of them - so a candidate can be ruled out but never missed by
omission. Istanbul, Sofia, Bucharest and Santiago all surfaced that way, and
none was on the original Europe-and-East-Asia guess the screen started from.

A negative from one method is not a finding. Positives are cheap to trust;
a negative silently ends a line of inquiry, so it earns a second,
differently-shaped probe before it is recorded. Re-testing the ruled-out group
by operator name across the whole catalogue with no country filter - a
different method from the country-then-guess passes - found Tokyo's Toei feed
and Buenos Aires' SUBTE, both missed the first time. Mark a negative
ASSERTED until two methods agree.

A pattern justifies deprioritising, never ruling out. Germany, Italy,
Sweden and Portugal were put in the "ruled out" tier as EU company registers -
asserted from the regional pattern, never probed for any of them - while
Berlin, Naples, Stockholm and Lisbon were simultaneously being rail-confirmed
into the tier above. They sat in two tiers at once, and it was caught only by
rebuilding the table by hand. If a country has not been probed, it stays in
the running with an ASSERTED label on it.

A discard list must name its evidence PER ROW. The rule above was already
written down when the same error happened again: eleven cities - Berlin,
Hamburg, Stockholm, Budapest, Naples, Messina, Hyderabad, Kochi, Tallinn,
Zagreb, Santiago - sat under the heading "the country's business leg failed" in
a file whose own sweep section said Germany and Czechia were unprobed, Sweden,
Hungary, Estonia, Croatia and India were never reached, and Italy was the
one country that passed. Naples was discarded for its country failing, in the
document announcing that its country succeeded.

Knowing the rule did not prevent it, because the discard list was prose in a
different section from the evidence, so nothing forced the two to agree. A
bare list of names cannot be checked; a row reading
Vienna - Austria: GISA strips the street address by design can. So: one row
per city, each carrying the finding that disqualified it, and any row whose
reason is "not reached", "unprobed" or a regional pattern is not a discard at
all - it belongs in the still-open band. Re-read the discard list against the
probe log whenever either changes.

Reading it did not hold either, so a script reads it now. On 2026-09-24 an
audit passed 21 discard rows and, the same night, filling in evidence columns
from the probe log failed five of them. The table carries Kind, Methods
and City host asked? per row, and python scripts/check_discard_evidence.py
refuses an absence resting on one search or on a host nobody asked.

The FIRST record an API returns is not a random one, and n=1 is not a
measurement. Finland was ruled out on a single record - the first the PRH
API returned, a financial holding company with an empty street, a PO box and a
c/o accounting firm. Measured across 500 companies instead, 89.8% carry a
real street address, and the visiting-address type is populated on 100% of
the records that have one. The stated reason for ruling out a whole country
was false.

Registers are ordered by identifier or registration date, so the first page is
systematically the oldest entities - holding companies, dormant shells,
long-established corporates. It is the least representative sample available.
Pull several pages and count. The cost is one loop.

A real premises table can still be a SAMPLE rather than a register - count
the rows. Malaysia's lookup_premise passes every structural test: rows are
individual premises, premise/address/premise_type/state/district are
100% populated, and it downloads as CSV with no account. It is unusable
anyway, because it exists to support a price-monitoring programme and lists
only the premises whose prices are surveyed - 372 in Kuala Lumpur, against
80,110 for Toronto's storefronts, 148,814 for Madrid and 197,276 for Seoul.

This is not the aggregate trap; the rows really are premises. It is a
coverage trap, and schema inspection passes it cleanly - only the row
count catches it. So after confirming the shape, always ask "is this
plausible for a city this size?" before recording a pass. Composition is the
second tell: 131 supermarkets and 69 mini-markets in a city of 1.8 million is a
survey frame, not a register.

And a premises register can be missing the one column that makes it a map.
Name the two columns you need - LOCATION and ACTIVITY - and check each
separately. A register that has one and not the other fails just as
completely as one that has neither, and both halves were seen on 2026-09-22
in the same sweep:

	Rows	Location	Activity	Why it fails
Colombia, RUES nb3d-v3n7	6,369,877	none at all	CIIU codes	Right entity - ESTABLECIMIENTO DE COMERCIO, not the company - and no address, no municipality, not even a city name. The finest unit is camara_comercio, a multi-province chamber region
Jakarta, OSS register	53,827	full street ALAMAT	none	Exactly 9 columns, confirmed against the portal's own component list. uraian_jenis_perusahaan is the legal form (KOPERASI, PT), skala_perusahaan the size. Nothing says what the business sells, so filter_to_storefront() has nothing to filter on

Colombia's is the more surprising of the two, because "6.4 million premises"
reads as a decisive pass right up until you list the columns. An entity-level
register is not a located register. Ask where each row is before counting
how many there are.

Send a nonsense search term before you believe any result list - or any
zero. A server that ignores your search parameter returns HTTP 200 and a
full, plausible page of results, and nothing in the response says the filter
was dropped. data.seoul.go.kr ignored both GET ?srchDetailWord= and
POST searchKeyword=: a real hygiene term and the term zzzzqqq returned
byte-identical pages, and the ten datasets on them looked like hygiene
results because the catalogue's default ordering happens to surface hygiene
datasets. The conclusion drawn from it - "only one dataset has a downloadable
file" - was wrong, and the correct answer (a keyless CSV export covering 3,063
datasets) was two facets away. Cost: one extra request per endpoint.

Corollary: prefer the site's own facet counts to your own sampling. Seoul's
제공유형 SHEET (182) OPENAPI (180) LINK (71) CHART (27), with no FILE row,
settled in one read what paging through 251 results would not - it is the
publisher's count over the whole result set, not an inference from page one.

1. Does urban rail exist, in DATA YOU CAN READ - which is not the same as a GTFS feed

Read this before touching the Mobility Database. The 2026-09-21 global
screen got this backwards and had to be corrected.

This project uses no timetables. It needs station coordinates and line
geometry. GTFS carries those, but so does any national railway GIS layer,
and national mapping agencies publish those routinely - national coverage,
better quality, clearer licence.

Japan is the proof. The Mobility Database holds 18 Japanese feeds, almost
all volunteer-run village buses, and no JR at all. MLIT's National Land
Numerical Information N02 layer holds 10,235 stations and 21,932 line
segments across 178 operators, every JR company included, shapefile and
GeoJSON, Shift-JIS and UTF-8, under PDL 1.0 with commercial use permitted.
Same country, opposite answers, because the wrong source was asked.

Probe in this order. The catalogue is LAST.

1. The national mapping or statistics agency's railway layer - MLIT in
   Japan, and its equivalent elsewhere. Ask TWO questions of it, not one:
   does it carry station points with coordinates, and does it carry line
   geometry? This project draws and labels every transit line, so stations
   alone are not enough. Japan's N02 has both - 10,235 stations and 21,932
   line segments. Korea's station standard dataset has 1,099 stations with
   WGS84 coordinates and no lines at all, which passes the first question and
   fails the second. A screen that asks only about stations records a false
   pass.
2. The national open-data portal's transit standard datasets - Korea
   publishes nationwide urban-railway station and line standard datasets.
3. The agency's own GTFS, where one is published.
4. The Mobility Database catalogue - a mirror of a mirror, demonstrably
   stale and incomplete. Useful for breadth, not for a verdict.

Commuter rail is not urban rail here, matching every built city: basic
route_type 2, and extended 109 (Suburban Railway, the S-Bahn family).

Rail-first is right for a country already known to publish transit data,
and wrong as a global filter. In Canada it removed 6 of 13 cities before a
catalogue was opened. Across Europe and East Asia it removes almost nothing,
because every candidate city has rail - so the business-data question does the
work instead, and should run first.

Four ways this step returns a confident wrong answer

All four happened in one day.

• Extended route types. An agency may publish only the TPEG-derived 3- and
  4-digit set, where a metro is 401, an underground 402, a tram 900.
  Reading only basic 0-12 reported Berlin, Hamburg, Stockholm and Oslo as
  having no urban rail. screen_rail.py handles both sets now.

• Wrong-feed selection. Picking "a feed that names the city" is not
  picking the rail operator. Barcelona and Madrid were tested against bus
  companies, Dublin against airport coaches, Malaysia against a bus operator
  in Kuala Terengganu. In a country with 169 feeds that is close to random -
  search by operator name across the whole catalogue, with no country
  filter, which is also how two feeds missed by country-filtered passes were
  found.

• A national portal that is unreachable. Before writing that down, work
  through all four causes, because three of the four have bitten:

  1. A transient outage. data.go.kr timed out from two independent
     networks and answered 200 an hour later. Retry before recording.

  2. The wrong hostname. datosabiertos.gob.pe fails; www.
     datosabiertos.gob.pe answers. Peru was nearly written off over a vhost.
     Always try www.

  3. A refusal, which is TWO different things - read the block page.

     • Client-signature refusal: a bare requests user agent is rejected
       where a real browser is not. Chicago's data terms do this. Use the
       browser, as read-licence says.
     • IP-level refusal: the block page names your IP address or shows
       a WAF request ID. The browser is useless - it shares the address.
       Tested 2026-09-21: data.go.th and opendata.tel-aviv.gov.il both
       refused the browser pane and printed 50.47.238.226 back, the same IP,
       one with a Cloudflare Ray ID and one as Status Code: 472. The trigger
       is a different network or region, plausibly in-country.

     This distinction was got wrong first: "bot-protected, so try the browser"
     was carried over from the licence workflow without testing it, and a
     country was kept warm on a false rationale. If the block page prints
     your IP, the client is not the problem.

  4. A genuinely dead host. Only after the first three. And note whether it
     is one host or a whole domain: every *.cdmx.gob.mx host times out,
     which is a more durable finding than one portal being down.

  Then try the CITY's own portal, which is the scope this project works at
  anyway. This has now paid off three times - Mexico via Guadalajara, Korea
  via data.seoul.go.kr, and Israel, whose national portal carries a business
  register for Be'er Sheva, a city with no rail, while Tel Aviv publishes its
  own. A national portal is the wrong default for a city-scoped project.

• A feed that exists and is broken. Buenos Aires' SUBTE feed is published
  on the city's own CDN and contains no routes.txt; Melbourne's PTV feed is
  a nested zip; Dubai's catalogue entry is an anonymous personal GitLab job
  artefact. Absent and broken are different findings.

• A download button that delegates to another portal. Check where the file
  actually lives before concluding a city portal publishes it. Busan's and
  Daegu's FILE buttons both point at
  www.data.go.kr/cmm/cmm/fileDownload.do?atchFileId=…&fileDetailSn=…, so
  neither city hosts its own data. Two consequences, both measured 2026-09-22:

  • The delegated id goes stale silently. Daegu's resolved correctly.
    Busan's returned 0 bytes at fileDetailSn 1 and 2, an unrelated
    PNG at 3 and a JPEG at 4 - a wordmark image from some other
    dataset. Content-Disposition named the PNG honestly; had the header been
    trusted instead of the magic bytes, the file would have been recorded as
    downloaded. Check magic bytes, not the filename the server claims.
  • A CAPTCHA on the download path disqualifies the source, however open
    the licence. data.go.kr routes downloads through
    /cmm/cmm/check-limit.json → needCaptcha → showLimitCaptcha, a rate
    limiter. This project does not defeat CAPTCHAs, so a source needing one
    download per district per refresh is not a pipeline - record it as closed
    and say why.

2. Does the country record WHERE COMMERCE HAPPENS, or only where companies are REGISTERED?

The biggest structural discriminator, and the sharp form of the question. An
earlier version of this skill asked "does the country license businesses
municipally", which is Canada's framing and mis-sorts half the world:
municipal-vs-national turns out to be an unreliable proxy for the question
that actually decides things.

A company register publishes a registered office - frequently an
accountant's address or a holding company's mailbox. Mapping those produces a
map of bookkeepers. Companies House, Australia's ABN, Japan's corporate-number
system and most of the EU's registers are this, and they fail here however
open they are.

Four shapes qualify. Sort a candidate into one before going further:

Shape	Examples	Notes
Municipal licence register	US, Canada, South Korea (195 permit types)	The shape this project is built for
National establishment register	France SIRET, Mexico DENUE, Norway beliggenhetsadresse	One source covering every city - see the caveat below
Premises field survey	Montréal locaux-commerciaux, Barcelona cens de locals	Records what is on the street rather than who registered; arguably the best answer to this project's premise
Sector inspection register	UK FSA food hygiene	One bucket only - Boston's shape, and a ceiling not a floor

Look for the field name that proves it. Norway's register keeps
beliggenhetsadresse (location address) deliberately distinct from the
registered business address; that distinction is the answer. France's
établissement is not the same object as its unité légale.

If every city reads the SAME BYTES, cache it per COUNTRY, not per city

add-city's scaffold gives each city its own data/<city>/raw/, which is right
for a city's own registry and a city's own feed. It is wrong for a national
register, and the cost compounds silently with each city added.

France is the worked example. SIRENE's two parquets are 2,210 MB and 811 MB
and identical for every French city, so the per-city layout would have held
15 GB to store one 3 GB pair across five cities. Fixed 2026-09-23 while
building the second city — deliberately before a third could set the pattern —
by putting SHARED_RAW in pipeline/countries/france.py and having each city's
config import the paths. data/*/raw/ already gitignores data/france/raw/, so
nothing about the repository changes.

The test is "the same bytes", not "one national source", and Mexico is the
counter-example that keeps it honest. DENUE is equally national, but it is
distributed per entidad federativa: Mexico City pulls state 09 and
Guadalajara state 14. Different files, no duplication, nothing to share — and
sharing them would be a worse design, because a city needs one state and not
thirty-two. Measured 2026-09-23: 45 MB and 39 MB, no overlap.

So when a country profile lands on a national register, ask which one it is:

	Cache	Example
One file, every city	per country (SHARED_RAW)	France SIRENE
Partitioned by region, cities in different partitions	per city, as scaffolded	Mexico DENUE
Partitioned, and two cities share a partition	per partition	none yet

The caveat on national registers. One source covering every city is a
different and possibly better shape, but it is not the shape this project
is built for: scoping and taxonomy were both designed around per-city
municipal registers, and a national register includes every office, depot and
administrative site alongside the shopfronts. That is a design decision to take
before a profile, not a fact to discover during one.

Measure COMPOSITION, not just presence - most registers are mostly not shops

"Does it have addresses" is the easy half and rarely the deciding one. The
deciding question is what the register is actually made of, and the answer
is reliably worse than the dataset title suggests:

Register	Share that is NOT a storefront
Philadelphia L&I	79% landlord registrations
Washington D.C.	61% residential rentals, plus 11,074 General Business office catch-all
Finland PRH, Helsinki	53% NACE 68 real estate - every apartment building is a Kiinteistö Oy or Asunto Oy
Vancouver	10,698 Long-term Rental, 3,910 Short-term Rental Operator, 3,575 General Contractor
Toronto	Taxicab Owner 4,212, Public Garage 3,023, Building Renovator 1,461

This is a family, not a series of one-offs: a business register records who
registered, and in most countries that is dominated by property-holding
entities and non-premises trades. The project already filters all of these by
category, successfully - so a high share is a cost, not a disqualifier.

The three numbers to produce for any candidate register, from several
pages rather than one:

1. Address presence rate - what fraction carry a real street, not a PO box
   or a c/o.
2. Category composition, top 10-12 - what the register is mostly made of.
3. The share in the project's three buckets - retail, food service,
   personal services. Finland's retail is 4.8%, and food and beverage and
   personal services do not reach its top twelve. That is the number that
   decides whether a country is worth a profile.

And then the question that composition does NOT answer: once filtered to
the storefront categories, are those rows shop premises or head offices? A
company register can pass all three tests above and still map one pin per
chain rather than one per shop. That needs its own probe.

3. What portal software, and does it refuse automated fetches?

Canada used all four of CKAN, Socrata, Opendatasoft and ArcGIS Hub. The
existing fetch code speaks all four, so this is usually cheap - but check for
refusals early, because they are silent:

Portal	What bit
donnees.montreal.ca	RBAC: access denied to plain curl; the project's identified agent is served. Never a browser string: a host refusing both is a refusal (owner, 2026-10-04)
ArcGIS Hub (Surrey)	/csv returns HTTP 202 with an async job, CSV on a later call
Toronto CKAN	datastore_search_sql 404s; use datastore_search with filters
Opendatasoft	CSV exports are semicolon-delimited, not comma

Open the portal in a real browser BEFORE concluding anything about it

Standing rule, and it is not a last resort. Guessing API paths answers
"this path 404s"; navigating the site answers "what this portal is". The 2026-09-22
sweep ran both against the same five portals and the browser won every time,
usually in one page load:

Portal	What guessing said	What the browser said
GeoMedellín ArcGIS Hub	data.json 404, api/search/v1 401	"Please sign in. This site requires credentials to access." The site is private - no path exists to find
data.gov.my	catalogue API paths all 404	Server-rendered: all 292 datasets are in the HTML, no API involved
datosabiertos.gob.pe	"Drupal, not CKAN, at every path tried"	Serves /profiles/dkan/* - it is DKAN, which speaks CKAN at /api/3/action/*. The earlier tries omitted the 3
satudata.jakarta.go.id	"API paths resolve nowhere"	Watching its own requests gave /backend/api/v2/satudata/* and a fully parameterised /search?q=... URL
FEHD (Hong Kong)	two guessed URLs 404'd	The real page was two clicks from the licensing index

Four of those five were recorded as "needs the browser" and every one of
them then resolved. So make it the first move on an unfamiliar portal,
not the fallback after a round of 404s:

1. Load the site and look at it. A sign-in wall, a CAPTCHA or a dead app
   is a verdict, and it arrives in one screenshot.
2. Read the network requests while its own UI works. The app calls its
   real API in front of you - endpoint, shape and all. This beats guessing
   even when guessing would eventually succeed, and it is how Seoul's keyless
   download.do and Jakarta's search-v2 were both found.
3. Check whether the page is server-rendered. No recorded XHR means the
   data is in the HTML; read the HTML.
4. Keep the parameterised URL the UI produces. Jakarta's search URL
   carries q, organisasi, status and page_no, which turns a JS app
   into something enumerable - and a nonsense q still gave the control
   (zzqqxx -> 0 results, so that search genuinely filters).

The same applies to a fetch that is refused rather than empty. Record
"could not reach" only once the browser has failed too - the rule
read-licence step 7 states for licence pages, generalised to the whole
probe.

But do not read a refusal's cosmetics as its cause. Bulgaria's 403 was
recorded here as a stock Apache page, i.e. a client-signature refusal worth
retrying in a browser. The browser returned the same 403, and DNS-over-
HTTPS showed the host up. The page's blandness said nothing about the
reason; only trying it did.

When the browser fails too, the portal is not the only copy

A blocked portal blocks a HOST, not a FACT. Government open data is
harvested upward - municipal into national, national into supranational -
and each tier is a different host with a different opinion of you. Three
routes, cheapest first:

Route	Use when	Worked on
National catalogue	a city portal is searchless, paginated, or down	datos.gob.es for Bilbao (600 federated vs 344 own)
data.europa.eu	a whole EU country refuses you	Bulgaria — see below
Internet Archive	a page is CAPTCHA-walled or retired	Barcelona's and Sevilla's licences
The dead page's own HTML	a portal is retired and you need its successor	Sevilla — see below
The platform's API, not the portal's UI	a Hub/catalogue front-end demands sign-in	Sevilla again

A retired portal names its successor, in its last capture. Sevilla's
datosabiertos.sevilla.org is NXDOMAIN and was written off as gone. Its
final Archive capture — 2024-04-25, already serving 503 — still carried,
in its own JavaScript, a KML URL pointing at the ArcGIS org that replaced it.
One CDX query. Fetch the last capture and read its links before concluding
a portal left no forwarding address.

And a sign-in wall on a Hub is not a verdict on the data. Sevilla's
ArcGIS Hub says "Can't access this content … Sign In", which was recorded
here as "the Medellín shape", i.e. private. It was not: sharing/rest/search
served 1,260 public items / 413 Feature Services anonymously, and
services1.arcgis.com/<orgId>/… served every layer. The Hub is a
front-end; the REST API is the data. Before accepting a wall, ask the layer
underneath it:

https://www.arcgis.com/sharing/rest/search?f=json&num=100&q=owner:<account>
https://services1.arcgis.com/<orgId>/arcgis/rest/services/<name>/FeatureServer/0
    ?f=json                                   <- fields, geometry, wkid
    /query?where=1%3D1&returnCountOnly=true   <- the row count that decides it

Get the account or org id by reading results, not by guessing:
owner:idesevilla returned 0 while the correct Ayto.Sevilla was sitting in
a search result already printed. Then enumerate by owner — a keyword
search over ArcGIS Online returns the whole world's content (title:Sevilla
gave 1,314 items, mostly other people's).

None of these is a bypass. They are other publications of the same public
record, which is why they are fine where defeating the block would not be.

Bulgaria, 2026-09-22, as the worked example. data.egov.bg 403s curl
and browser alike. data.europa.eu's SPARQL endpoint harvests it, so
11,635 Bulgarian datasets were enumerated without touching the blocked
host - enough to settle Sofia as a firm negative rather than an unknown.

The endpoint is a Virtuoso holding ~1.9M datasets, and it times out on
anything that must see them all. The ladder, measured:

COUNT / GROUP BY  over a filter   -> 504, every time
FILTER CONTAINS   unscoped        -> 504, even the nonsense control
LIMIT 15          streaming       -> works
OFFSET                            -> works to ~500, dies by 2000
VALUES <uri> <uri> ...            -> instant, any volume

So convert the search into a lookup. Dataset URIs there are sequential
integers (…/88u/dataset/18209); one VALUES block of 400 explicit URIs is
an indexed fetch that cannot time out. Map the id space with one sparse
sweep, then densely sweep the bands that answer. Match keywords locally in
Python, never server-side - titles are machine-translated into ~24 EU
languages, so a server-side match hits one variant and misses the rest.

Two traps this route carries, both measured:

• dct:spatial is COVERAGE, not provenance. Filtering datasets to
  country/BGR returns 8,157 - of which 6,394 are Eurostat and zero
  have a Bulgarian publisher. It means "this study covers Bulgaria". The
  language tag is the better proxy for who published.
• The harvest is METADATA-ONLY. 0 of 20 sampled records carried any
  dcat:distribution. You learn what exists; the file stays behind the
  wall. Enough to screen a blocked country, not to build one - so use it
  to decide whether to spend more effort, never as the data source itself.

4. How does the country define PERSONAL INFORMATION, and does its licence carve it out?

Do this before any city's privacy work, because the answer may be a licence
condition rather than a policy choice. Every Canadian Open Government Licence
excludes Personal Information from the grant and defines it by pointing at the
province's own statute - so the carve-out cannot be read without the statute,
and the provinces did not agree with each other.

What to establish:

• Is information about an individual in a business capacity excluded from
  the definition? (Ontario: yes, explicitly, even when the business is run
  from a dwelling. BC: yes for "an individual at a place of business",
  which does not clearly reach a dwelling. Québec: no - it narrows the
  Act's reach rather than excluding the information, and frames the exemption
  around "une fonction au sein d'une entreprise", which does not obviously
  cover a sole trader who is the business.)
• Is the regulator's interpretation published? Ontario's IPC bulletin is
  stored in docs/licenses/ because it is load-bearing.

Expect to be wrong here. An early reading during the Canada profile
concluded that publishing a sole trader's name would breach the licence. The
statute says the opposite for Ontario. Read the definition, never infer it
from the carve-out.

5. Is there a residence / owner-occupancy signal, and where does it live?

Every US city in this project infers "is this someone's home?" from a parcel
join - assessor land use plus an owner-occupancy or homestead flag. That
chain does not exist in Canada: no province publishes owner-occupancy. BC
Assessment is not open data.

Canada compensates with something more direct, at the licence level:

• Surrey: LicenseType = 'Home Occupation' (14,015 of 27,082)
• Edmonton: business_address = '<Home Based Business>' (and those rows carry
  no coordinates, so they cannot be mapped at all)
• Calgary: homeoccind
• Vancouver: nothing - the only one needing the US-style parcel join

So ask: does the licensing authority state that a business is home-based?
That is better evidence than a parcel inference, because the city asserts it.
If not, find the zoning/assessment layer and plan a spatial join.

6. Is there a national geocoder, or must it be per-city?

The US Census bulk geocoder is free, national and keyless. There is no
Canadian equivalent, and that sounds like a blocker until you check what
actually needs geocoding:

City	Coordinates	Real address, none	Needed?
Montréal, Surrey, Calgary	100%	0	No
Edmonton	53.3%	111	No - the rest are placeholders
Vancouver	50.8%	1,583	Marginal
Toronto	0%	159,872	Yes

Only one city of six genuinely needed it, and the answer was not a geocoder:
join against the city's own address-point layer. Toronto's One Address
Repository (525,440 points, same licence as the business data) matched 71.4%
on exact string match with no street normalisation, and returns
MUNICIPALITY_NAME, so it doubles as the in-city filter.

A row missing coordinates is not automatically a row needing geocoding. It
may have no address either - Edmonton's placeholders, Vancouver's rentals and
contractors.

⛔ Before writing ANY geocoder: find the address file that already carries the coordinate

The procedure is the address-join skill — find the file, measure the join
against a control, report tiers, read the misses. This section is why.

Four for four, the same week. Every country this project filed under
"geocoding at national scale" turned out to publish an address-point file
with coordinates, and the geocoding project became a JOIN - or vanished:

Country	The file	Result
Czechia	RÚIAN, keyless	a join, 99.8%
Denmark	DAWA, keyless	a join on the DAR UUID, 96.9%
Brazil	IBGE's census address file (CNEFE 2022)	no geocoder AND no register - it records every establishment with a field coordinate
Taiwan	each city's door-plate file (門牌位置數值資料), from the civil-affairs bureau	a join, 92.4-95.5% in four cities

Where to look, in order: the census bureau's address frame; the national
mapping agency's address or location-reference layer; the municipal
civil-affairs or registry office (it numbers the doors). Never under
"geocoding" - none of the four was filed there. Enumerate the catalogue
(its export, not its search API - Taiwan's ER0001 key error was one
endpoint); read the rows' fill, not the header.

Then measure the join with a CONTROL city that must reproduce before any
other counts - it caught a 0.8-point parser regression in Taiwan that would
have shipped silently in three cities - report match TIERS (exact /
degraded / unplaceable), and read the misses by place: Brazil's dropped
rows under-draw affluent districts, Taiwan's are market stalls.
docs/geocoding_retrospective.md carries the full method and the Japan
plan; scripts/screen_taiwan_join.py is the reusable harness.

7. What classification standard, and is the field single-valued?

Canada uses NAICS (it is a trilateral US/Canada/Mexico standard, so
pipeline/taxonomies/naics.py may transfer unchanged - Montréal's SCIAN is
NAICS in French, populated on 99.6%). Municipal licence registers mostly use
local taxonomies instead, which the per-city module pattern already handles.

The trap that is not in the US playbook: three of six Canadian registers
store SEVERAL categories per row, each with a different delimiter.

City	Naive distinct	Delimiter	True distinct
Calgary	1,169	,\n	173
Edmonton	1,534	;	67
Surrey	628	\n	210

A value_counts() on the raw column returns combinations. A taxonomy module
built on 1,169 Calgary "categories" would be nonsense. Split first, then a
multi-licence premises needs a dispatch rule - Boston's FT+RF question again.

8. Language and encoding

• Declare SOURCE_ENCODING per city (already an add-city requirement).
  All six Canadian sources are UTF-8; Québec data is often latin-1.

• Mojibake in a terminal is usually the console codepage, not the file.
  Set PYTHONIOENCODING=utf-8 before blaming the data.

• NORMALISE DASHES, APOSTROPHES AND UNICODE FORM BEFORE ANY JOIN. This has
  already cost time once: Montréal's commercial survey writes
  Ahuntsic–Cartierville with an EN DASH where the City's own boundary layer
  uses a hyphen, across six borough names, which made a 34-feature layer look
  like it covered 32. Same class: L’Île (U+2019) against L'Île (U+0027),
  and Baie-d'Urfé against Baie-D'Urfé. Map U+2013/U+2014 to U+002D,
  U+2018/U+2019 to U+0027, apply NFC, and compare with casefold() rather
  than .upper(). Two files can look identical on screen and fail to match.

• Check whether the feed ships translations.txt before transliterating a
  non-Latin name by hand - TransLink's and STM's both do.

• Column names in another language are already solved - each city's config
  names its own columns and step 2 renames to the taxonomy's VALUE_COLUMN.
  NOM_ETAB is no harder than dbaname.

• Category labels belong to the taxonomy module, which the project's own
  invariant already makes the single owner of tooltip and legend text. Decide
  there whether to translate; prefer English bucket labels with the source
  value in the tooltip, so cross-city comparison survives without hiding the
  source. An English label is reader text, so it is American English
  ("License type", "Liquor store"); the source value stays as written
  (docs/city_page_format.md, section 4; premises-taxonomy Step 8).

• Business names stay in their own language, always.

• NORMALISE FOR JOIN KEYS, NEVER FOR DISPLAY. The stdlib does all of this;
  no package is needed, which matters because requirements.txt has to stay
  lean for Streamlit Cloud.

  • unicodedata.normalize("NFKC", s) fixes the full-width numerals in
    Taiwanese addresses - 濱海一路２３號１樓 becomes 濱海一路23號1樓, which
    is the difference between a geocode match and a miss. It also folds
    Japanese half-width katakana (ﾏｸﾄﾞﾅﾙﾄﾞ to マクドナルド) and leaves
    Korean untouched.
  • Accent folding for a join key is NFD plus a combining-mark filter:
    "".join(c for c in normalize("NFD", s) if not combining(c)) turns
    Montréal into Montreal, Plzeň into Plzen, Rīgas into Rigas.
  • THE TRAP: NFKC CHANGES DISPLAY TEXT. Ⅳ号店 becomes IV号店,
    ㈱丸井 becomes (株)丸井, Ｃａｆｅ becomes Cafe. Every one of those
    is a shop's actual name being quietly rewritten. Normalise a copy used
    for matching and keep the original for the tooltip - which is the rule
    above, stated mechanically.

• The map's font stack is in pipeline/theme.py as FONT_STACK, and it
  matters for non-Latin names. A bare sans-serif lets the browser choose:
  missing glyphs render as tofu boxes, and a substituted face changes line
  metrics so a tooltip outgrows its box. The order is deliberate - browsers
  fall through per glyph, so Latin/Greek/Cyrillic faces come first and CJK
  after, because Segoe UI and Noto Sans carry no CJK glyphs and a Japanese
  name therefore falls past them to Yu Gothic or Hiragino. Putting a CJK face
  first would restyle every Latin name on every map.

• Search the catalogue in the local vocabulary. Montréal was almost ruled
  out because locaux-commerciaux - a 28,621-premises survey that is the best
  source in the project - contains none of the words business, licence,
  permis, entreprise or commerce. Which is why add-city Step 0 now
  requires reading the whole catalogue rather than grepping it.

How a country's cities differ from the US ones

Carry this table into the first build; it is what actually changed.

	US pattern	Canada
Licence regime	Heterogeneous - PDDL through to MTA forbidding modification	One OGL template, 4 of 6 near-identical
Attribution	Varies; several require none	Prescribed wording, 5 of 6
Breach	Usually unspecified	Automatic termination in 3 of 4 OGLs
Personal information	Project chose its own line	Carved out of the licence grant
Home signal	Inferred: parcel land use + owner-occupancy	Stated on the licence
Owner-occupancy data	Open (assessor rolls)	Not open anywhere
Geocoding	Census bulk geocoder, national	None - city address points
Classification	NAICS usually	Local, and often multi-valued
Coordinate quality	LA had ~9% corrupt	99.8-100% clean in all six
Transit licence	The loosest end of every review	Mostly the city's own OGL

The country file is PROVISIONAL, and promoting it is a step, not a tidy-up

This is the failure mode this skill has actually produced, twice, and it is
invisible at the moment it happens.

A country profile ends with docs/<country>_step0_endpoints.md: every endpoint
fetched and verified, written in the future tense because no city is built yet.
"When one is, these rows merge into data_sources.md." Then a city IS built,
and the person building it is following add-city, which tells them to record
their city's sources — so they record the notices, which are the visible
obligation, and the country file goes on saying "no city is built" while five
of them are.

Canada, 2026-09-21. Six municipalities built. Notices recorded that day,
correctly. The endpoints never reached the three provenance tables, the five
transit feeds never reached the GTFS licence table, the Province of British
Columbia's naming layer was never read at all, and three files — the country
endpoints file, docs/licenses/canada-required-notices.md and the licence
store's own README — all still said "none built". Found and closed 2026-09-22.

Mexico, 2026-09-22. Mexico City and Guadalajara built the same week. Same
omission, found the same day by scripts/check_provenance.py.

So when the FIRST city in a profiled country is committed:

1. Run python scripts/check_provenance.py. It fails until the city's
   rows exist in all three provenance tables and every URL its config.py
   resolves to is recorded. Do not add the city to KNOWN_GAPS to make it
   pass — that list is for defects someone has decided to carry, with a date,
   and it may only shrink.

2. Promote the country file's rows into a new
   docs/data_sources/<country>.md (plus the country's row in the index at the
   top of docs/data_sources.md), writing
   them from each city's own config.py rather than copying them. The copy is
   not the same document: for Canada, re-deriving corrected three Step 0
   findings that had hardened into confident claims — a CRS trap that belonged
   to a different endpoint than recorded, a category count that was an artefact
   of splitting on the wrong delimiter, and a "this feed has no feed_info.txt"
   that was true of the mirror and false of the agency.

   The new file renders on About the Data, so write it for the reader
   (docs/city_page_format.md, sections 2 to 5): rows and notices per city,
   each city's own gaps and limits in its own rows or section, American
   spelling outside notices, licence titles and quotes, and no process notes
   (check scripts, DECISIONS.md, PLAN.md, skill or agent names, briefs,
   the step-0 file itself) except between <!-- internal --> and
   <!-- /internal -->. An existing country file's opening paragraph shows
   the pattern (docs/data_sources/latvia.md). How the file is named and
   filed is the next section.

3. Relabel the country file as the evidence trail, with an explicit
   precedence line ("where this file and data_sources/<country>.md
   disagree, data_sources/<country>.md wins") and the corrections named. Do the same for any
   docs/licenses/<country>-required-notices.md and for the licence store's
   README section. Grep the repo for the country's name plus "not built",
   "none built", "if any", "would require" and "eventually" — every one of
   those is a sentence written in the future tense about a present that has
   arrived.

4. Renumber the notices list if you appended a block to it. Canada's block
   went in after Mexico's without renumbering and produced two item 8s and two
   item 15s in the list that gates the public deploy. check_provenance.py
   fails on this now.

The new country on the reference pages: its name, file, region and headings

What Is Excluded and About the Data show one country at a time, chosen in a
two-row selector (a region, then that region's countries), and every city
page links to both opened on its country (render_country_links). The first
city in a new country is what puts the country there. The rules are in
docs/city_page_format.md, section 2, and app/country_sections.py's
docstring; what a new country needs from them:

1. One country string, spelled once. scaffold_city.py --country
   writes it into the city's entry in app/cities.py. The selector, the
   ?country= deep link and the Cities menu all derive from it
   (cities.COUNTRY_ORDER, built from CITIES; there is no list to edit), so
   every later city in the country uses the identical string.

2. The country file, named from that string.
   docs/data_sources/<slug>.md, where the slug is
   country_sections.country_slug(): lowercase, spaces to hyphens ("South
   Korea" -> south-korea.md). A misnamed file still renders, but at the
   foot of About the Data under every country, while the country's own view
   warns that its file is missing. Add the country's row to the index table
   "Where each country's sources live" in docs/data_sources.md.

3. A region, which decides its selector group. The group is the city's
   macro-map "region", folded through REGION_GROUP in
   app/country_sections.py:

   • an existing region (every European country is Europe, per
     add-city; an Australian or New Zealand city Oceania; another South
     American country South America) joins that group with no edit;
   • a region new to the project (scaffold_city.py --new-region, which
     appends it to REGION_ORDER; a macro-map view, so the owner's call)
     becomes a selector group of its own, under the region's name;
   • a sub-view of an existing group (as France North and South,
     Czechia and the Seoul Capital Area are) needs a REGION_GROUP line
     mapping it to that group, or it shows as a group of its own. On the
     macro map it also joins cities.COUNTRY_VIEWS, so the region menu lists
     it after every larger view (owner, 2026-10-02).

   The country's place within its group follows REGION_ORDER and build
   order; there is nothing to set.

4. Headings that name the city. A section is filed under a country when
   its heading names exactly one country's city (as app/cities.py names
   it, less any " (Regional)") or the country itself; a heading naming
   neither inherits its parent's. A heading or notice title that names only a
   publisher ("TransLink", "Tailte Éireann") would show under every country,
   which is why those two are in ALIASES in app/country_sections.py.
   Numbered notices in docs/data_sources.md are filed by their title (the
   text before the first dash) the same way. Name the city in the heading;
   add an alias only where a title has to name its publisher.

REGION_GROUP and ALIASES live in app/, so an edit to either lands with
the city's publish (publish-city).

Profile by SOURCE KIND, not by city — there are more than three

Questions 1, 2 and 5 below ask for rail, commerce and boundary, and a profile
that answers them feels complete. It is not, and the gap is the same every
time: the sources that belong to no single city.

• A naming layer — the polygon set that says which municipality an excluded
  station is in. Vancouver's is the Province of British Columbia's, which is a
  different publisher from either city in the build and needed its own licence
  read and its own required notice. D.C.'s is the US Census, which needed
  neither, and drawing the general lesson from that one was the mistake.
• A residence join — the parcel or assessor layer that decides whether a
  licence is somebody's home. Four cities use one and it is published by a
  county or a province, not by the city whose map it filters.
• A geocoder, where the register carries no coordinates.

For each, ask the same three questions as for the main three: who publishes it
(often not the municipality), what does its licence require, and does it
prescribe a notice. Budget a notice per source, never a notice per city —
the US sources mostly prescribed no wording and the Canadian ones almost all
prescribed their own.

What to produce

1. docs/<country>_step0_endpoints.md - every endpoint fetched, per leg
   (business, transit, boundary), with the traps. Merge into
   docs/data_sources/<country>.md only when a city is actually built, as
   the two sections above describe.
2. Licence texts in docs/licenses/, one per publisher, with source URL,
   retrieval date and SHA-256 in that directory's README.
3. A privacy note working through each jurisdiction's definition.
4. A required-notices file with exact wording per city, to be promoted into
   data_sources.md's numbered gate when a city is committed - not before, so
   the deploy gate stays a list of things that actually apply.
5. A DECISIONS.md entry with the ranking and its evidence.
6. docs/city_master_list.md gains the country's cities, each in one band
   or in DISCARDED; the evidence behind them goes in
   docs/global_country_shortlist.md. (docs/city_shortlist.md was retired
   2026-09-27.)

Rank ALSO on cost per marginal city - it decides how far a budget goes

Station density (below) measures value per city. This measures cost per
additional city, and the two are independent. A fixed research budget goes
several times further in a country where the second city is nearly free.

Ask: is the business data NATIONAL, STANDARDISED, or BESPOKE per city?

Shape	Marginal city costs	Examples
One national register	A boundary file and a rail check. The data is already downloaded	France SIRENE · Korea 상가(상권)정보 · Mexico DENUE · Taiwan 商業登記 · Brazil CNPJ
National STANDARD schema, per-municipality publication	A download per city, but identical columns, so one parser serves all	Japan 推奨データセット · Korea 표준데이터
Bespoke per city	A full integration each time	US · Canada · Spain · Italy

The 2026-09-21 screen got this wrong by omission, and it distorted the
ranking. Each country was assessed as though it were one city - France as
"Paris", Korea as "Seoul", Mexico as "Guadalajara", Taiwan as "Taipei". In
fact:

Country	Metro cities reachable from the SAME source
Japan	~9 - Tokyo, Osaka, Nagoya, Yokohama, Kobe, Kyoto, Fukuoka, Sapporo, Sendai
France	6+ - Paris, Lyon, Marseille, Lille, Toulouse, Rennes
Korea	6 1 - Seoul only. Corrected 2026-09-22, see below
Brazil	6+ - São Paulo, Rio, Belo Horizonte, Brasília, Recife, Porto Alegre
Taiwan	4 - Taipei, Kaohsiung, Taichung, Taoyuan
Mexico	3 - CDMX, Guadalajara, Monterrey

Canada is the worked counter-example and explains its cost. Six cities
needed four different portal types - Toronto CKAN, Vancouver Opendatasoft,
Calgary and Edmonton Socrata, Surrey ArcGIS Hub, Montréal CKAN - so six cities
meant six integrations. That is why the Canadian screen cost a day, and it is
not a fact about Canada's data quality.

Ask WHICH TIER OF GOVERNMENT LICENSES, not which one publishes - and check
one non-capital city before believing a country count. Korea's "6 cities from
one source" was the single largest error in the 2026-09-21 screen, and it
survived because only Seoul was probed. In Korea the licensing authority is the
district (자치구/군), so the register is published per district, in whatever
schema and on whatever schedule that district chose:

	Districts publishing a usable premises register
Seoul	25 of 25 - the city aggregates. Verified: 강남구 4,744 → 도봉구 794
Busan	4 of 16. Others are single-sector (식품소분업 only) or enforcement actions, which is a violations list, not a register
Daegu	4 of 9 - and the missing one is 중구, the downtown, where all three metro lines converge

So the capital was the exception and was mistaken for the rule. A single
aggregating city says nothing about the other five. The generalisable check:
probe the second city, and probe its central district specifically - a
country count built from the capital alone is a guess wearing a number. Partial
district coverage is also not a partial city: a commercial-density map missing
the downtown is not a map of that city, however good the other districts are.

The consequence for ordering: where two countries are otherwise close,
prefer the national-register one. On this axis Korea outranks Spain -
Spain's six metro cities are six separate integrations, Korea's six are one
download plus six boundary files - even though Spain's per-city data is
excellent.

And check this BEFORE the station-density ranking, because it changes what
you are ranking: a country's entry is a set of cities, not one.

Rank on station density, not on population or city count

The number that decides whether a city is worth building is businesses within
the outermost ring per in-city station - the measure that made D.C. strong
(~173) despite keeping only 40 of 98 stations, and Boston weak (~39).

Canada's result as built (docs/canada_retrospective.md, Part Two; the
screen's figures - Vancouver 861 to Toronto 41 - were wrong station counts and
denominators, moved by up to 1.8x in the builds), and why it matters that
population was not the guide:

City	Storefronts per in-city station
Vancouver (with Surrey)	206
Montréal	151
Calgary	137
Surrey	135
Toronto	81
Edmonton	79

The largest city came fifth of six, because it licenses food and trades but
not general retail - two buckets, Boston's shape. Anyone choosing on population
would have built the worst of the six first.

Failure modes seen in one country profile

All of these produced a confidently wrong answer that survived until something
was measured. Expect them.

• A catalogue mirror three months stale and missing an entire mode.
  Toronto's TTC feed on the Mobility Database had no subway at all; the screen
  concluded Toronto mis-codes its subway. screen_rail.py now flags expiry.

  But do NOT generalise that into "never use a mirror" - the 2026-09-21
  screen did, and contradicted itself within the hour. Mexico City is the
  opposite case: datos.cdmx.gob.mx times out, the S3 direct_download
  returns 403, and the Mobility Database mirror is the only route that
  works - serving a current feed that screens as subway 12, the correct line
  count.

  The distinction is whether the artifact SELF-ATTESTS, not whether it came
  from a mirror. GTFS does: feed_info.txt carries feed_end_date, and mode
  counts are checkable against a known network, so "Toronto has a subway and
  this has zero type-1 routes" is detectable. A business-register CSV does not:
  no embedded validity date, its date living in a filename, and a short row
  count invisible - 28,000 against a true 31,000 looks fine.

  So the Toronto failure was not "a mirror was used". It was a stale mirror
  used without reading the expiry field already in the file, and the fix was
  the expiry check rather than abstinence.

  The rule: a mirror is usable when the artifact self-attests to its own
  freshness and completeness; it is not when you must take the mirror's word
  for both. And a feed has at least three addresses - the agency's,
  urls.latest and urls.direct_download - so try all of them before
  recording a city as unreachable.

• A boundary GeoJSON declaring the wrong CRS. Surrey's says EPSG:4326 and
  contains UTM metres; reprojecting silently puts it millions of metres away
  and every containment test returns zero.

• Two datasets with the same name, one returning null geometry. Calgary
  publishes two "City Boundary" layers; one is 184 bytes of valid, useless
  GeoJSON.

• A boundary layer that is a LINE. Vancouver's city-boundary is a
  MultiLineString; point-in-polygon matches nothing.

• A boundary layer that is mostly something else. Surrey's has 10 features,
  9 of them town centres.

• Case-sensitive join keys. Toronto's ADDRESS_FULL is title case and
  CKAN's filters matches exactly: uppercase matched 0 of 150.

• A dataset named in a vocabulary you did not search. Edmonton's boundary
  is a corporate boundary; Montréal's best source is locaux commerciaux.

• An opt-in dataset presented as a directory. Mississauga's lists "only the
  businesses that agreed to be included" - the Boston survey problem, which
  maps who filled in a form rather than where commerce is.

• Asking the wrong SOURCE, not just the wrong question. A transit
  catalogue answers "is there a GTFS feed", which is not what this project
  needs. Japan looked feed-poor and is data-rich. Check the national mapping
  agency before believing any transit verdict - see question 1.

• A screening tool reporting what it cannot parse as absent. Extended
  route types cost four European capitals; a nested zip cost Melbourne; a
  LineString where a Point was expected would have cost Japan, whose
  10,235 station records are platform centrelines needing centroids.

• Sampling the first record and calling it a measurement. It cost Finland
  a whole-country ruling. See the evidence-discipline section.

• Answering "are there addresses?" and skipping "what is this register made
  of?" Presence is the easy half. Composition decides.

The single recurring cause: asserting from a column's existence, a dataset
title, or a plausible-looking flag instead of measuring. Five separate
conclusions in the Canada profile had to be reversed for exactly that reason.
The project's first invariant already says this; a country profile is where it
is easiest to forget, because the volume is high and each individual check
feels small.