Method

Last Reviewed: September 26, 2026

Maintainer: Duncan Burns

Purpose: Reference for hand-verified data points used across dashboards

Immigration

National — Border Encounters (CBP)

• FY2025 pace: Lowest since 1970
• FY2021-24 total: ~10.8M nationwide encounters
• Source URL: https://www.cbp.gov/newsroom/stats/southwest-land-border-encounters

⚠️ Methodology break at FY2025 — do not compare across it.

FY2017–FY2024 are total SWB encounters (Border Patrol + Office of Field Operations

ports of entry), from OHSS/KHSM. CBP stopped publishing the OFO port-of-entry component

for FY2025+, so FY2025 and FY2026 above are Border Patrol only and are therefore not

like-for-like with the years above them. The immigration dashboard's total series

deliberately ends at FY2024 for this reason.

Live figures: data/cbp-encounters-latest.json (live refreshed monthly on the 25th).

National — Net International Migration (Census)

• Census methodology revision (Dec 2024): 69–102% undercount in prior estimates
• 2022-24 revised total: +6.8M (revised upward)
• Source URL: https://www.census.gov/programs-surveys/popest.html

National — Deportation Math

National — Fiscal Impact

• Key finding: Legal immigrants net positive at all levels; undocumented net negative at state/local
• Cato caveat: Paper does NOT separate legal from illegal; counts US-born children as "US-born"

Massachusetts — Immigration Costs

Massachusetts — Shelter Crisis

Massachusetts — H-1B Workers

Massachusetts — Concentration Cities

live Live values: data/census-latest.json (ACS 5-year, vintage 2024, refreshed monthly on the 20th). The figures below are that file's current contents; the Feb 2026 column showed an older ACS vintage.

• 60-mile radius of Beacon Hill — inner-ring cities bearing disproportionate burden
• Lawrence multilingual population: 78.1%
• Chelsea multilingual population: 70.6%
• Everett: 63.4% · Revere: 56.3% · Lynn: 53.5% · Malden: 50.9%
• MA median household income: $103,960 (ACS 2024 5-yr)

Massachusetts — Population & Migration

Housing & Real Estate

Massachusetts — Median Prices (Warren Group)

• Source URL: https://www.thewarrengroup.com/

Massachusetts — Average Prices (MLS PIN, 5-Year)

live Source of truth: data/mls-history.json (closed sales per calendar year), regenerated by update-mls-figures.py. The dashboards read that cache directly — do not hand-edit a price here or there. Values below are the cache as of its 2026-07-17 generation.

• 5-Year Growth (2021→2025): MA +28.8%, Newburyport +40.8%, Essex +25.6%, Boston +19.6%
• Note: These are AVERAGES (skewed by luxury); Warren Group MEDIANS used for affordability analysis

Live trailing-12-month window (changes daily — do not transcribe)

The dashboards' last column is not calendar 2026 — it is a rolling trailing-12-month window rewritten every day. As of September 29, 2026: MA $865,253 · Essex $905,187 · Boston $1,351,395. Quote it with its as-of date or not at all.

Massachusetts — Market Indicators

Earlier revisions listed the trailing-12-month figures under a "2025" heading —

most recently "Units Sold 39,323 verified Feb 2026", which is the live window, not the

calendar year, and not a February figure. They are separated above.

Haverhill Specific

New Hampshire — Market Data (PrimeMLS)

live Source of truth: data/nh-figures.json, refreshed daily by update-nh-figures.py. Values below are that file as of 2026-09-29. (Feed is PrimeMLS, not Paragon.)

• Active inventory: 3,088 listings · median list $610,000 · avg list $852,851
• Also tracked per-market: Portsmouth, Salem, Derry, Windham

Boston — Commercial Real Estate (CMBS)

**Vacancy definitions differ by brokerage — these are not competing estimates of one

number.** C&W 19.0%, CBRE 18.7% and Colliers 23.7% all describe Greater Boston Q2 2026

on different footprints (submarket set, building class, sublease treatment). Quote the

brokerage with the figure, and don't mix them in a single time series.

Employment & Labor

Massachusetts — Current Labor Market

live Source of truth: data/employment-latest.json, refreshed monthly from the BLS API. Values below are that file as of 2026-09-25 (reference month August 2026).

⚠️ BLS discontinued monthly state-level JOLTS in 2026. National JOLTS is still monthly;

state figures are annual now, first annual release July 2026. Any "MA job openings rate"

quoted monthly is a pre-2026 series and cannot be extended.

Massachusetts — JOLTS (historical, pre-discontinuation)

National — Labor Market (2025)

• Key finding: "Hiring recession" — low-hire/low-fire environment, worst since 2009 by multiple measures
• Healthcare masks weakness: 713K healthcare jobs > total net gains, meaning all other sectors flat/negative

National — Employment Situation Release Dates

Machine-readable copy lives in data/bls-release-schedule.json; the employment updater reads it to stamp "Released <date>" on the national block.

• Not a first-Friday rule. April 2026 went out on the *second* Friday (May 8) and June 2026 on a *Thursday* (Jul 2, moved for July 4). These dates cannot be computed — they must be read from the published schedule.
• Source URL: https://www.bls.gov/schedule/news_release/empsit.htm

Massachusetts — State Employment & Unemployment Release Dates

The MA vintage. Lands ~3 weeks after the national release, which is why the MA blocks on the employment dashboard legitimately sit a month behind the national block. Same JSON file, state_employment key.

• Even less regular than the national schedule. The January 2026 reference month did not appear until April 8, 2026.
• Source URL: https://www.bls.gov/schedule/news_release/laus.htm
• Maintenance: BLS publishes ~13 months ahead. When the newest reference month is missing, the updater logs a ::warning:: in the Actions run and renders an em dash instead of inventing a date — that warning is the cue to top up the table above and the JSON.
• Note: bls.gov HTML 403s non-browser clients, so this cannot be scraped from CI (the same block that stalled the CBP feed). api.bls.gov serves the figures but carries no release dates.

Education

Statewide Enrollment Shifts (2019–2025)

Gateway Cities vs. Wealthy Towns (DESE FY2024 / MCAS 2025)

• Key finding: Wealthy towns spend only 14% more but achieve 3.2x higher proficiency
• Sources: DESE PerPupilExpenditures.xlsx (FY2024), NextGenMCAS.xlsx (Spring 2025), TeacherSalaries.xlsx (2020-21)

Zoning — MBTA Communities Act § 3A

Feed status: live. update-mbta-3a.py reads three EOHLC tables weekly and rewrites mbta-3a-dashboard.html plus data/mbta-3a-latest.json. Do not hand-edit the figures below or on the page — change the source or the updater.

What the law requires

Category shares of 2020 stock: rapid transit 25%, commuter rail 15%, adjacent community 10%, adjacent small town 5%. Counts: 12 / 72 / 58 / 35.

• Capacity is not construction. The required figure is zoned capacity under

EOHLC's compliance model. EOHLC states the model gives "reasonable estimates" and does "not provide absolute measures of development potential."

• For 44 of 177 communities the percentage is not what binds. Minimum land

area is 50 acres, and at 15 units/acre that is a floor of about 750 units regardless of town size, which lands above the category share in a small town. Georgetown is nominally a 10% community and is required to zone for 24%.

Compliance and what has been built

• Tracked is not built. The tracker mixes completed, permitted and proposed

developments, and EOHLC says it is compiled from third-party information and may not be comprehensive.

• Tracked units are 3.4% of required capacity. Deed-restricted units are 18.1%

of tracked units — and they come from local inclusionary zoning, 40B and deal-level subsidy, not from § 3A, which requires no affordability.

Sources and transport

• Page: <https://www.mass.gov/info-details/multi-family-zoning-requirement-for-mbta-communities>
• Three CSVs are linked from that page and the updater **scrapes the links each

run**. Two filenames carry the edition date ("as of 8-31-26"), so a hardcoded URL would 404 silently the moment EOHLC published the next edition.

• mass.gov returns 403 to a bare request. The filter is header-based, not the

client fingerprinting that defeats cbp.gov — curl with a full browser header set (including the Sec-Fetch-* trio) gets 200, verified from a GitHub-hosted runner. If that ever changes the updater exits 75 and the page keeps its edition.

The $4.49B cost figure — attributed, and it does not reconcile

The Brensley Cost Impact Report (Dec 2025, prepared for the State Auditor) is quoted widely at $4.49B in designer base fees. The dashboard reproduces it as her attributed estimate with the arithmetic shown, because the published method does not produce it:

`` 273,080 units x 900 sf x $250/sf = $61.44B construction cost $61.44B x 5.9% DCAMM designer fee = $3.63B <- not $4.49B reaching $4.49B from that base needs a rate of about 7.3% ``

It also uses 273,080 units, which is no longer EOHLC's published total (297,190). Neither point makes the estimate worthless; both mean it is an order of magnitude from a stated method, not a precise liability. Do not restate $4.49B without the caveat, and do not recompute it against 297,190 and present the result as Brensley's — that would be this site's arithmetic wearing her attribution.

The campaign-finance section is fixed, not live

"Follow the Money" on that page is a carried-over investigation current as of February 2026 and is labelled as such on the page. It aggregates roughly 107,900 OCPF contribution records for the four officials who authored, implemented or enforce § 3A (Sen. Brendan Crighton, Gov. Maura Healey, AG Andrea Campbell, Rep. Kevin Honan), plus Secretary of the Commonwealth lobbyist registrations and IRS Form 990 filings via ProPublica. It has not been re-run against OCPF since. Donation patterns establish association, not agreement.

Healthcare & Insurance

2026 MA Rate Increases (Division of Insurance)

• Average increase: 9.9% (3x inflation rate of ~3%)
• State benchmark: 3.6%
• Consumers affected: 711,563
• Saved via negotiations: $54M

Premium Comparisons

• MA premium cost growth (2021-23): +12.1%
• MA wage growth (2021-23): +9.7%
• Sources: KFF Employer Health Benefits Survey, CHIA Annual Reports

Cost of Living

SNAP vs. Poverty

SNAP Fraud Detection

Tax Burden — Federal & Massachusetts

Machine-readable source of truth: data/burden-constants.json. tax-burden-dashboard.html is generated from it by update-burden-constants.py -- that script's TARGETS list holds that page and no other. Change the JSON, not the HTML. tax-burden-nh-comparison.html is NOT generated: it is maintained by hand and carries its own data-checked stamp, which the freshness watchdog ages.

The two MA median-income figures — do not merge them

CPS and ACS disagree by ~$9K on the same concept. Carry both and cite which one any given chart uses — this is exactly the kind of thing a hostile reader finds first.

~~Known defect to fix~~ — FIXED 2026-08-11. tax-budget-dashboard.html labelled $113,900 as "Census 2024 ACS"; it is the CPS figure and is now labelled "Census CPS ASEC (2024)" in both places, with the $104,828 ACS comparator named inline. The same file carried an unlabelled "$7,732/year" property tax figure; the hero stat, the prose and the stat card now read $8,113 (FY2026). The FY2025 county table keeps its own FY2025 source line — it was already labelled correctly and its county figures are FY2025.

The chartIncomeReal cross-state bar series still uses 113,900 for MA. Left alone deliberately: the other seven states' vintage is undocumented, and swapping MA to ACS without knowing theirs would trade a labelling error for a comparability error. Open item: establish the vintage of the other states, then make all eight consistent.

Never use a median as an allocation divisor. Per-household shares must sum back to the statewide pool, which only works with the mean. The reconciliation panel on the burden dashboard checks this.

State Budget — the scope reconciliation

Read this before quoting any Massachusetts budget total. Three different numbers

circulate for the same fiscal year. They are all correct; they are different scopes.

Quoting one without naming its scope is how the same year ends up with three answers.

The official GAA workbooks split every account into four types. The headline figure the press release and the media quote is the first three, and excludes Intragovernmental Service Spending.

$B. Source: official mass.gov GAA export gaabudgets.xlsx. verified Verified Aug 2026.

Proof the definition is right, not inferred: 60.292 + 2.288 + 0.836 = $63.416B, matching summary-fy-27-enacted-budget-summary.csv's Total row of $63,416,035,478 to the dollar, and the press release's "$63.42 billion".

FY2027 enacted — key figures

Enacted ≠ spent — the supplemental gap

Like-for-like (identical 913-account universe, headline scope, inter-fund transfer lines excluded from both sides):

Plotted as h5Outturn on tax-budget-dashboard.html.

Do not compute this by summing gaaspend.xlsx raw. That file contains inter-fund

transfer accounts — 1595-0029 (GF → Education & Transportation Fund) alone is $5.3B

in FY25 — which were never appropriated and appear on only one side. Summing raw

overstates the FY25 "overrun" as +25.7% instead of the true +10.0%. Exclude accounts whose

description begins *Transfer* or *XFR*, and restrict to accounts present in both workbooks.

Local-aid CSV ingestion gotcha

summary-fy-27-local-aid-aid-to-municipalities.csv carries a Total Municipal Aid footer row inside the data body, plus a blank row before it. A naive read_csv().sum() double-counts it and reports Chapter 70 at $13.39B instead of $6.70B — a 100% overstatement. Filter out any row whose municipality name contains "Total" before aggregating. The per-town figures themselves are correct.

Note also that the municipal CSV omits regional school districts (Triton, Pentucket, Whittier and the rest). Regional Ch70 flows to the *district*, not the member towns, which is why Newbury, Rowley, West Newbury and Salisbury show only $13K–$19K of Chapter 70 and are effectively UGGA-only. Use sect3.xlsx when districts matter.

Tax Burden — New Hampshire

Why pension debt is excluded from the burden calculators

The burden dashboards count what a household pays. An unfunded pension liability isn't billed to anyone — it's a claim on future tax capacity, and the portion collected today arrives through the appropriation funded by the income and property taxes Layer 1 already counts. Counting it again in Layer 5 double-counts.

It also cannot be sourced symmetrically across the border, in two opposite directions:

Use $42.1B MA vs $5.58B NH for any pension-to-pension read, and carry the caveat. The stocks are disclosed beside Layer 5 on both dashboards; the full story lives in pension-dashboard.html.

Removing it improved the analysis. Layer 5 is now entirely federal and therefore genuinely identical on both sides of the border, and the compression finding now holds at $75k — the old asymmetric placeholders (MA $2,000 vs NH $1,600) had been manufacturing a $400 NH advantage out of pure guesswork.

The Layer 4 inclusion rule

A charge is a policy cost if it exists because of a legislative or regulatory

mandate, rather than because of the physical cost of generating and delivering energy.

Excluded on both sides as infrastructure or mechanism: distribution, transmission, revenue decoupling, regulatory reconciliation, stranded cost, customer charge.

Electricity is genuinely like-for-like — both figures are Eversource, the same company on both sides of the border. Gas is not: Eversource doesn't serve NH gas, so that side is Liberty EnergyNorth.

Both figures are conservative in the same direction. MA's is a floor — its own tariff footnote places the Long Term Renewable Contract Adjustment (83C/83D offshore wind), Solar Program Cost Adjustment and Grid Modernization *inside* the Distribution charge, unbroken out, and RPS/CES sits inside supply. NH's gas figure is a ceiling — the LDAC bundles mechanisms MA excludes. So the true gap is wider than stated.

Headline finding: Massachusetts gas policy charges (59.08¢/therm) cost more than the gas itself (38.42¢/therm). The prior 18¢ placeholder understated it by over 3×.

Worth watching separately: NH electricity rose 13.8% year over year while MA fell 3.6%. Whatever advantage NH holds on the meter is closing fast.

NH median property tax bill — sources disagree, carry all three

The dashboard uses 1.71%, derived from the ~$17.10 per $1,000 statewide average total rate, which is the cleanest of the four. **Note this is a rate on *assessed* value** — NH DRA publishes equalization ratios precisely because assessed and market values diverge, so applying it to a market price overstates the bill.

The constants-consistency rule

households × meanHouseholdIncome ÷ incomeShare must return the same national aggregate for every state. Before this was enforced, MA implied a $14.91T national total and NH implied $17.50T — a 17.3% disagreement that made the "federal borrowing is identical across the border" claim false by $2,674 on a line captioned *"Identical basis"*. update-burden-constants.py now fails the build if this breaks.

Energy

Figures on energy-dashboard.html. Refreshed by update-energy-dashboard.py and update-fuel-prices.py on the 2nd and 26th, so the rate series carries the feed mark; the legislative analysis beside it does not and is dated to the document it reads.

Pensions

Figures on pension-dashboard.html. Maintained by hand against the actuarial reports and the Commonwealth’s own accounts; there is no feed behind it.

Headline figures

The rows above document the charts. These are the summary cards, which the page displays without a source line of their own.

Attribution here is to the publisher, not to a file. The specific documents these were read from are not retained, and the page’s own Source: lines belong to its charts rather than to these cards — borrowing one by proximity credited Massachusetts SERS headcounts to a national average, so it was not used. Naming the body that publishes each series is what can be stood behind: the FY2024 ACFR, the PERAC valuation and the PRIT report are all reachable from the links below, and a precise citation that might be wrong is worse here than an honest general one.

• Commonwealth ACFR — <https://www.macomptroller.org/annual-comprehensive-financial-reports/>
• PERAC — <https://www.mass.gov/orgs/public-employee-retirement-administration-commission>
• PRIM / PRIT Fund — <https://www.mapension.com/>
• State Retiree Benefits Trust Fund — <https://www.mass.gov/orgs/state-retiree-benefits-trust-fund>

Lobbying & Political Spending

Figures on pay-to-play-dashboard.html. Three authorities: OpenSecrets for sector and multi-year firm totals, the Secretary of the Commonwealth’s Lobbyist Public Search for single-year firm filings, and the state’s own OCPF database for individual donations.

The SOS figures are the only ones here backed by a file in data/. They cannot be fetched by a workflow — the SOS blocks server-side requests — so they are collected from a browser session with The-Peoples-Audit/userscripts/sos-lobbyist-detail-scraper.user.js and land in data/ma-lobbying-firms-latest.json. scripts/check-consistency.py holds the dashboard to that file, and scripts/check-freshness.py ages it at 400 days. Everything else in this section is maintained by hand.

Registration year 2026 — first-half filings. MA lobbying disclosure is semi-annual (due Jul 15 and Jan 15), so these cover roughly January to June and are not comparable to the full-year totals for 2016–2025. Collected 2026-09-27.

Both flows reconcile to the cent against the other side of the ledger: the client→firm total equals what the 169 firms reported receiving, and the client→lobbyist total equals what the 1,528 individual lobbyists reported receiving. 1,733 clients filed. For reference, registration year 2025 — a full year — had 175 entities receiving $88,409,949.

Joining these files

Within a registration year, join on sysvalue. Names drift even inside one year — the SOS renormalised “Smith, Costello & Crawford” (2025) to “Smith Costello & Crawford” (2026), and some client names carry double spaces in the industry results but not on their own pages.

Across years, sysvalue does not work at all. It is year-scoped: every 2025 key begins hvS0w46oEfUey9UWwnkaG… and every 2026 key j980aMf4IpOwXPZWPGG1x…. The overlap between the two years is zero — for clients and for firms. An earlier version of this file said “join on sysvalue, not on name” without that qualification, which would lead anyone comparing years to conclude that all 1,697 clients left and 1,733 different ones arrived.

Year-over-year therefore has to match on normalised names (lowercase, punctuation and corporate suffixes stripped, whitespace collapsed), which resolves 1,454 of 1,733 clients. The remainder are a mix of genuine entries and exits and of names that moved too far to match, so treat churn counts as approximate and spend comparisons on matched names as sound.

Industry labels are the state’s, not ours. The SOS’s Industry Type register assigns every client one of 36 labels at registration; 35 are in use (“Other” is empty). Joined on sysvalue it covers 1,733 of 1,733 clients. A name-keyword classifier managed 46.4%, which is why nothing inferred is published here.

The single “Healthcare” line understates health, because the SOS splits it four ways:

That 25.2% independently matches OpenSecrets’ 25% for the health sector over 2015–2025 — a different source, method and window — and it holds across a second full scrape:

The totals are not comparable — one is a year, the other half of one. The shares are.

A scrape is a snapshot, not a final figure

The 2025 firm file, scraped 2026-05-28, says firms received $88,409,948.93. The client file scraped 2026-09-27 says clients paid them $88,602,703.93 — $192,755.00 more, 0.218%. It decomposes exactly:

Four of the five amendments are exactly double the earlier figure — first-half filings that later gained their second half. The series above uses the amended $88.6M for 2025 and keeps the scrape-time figure alongside it as received_at_scrape.

This matters for the 2026 numbers too. They reconcile to the cent because both sides were scraped the same day. That is a same-day property, not a permanent one; expect 2026 to drift upward as amendments land, exactly as 2025 did.

Boston

Figures on all-things-boston.html. The page states its sources in prose rather than under each chart, so the authorities are listed here and the page itself carries the detail. Maintained by hand.

Dashboard Registry

All dashboards live at https://massachusettsdatahub.com/<file>. Cadences below are the cron lines in .github/workflows, and the refresh column names the script that actually writes each page.

Consumer-price figures refresh on the 10th&ndash;16th (update-cpi.yml) and feed the affordability and energy pages rather than a page of their own. The site itself redeploys every three hours, and after every one of these runs.

Old repos (superseded — these still serve older copies)

• https://duncanburns2013-dot.github.io/Immigration/
• https://duncanburns2013-dot.github.io/Housing-Market-Data/housing-market-dashboard.html
• https://duncanburns2013-dot.github.io/Housing-Market-Data/nh-housing-market-dashboard.html
• https://duncanburns2013-dot.github.io/Housing-Market-Data/haverhill-market-report.html
• https://duncanburns2013-dot.github.io/Master-Massachusetts-Affordability-Employment-/dashboard.html
• https://duncanburns2013-dot.github.io/Merrimac-Valley-Education-/
• https://duncanburns2013-dot.github.io/Merrimac-Valley-Education-/boston-dashboard.html