Skip to content

Data sources & provenance

Every upstream source we ingest, publisher, refresh cadence, licence, and what's loaded into the platform. Maintained alongside the nightly refresh orchestrator (see /data/coverage for live status).

Official Sources

UK Parliament Members API

https://members-api.parliament.uk/
Open Parliament Licence v3

Official source for MPs, Lords, constituency data, and biographical information.

Data Provided
MPsLordsConstituenciesTerms of serviceContact information
Update Frequency

Real-time via API

IPSA (Independent Parliamentary Standards Authority)

https://www.theipsa.org.uk/
Open Government Licence v3

Official source for MP expense claims, including staff costs, office costs, and travel.

Data Provided
MP expensesStaff salariesOffice costsTravel claimsAccommodation
Update Frequency

Published quarterly

Open Government Licence v3

Official source for political party finance data including donations, loans, and annual accounts.

Data Provided
Party donationsParty loansAnnual accountsElection spendingRegistered parties
Update Frequency

Weekly (donations), Annual (accounts)

UK Parliament Hansard & Votes

https://hansard.parliament.uk/
Open Parliament Licence v3

Official record of parliamentary proceedings, debates, and division (vote) records.

Data Provided
Division recordsMP voting historyDebate transcriptsWritten questions
Update Frequency

Real-time when Parliament sits

Open Parliament Licence v3

Official register of MPs' financial and other interests that could influence their work.

Data Provided
Employment incomeShareholdingsPropertyGifts & hospitalityOverseas visits
Update Frequency

Updated as changes reported

Open Government Licence v3

Official registry of UK companies used for donor verification and company information.

Data Provided
Company detailsDirectorsFiling historyAccounts
Update Frequency

Real-time via API

Office for National Statistics (ONS)

https://www.ons.gov.uk/
Open Government Licence v3

Official source for demographic, geographic, and Census data including population estimates and ethnicity.

Data Provided
Population estimatesCensus 2021 ethnicityConstituency boundariesGeographic codes
Update Frequency

Annual (population), Decennial (Census)

NOMIS (ONS Labour Market)

https://www.nomisweb.co.uk/
Open Government Licence v3

Official source for mid-year population estimates by age band for all local authorities.

Data Provided
Mid-year population (2015-2024)Age band breakdownsWorking-age population
Update Frequency

Annual

Open Government Licence v3

Department for Work and Pensions data on benefit claimants by local authority.

Data Provided
Universal Credit householdsHousing BenefitPIP recipientsPension Credit
Update Frequency

Monthly

Open Government Licence v3

Official immigration statistics including asylum dispersal and unaccompanied children placements.

Data Provided
Asylum dispersal placementsUASC placementsQuarterly LA-level data
Update Frequency

Quarterly

Open Government Licence v3

Criminal court statistics including Crown Court backlog, conviction rates, and sentencing data.

Data Provided
Crown Court receipts/disposalsConviction rateCustody rateSentencing data
Update Frequency

Quarterly

Contracts Finder (Cabinet Office)

https://www.contractsfinder.service.gov.uk/
Open Government Licence v3

Official register of government contracts published in OCDS format.

Data Provided
127,000+ contractsBuyer/supplier detailsContract valuesTender status
Update Frequency

Daily

Open Government Licence v3

Official house price data by local authority including average prices, annual changes, and sales volumes.

Data Provided
Average house priceAnnual price changeSales volume
Update Frequency

Monthly

Open Government Licence v3

Education attainment data and per-pupil spending by local authority.

Data Provided
Attainment 8 scoresProgress 8EBacc entryPer-pupil spending
Update Frequency

Annual

Police UK / data.police.uk

https://data.police.uk/
Open Government Licence v3

Street-level crime data for England, Wales, and Northern Ireland.

Data Provided
Street-level crimeCrime categoriesPolice forcesStop and search
Update Frequency

Monthly

Focaldata / Election Maps UK

https://electionmaps.uk/
Research use

Constituency-level MRP polling estimates and election projections.

Data Provided
Party vote share estimatesSeat projectionsConfidence ratingsChange since 2024
Update Frequency

Varies (MRP surveys)

CC-BY-SA

Open data on UK elections, candidates, and elected representatives.

Data Provided
Councillors (29,000+)Local election resultsCandidate data
Update Frequency

As elections occur

Methodology

Data Collection

We collect data from official sources using their published APIs where available. For sources without APIs, we use official bulk downloads published under open licences.

Entity Matching

MPs and other entities are matched across data sources using Parliament IDs as the primary key. Where Parliament IDs are unavailable, we use fuzzy name matching with manual verification.

Data Cleaning

Raw data is validated, normalised, and deduplicated. Currency values are stored in pence and converted for display. Dates are stored in ISO 8601 format.

Conflict Analysis

Conflict of interest scores are calculated algorithmically based on income sources, voting patterns, and sector alignments. The methodology is transparent and documented.

Update Schedule

Pipelines run automatically to fetch new data at intervals matching source update frequencies. All data includes timestamps showing when it was last verified.

Benchmarking

Percentile calculation

Percentile ranks show where a place stands relative to others of the same type (e.g., LAD vs LAD):

Percentile = (count_below + 0.5 × count_equal) / total × 100

Ties are handled by assigning the midpoint of the range to tied values.

Coverage warnings

  • Low (<60%): Results may not be representative.
  • Moderate (60-80%): Reasonably reliable but some places are missing.
  • Adequate (80%+): Reliable for comparison.

Metric Categories

Demographic

Population, age structure, density

Fiscal

Expenditure, council tax, business rates

Social

Deprivation indices, wellbeing indicators

Economic

Employment, income, business activity

Health

Life expectancy, health outcomes

Safety

Crime rates, safety indicators

Limitations

  • Time lag: Official statistics are published with a delay. Most recent data may be 1-2 years old.
  • England focus: Some metrics (e.g., IMD) are only available for England. Devolved equivalents use different methodologies.
  • Boundary changes: Local authority boundaries change over time. We use the most recent GSS codes but historical comparisons may be affected.
  • Causation: Correlations between metrics do not imply causation.

Our Principles

  • Political Neutrality: We do not advocate for any political position. All framing is neutral and factual.
  • Explainable Logic: No black-box scores. Every calculation is documented. You can reproduce our results.
  • Data Provenance: Every metric has a source, time period, and methodology note.
  • No Personal Data: We do not collect or store personal information.

Attribution Requirements

When using data from Civic Core Institute, please include attribution to both us and the original sources. Example attribution:

"Data from Civic Core Institute (civiccore.org), sourced from UK Parliament, Electoral Commission, and IPSA under the Open Government Licence v3."