Data Sources
Provenance, methodology, and update schedules for all datasets.
Official Sources
UK Parliament Members API
https://members-api.parliament.uk/Official source for MPs, Lords, constituency data, and biographical information.
Real-time via API
IPSA (Independent Parliamentary Standards Authority)
https://www.theipsa.org.uk/Official source for MP expense claims, including staff costs, office costs, and travel.
Published quarterly
Electoral Commission
https://www.electoralcommission.org.uk/Official source for political party finance data including donations, loans, and annual accounts.
Weekly (donations), Annual (accounts)
UK Parliament Hansard & Votes
https://hansard.parliament.uk/Official record of parliamentary proceedings, debates, and division (vote) records.
Real-time when Parliament sits
Register of Members' Interests
https://publications.parliament.uk/pa/cm/cmregmem.htmOfficial register of MPs' financial and other interests that could influence their work.
Updated as changes reported
Companies House
https://www.gov.uk/government/organisations/companies-houseOfficial registry of UK companies used for donor verification and company information.
Real-time via API
Office for National Statistics (ONS)
https://www.ons.gov.uk/Official source for demographic, geographic, and Census data including population estimates and ethnicity.
Annual (population), Decennial (Census)
NOMIS (ONS Labour Market)
https://www.nomisweb.co.uk/Official source for mid-year population estimates by age band for all local authorities.
Annual
DWP Stat-Xplore
https://stat-xplore.dwp.gov.uk/Department for Work and Pensions data on benefit claimants by local authority.
Monthly
Home Office Immigration Statistics
https://www.gov.uk/government/collections/immigration-statistics-quarterly-releaseOfficial immigration statistics including asylum dispersal and unaccompanied children placements.
Quarterly
Ministry of Justice
https://www.gov.uk/government/organisations/ministry-of-justiceCriminal court statistics including Crown Court backlog, conviction rates, and sentencing data.
Quarterly
Contracts Finder (Cabinet Office)
https://www.contractsfinder.service.gov.uk/Official register of government contracts published in OCDS format.
Daily
UK House Price Index (Land Registry)
https://www.gov.uk/government/collections/uk-house-price-index-reportsOfficial house price data by local authority including average prices, annual changes, and sales volumes.
Monthly
Department for Education (DfE)
https://explore-education-statistics.service.gov.uk/Education attainment data and per-pupil spending by local authority.
Annual
Police UK / data.police.uk
https://data.police.uk/Street-level crime data for England, Wales, and Northern Ireland.
Monthly
Focaldata / Election Maps UK
https://electionmaps.uk/Constituency-level MRP polling estimates and election projections.
Varies (MRP surveys)
Democracy Club
https://democracyclub.org.uk/Open data on UK elections, candidates, and elected representatives.
As elections occur
Methodology
Data Collection
We collect data from official sources using their published APIs where available. For sources without APIs, we use official bulk downloads published under open licences.
Entity Matching
MPs and other entities are matched across data sources using Parliament IDs as the primary key. Where Parliament IDs are unavailable, we use fuzzy name matching with manual verification.
Data Cleaning
Raw data is validated, normalised, and deduplicated. Currency values are stored in pence and converted for display. Dates are stored in ISO 8601 format.
Conflict Analysis
Conflict of interest scores are calculated algorithmically based on income sources, voting patterns, and sector alignments. The methodology is transparent and documented.
Update Schedule
Pipelines run automatically to fetch new data at intervals matching source update frequencies. All data includes timestamps showing when it was last verified.
Benchmarking
Percentile Calculation
Percentile ranks show where a place stands relative to others of the same type (e.g., LAD vs LAD):
Percentile = (count_below + 0.5 × count_equal) / total × 100Ties are handled by assigning the midpoint of the range to tied values.
Coverage Warnings
- Low (<60%): Results may not be representative.
- Moderate (60-80%): Reasonably reliable but some places are missing.
- Adequate (80%+): Reliable for comparison.
Metric Categories
Demographic
Population, age structure, density
Fiscal
Expenditure, council tax, business rates
Social
Deprivation indices, wellbeing indicators
Economic
Employment, income, business activity
Health
Life expectancy, health outcomes
Safety
Crime rates, safety indicators
Limitations
- Time lag: Official statistics are published with a delay. Most recent data may be 1-2 years old.
- England focus: Some metrics (e.g., IMD) are only available for England. Devolved equivalents use different methodologies.
- Boundary changes: Local authority boundaries change over time. We use the most recent GSS codes but historical comparisons may be affected.
- Causation: Correlations between metrics do not imply causation.
Our Principles
- Political Neutrality: We do not advocate for any political position. All framing is neutral and factual.
- Explainable Logic: No black-box scores. Every calculation is documented. You can reproduce our results.
- Data Provenance: Every metric has a source, time period, and methodology note.
- No Personal Data: We do not collect or store personal information.
Attribution Requirements
When using data from Civic Core Institute, please include attribution to both us and the original sources. Example attribution:
"Data from Civic Core Institute (civiccore.org), sourced from UK Parliament, Electoral Commission, and IPSA under the Open Government Licence v3."