Full workflow for importing WorldPop population data into DHIS2

In this notebook we will show a complete workflow showing how to import yearly population count data from WorldPop v2 (2015-2030) into DHIS2. Because the WorldPop population data is static, this is a one-time import and does not need to run continuously.

Library imports¶

Start by importing the libraries that we need:

from datetime import date
import json

import geopandas as gpd
import xarray as xr
from earthkit import transforms

from dhis2_client import DHIS2Client
from dhis2_client.settings import ClientSettings

from dhis2eo.data.worldpop import pop_total
from dhis2eo.integrations.pandas import dataframe_to_dhis2_json

Input parameters¶

Let’s first define all the input parameters so they are clearly stated at the top of the notebook.

For this example we will connect to a public DHIS2 instance, and set the DHIS2_DATA_ELEMENT_ID below to import directly into the existing “Total Population” Data Element.

Note that we recommend setting DHIS2_ORG_UNIT_LEVEL to the highest available level to get the most detailed possible population data; DHIS2 will take care to aggregate up to lower levels when needed.

Unlike other import workflows, the WorldPop data cannot be accessed by organisation unit bounding box, but rather requires that we specify the country we are requesting data for, which we specify with the IMPORT_COUNTRY_CODE parameter.

# DHIS2 connection
DHIS2_BASE_URL = "https://play.im.dhis2.org/stable-2-42-3-1"
DHIS2_USERNAME = "admin"
DHIS2_PASSWORD = "district"

# DHIS2 import settings
DHIS2_DATA_ELEMENT_ID = 'WUg3MYWQ7pt'
DHIS2_ORG_UNIT_LEVEL = 2
DHIS2_DRY_RUN = True                            # default to safe dry-run mode; set to False for actual import

# Population import configuration
IMPORT_VALUE_COL = "pop_total"                  # variable name in the downloaded xarray dataset
IMPORT_START_DATE = "2015"                      # first year of the dataset
IMPORT_END_DATE = "2030"                        # last year of the dataset
IMPORT_COUNTRY_CODE = "SLE"                     # 3-letter ISO country code

# Download settings
DOWNLOAD_FOLDER = "../../guides/data/local"
DOWNLOAD_PREFIX = "worldpop-pop"    # prefix for caching downloads; existing files are reused

# Aggregation settings
SPATIAL_AGGREGATION = "sum"

Connect to DHIS2¶

First, we connect the python-client to the DHIS2 instance we want to import into. You can point this to your own instance, but for the purposes of this example we will use one of the public access DHIS2 instances, since these are continuously reset:

# Client configuration
cfg = ClientSettings(
  base_url=DHIS2_BASE_URL,
  username=DHIS2_USERNAME,
  password=DHIS2_PASSWORD
)

client = DHIS2Client(settings=cfg)
info = client.get_system_info()

# Check if everything is working.
# You should see your current DHIS2 version info.
print("Current DHIS2 version:", info["version"])

Current DHIS2 version: 2.42.3.1

Get the DHIS2 organisation units¶

In order to download and aggregate the data to our DHIS2 organisation units, we also use the python-client to get the requested organisation units from our DHIS2 instance:

### Get org units GeoJSON from DHIS2
org_units_geojson = client.get_org_units_geojson(level=DHIS2_ORG_UNIT_LEVEL)

# Convert GeoJSON to geopandas
org_units = gpd.read_file(json.dumps(org_units_geojson))
org_units

Skipping field groups: unsupported OGR type: 5

Download the necessary data¶

In the next step we download all the requested data to the local file system, using convenience functionality from the dhis2eo.data.worldpop.pop_total module.

Note that after the initial data download, subsequent runs of this notebook will re-use the previously imported files to avoid repeated downloads of the same data.

For more details on this step, see our guide for Downloading WorldPop population data.

files = pop_total.yearly.download(
    start=IMPORT_START_DATE, 
    end=IMPORT_END_DATE, 
    country_code=IMPORT_COUNTRY_CODE, 
    dirname=DOWNLOAD_FOLDER, 
    prefix=DOWNLOAD_PREFIX, 
)
files

INFO - 2026-01-17 22:20:34,066 - dhis2eo.data.worldpop.pop_total.yearly - Year 2015
INFO - 2026-01-17 22:20:34,070 - dhis2eo.data.worldpop.pop_total.yearly - File already downloaded: C:\Users\karimba\Documents\Github\climate-tools\docs\guides\data\local\worldpop-pop_2015.nc
INFO - 2026-01-17 22:20:34,072 - dhis2eo.data.worldpop.pop_total.yearly - Year 2016
INFO - 2026-01-17 22:20:34,077 - dhis2eo.data.worldpop.pop_total.yearly - File already downloaded: C:\Users\karimba\Documents\Github\climate-tools\docs\guides\data\local\worldpop-pop_2016.nc
INFO - 2026-01-17 22:20:34,080 - dhis2eo.data.worldpop.pop_total.yearly - Year 2017
INFO - 2026-01-17 22:20:34,084 - dhis2eo.data.worldpop.pop_total.yearly - File already downloaded: C:\Users\karimba\Documents\Github\climate-tools\docs\guides\data\local\worldpop-pop_2017.nc
INFO - 2026-01-17 22:20:34,088 - dhis2eo.data.worldpop.pop_total.yearly - Year 2018
INFO - 2026-01-17 22:20:34,094 - dhis2eo.data.worldpop.pop_total.yearly - File already downloaded: C:\Users\karimba\Documents\Github\climate-tools\docs\guides\data\local\worldpop-pop_2018.nc
INFO - 2026-01-17 22:20:34,095 - dhis2eo.data.worldpop.pop_total.yearly - Year 2019
INFO - 2026-01-17 22:20:34,100 - dhis2eo.data.worldpop.pop_total.yearly - File already downloaded: C:\Users\karimba\Documents\Github\climate-tools\docs\guides\data\local\worldpop-pop_2019.nc
INFO - 2026-01-17 22:20:34,101 - dhis2eo.data.worldpop.pop_total.yearly - Year 2020
INFO - 2026-01-17 22:20:34,104 - dhis2eo.data.worldpop.pop_total.yearly - File already downloaded: C:\Users\karimba\Documents\Github\climate-tools\docs\guides\data\local\worldpop-pop_2020.nc
INFO - 2026-01-17 22:20:34,107 - dhis2eo.data.worldpop.pop_total.yearly - Year 2021
INFO - 2026-01-17 22:20:34,111 - dhis2eo.data.worldpop.pop_total.yearly - File already downloaded: C:\Users\karimba\Documents\Github\climate-tools\docs\guides\data\local\worldpop-pop_2021.nc
INFO - 2026-01-17 22:20:34,112 - dhis2eo.data.worldpop.pop_total.yearly - Year 2022
INFO - 2026-01-17 22:20:34,115 - dhis2eo.data.worldpop.pop_total.yearly - File already downloaded: C:\Users\karimba\Documents\Github\climate-tools\docs\guides\data\local\worldpop-pop_2022.nc
INFO - 2026-01-17 22:20:34,117 - dhis2eo.data.worldpop.pop_total.yearly - Year 2023
INFO - 2026-01-17 22:20:34,120 - dhis2eo.data.worldpop.pop_total.yearly - File already downloaded: C:\Users\karimba\Documents\Github\climate-tools\docs\guides\data\local\worldpop-pop_2023.nc
INFO - 2026-01-17 22:20:34,125 - dhis2eo.data.worldpop.pop_total.yearly - Year 2024
INFO - 2026-01-17 22:20:34,129 - dhis2eo.data.worldpop.pop_total.yearly - File already downloaded: C:\Users\karimba\Documents\Github\climate-tools\docs\guides\data\local\worldpop-pop_2024.nc
INFO - 2026-01-17 22:20:34,131 - dhis2eo.data.worldpop.pop_total.yearly - Year 2025
INFO - 2026-01-17 22:20:34,135 - dhis2eo.data.worldpop.pop_total.yearly - File already downloaded: C:\Users\karimba\Documents\Github\climate-tools\docs\guides\data\local\worldpop-pop_2025.nc
INFO - 2026-01-17 22:20:34,136 - dhis2eo.data.worldpop.pop_total.yearly - Year 2026
INFO - 2026-01-17 22:20:34,139 - dhis2eo.data.worldpop.pop_total.yearly - File already downloaded: C:\Users\karimba\Documents\Github\climate-tools\docs\guides\data\local\worldpop-pop_2026.nc
INFO - 2026-01-17 22:20:34,140 - dhis2eo.data.worldpop.pop_total.yearly - Year 2027
INFO - 2026-01-17 22:20:34,148 - dhis2eo.data.worldpop.pop_total.yearly - File already downloaded: C:\Users\karimba\Documents\Github\climate-tools\docs\guides\data\local\worldpop-pop_2027.nc
INFO - 2026-01-17 22:20:34,150 - dhis2eo.data.worldpop.pop_total.yearly - Year 2028
INFO - 2026-01-17 22:20:34,153 - dhis2eo.data.worldpop.pop_total.yearly - File already downloaded: C:\Users\karimba\Documents\Github\climate-tools\docs\guides\data\local\worldpop-pop_2028.nc
INFO - 2026-01-17 22:20:34,155 - dhis2eo.data.worldpop.pop_total.yearly - Year 2029
INFO - 2026-01-17 22:20:34,158 - dhis2eo.data.worldpop.pop_total.yearly - File already downloaded: C:\Users\karimba\Documents\Github\climate-tools\docs\guides\data\local\worldpop-pop_2029.nc
INFO - 2026-01-17 22:20:34,161 - dhis2eo.data.worldpop.pop_total.yearly - Year 2030
INFO - 2026-01-17 22:20:34,166 - dhis2eo.data.worldpop.pop_total.yearly - File already downloaded: C:\Users\karimba\Documents\Github\climate-tools\docs\guides\data\local\worldpop-pop_2030.nc

[WindowsPath('C:/Users/karimba/Documents/Github/climate-tools/docs/guides/data/local/worldpop-pop_2015.nc'),
 WindowsPath('C:/Users/karimba/Documents/Github/climate-tools/docs/guides/data/local/worldpop-pop_2016.nc'),
 WindowsPath('C:/Users/karimba/Documents/Github/climate-tools/docs/guides/data/local/worldpop-pop_2017.nc'),
 WindowsPath('C:/Users/karimba/Documents/Github/climate-tools/docs/guides/data/local/worldpop-pop_2018.nc'),
 WindowsPath('C:/Users/karimba/Documents/Github/climate-tools/docs/guides/data/local/worldpop-pop_2019.nc'),
 WindowsPath('C:/Users/karimba/Documents/Github/climate-tools/docs/guides/data/local/worldpop-pop_2020.nc'),
 WindowsPath('C:/Users/karimba/Documents/Github/climate-tools/docs/guides/data/local/worldpop-pop_2021.nc'),
 WindowsPath('C:/Users/karimba/Documents/Github/climate-tools/docs/guides/data/local/worldpop-pop_2022.nc'),
 WindowsPath('C:/Users/karimba/Documents/Github/climate-tools/docs/guides/data/local/worldpop-pop_2023.nc'),
 WindowsPath('C:/Users/karimba/Documents/Github/climate-tools/docs/guides/data/local/worldpop-pop_2024.nc'),
 WindowsPath('C:/Users/karimba/Documents/Github/climate-tools/docs/guides/data/local/worldpop-pop_2025.nc'),
 WindowsPath('C:/Users/karimba/Documents/Github/climate-tools/docs/guides/data/local/worldpop-pop_2026.nc'),
 WindowsPath('C:/Users/karimba/Documents/Github/climate-tools/docs/guides/data/local/worldpop-pop_2027.nc'),
 WindowsPath('C:/Users/karimba/Documents/Github/climate-tools/docs/guides/data/local/worldpop-pop_2028.nc'),
 WindowsPath('C:/Users/karimba/Documents/Github/climate-tools/docs/guides/data/local/worldpop-pop_2029.nc'),
 WindowsPath('C:/Users/karimba/Documents/Github/climate-tools/docs/guides/data/local/worldpop-pop_2030.nc')]

Open the downloaded data¶

Once the data has been downloaded, we can then pass the list of files to xr.open_mfdataset(). This allows us to open and work with the data as if it were a single xarray dataset:

ds_yearly = xr.open_mfdataset(files)
ds_yearly

C:\Users\karimba\AppData\Local\Temp\ipykernel_42904\435495002.py:1: FutureWarning: In a future version of xarray the default value for data_vars will change from data_vars='all' to data_vars=None. This is likely to lead to different results when multiple datasets have matching variables with overlapping values. To opt in to new defaults and get rid of these warnings now use `set_options(use_new_combine_kwarg_defaults=True) or set data_vars explicitly.
  ds_yearly = xr.open_mfdataset(files)

Aggregate to organisation units¶

Since the data is already at the correct yearly level, we can proceed directly to aggregate the gridded data to the organisation units from your DHIS2 instance:

print("Aggregating to organisation units...")
ds_org_units = transforms.spatial.reduce(
    ds_yearly[IMPORT_VALUE_COL],
    org_units,
    mask_dim="id",
    how=SPATIAL_AGGREGATION,
)
ds_org_units

Aggregating to organisation units...

Post-processing¶

After aggregating the population data to the desired organizational units, we convert the xarray Dataset to a Pandas DataFrame. This makes it easier to inspect the data and prepare it for subsequent post-processing:

dataframe = ds_org_units.to_dataframe().reset_index()
dataframe

The only thing we have to change before importing to DHIS2, is that we need the time column to only record the year so that DHIS2 assigns it the correct period type:

dataframe['time'] = dataframe["time"].dt.year.astype(str)

This should now display correctly:

dataframe

Create DHIS2 payload¶

At this point we have the final data that we want to import into DHIS2. In order to submit the data to DHIS2 we first have to convert the data a standardized JSON format, which can be done with the help of the dhis2eo library:

print(f"Creating payload with {len(dataframe)} values...")
payload = dataframe_to_dhis2_json(
    df=dataframe,
    org_unit_col="id",
    period_col="time",
    value_col=IMPORT_VALUE_COL,
    data_element_id=DHIS2_DATA_ELEMENT_ID,
)
payload['dataValues'][:3]

Creating payload with 208 values...

[{'orgUnit': 'O6uvpzGd5pu',
  'period': '2015',
  'value': '701020.0837291926',
  'dataElement': 'WUg3MYWQ7pt'},
 {'orgUnit': 'O6uvpzGd5pu',
  'period': '2016',
  'value': '721804.8800076675',
  'dataElement': 'WUg3MYWQ7pt'},
 {'orgUnit': 'O6uvpzGd5pu',
  'period': '2017',
  'value': '744126.2547043348',
  'dataElement': 'WUg3MYWQ7pt'}]

Import to DHIS2¶

print(f"Importing payload into DHIS2 (dryrun={DHIS2_DRY_RUN})...")
res = client.post("/api/dataValueSets", json=payload, params={"dryRun": str(DHIS2_DRY_RUN).lower()})
print(f'Result: {res["response"]["importCount"]}')

Importing payload into DHIS2 (dryrun=True)...
Result: {'imported': 208, 'updated': 0, 'ignored': 0, 'deleted': 0}

We have now successfully completed a full workflow for downloading, postprocessing, aggegating, and importing yearly WorldPop population data into DHIS2.