Importing data values using dhis2-python-client

This section walks you through importing data values into DHIS2 using the dhis2-python-client library.

Below are crucial steps to follow:

Get credentials
Connect to DHIS2 and verify access
Build a valid dataValueSets payload (JSON or CSV)
Send your payload

We start by importing the libraries we’ll need:

import pandas as pd

from dhis2_client import DHIS2Client
from dhis2_client.settings import ClientSettings

from dhis2eo.integrations.pandas import dataframe_to_dhis2_json

1) Configure your environment and connect to DHIS2¶

Connect to your DHIS2 instance using your credentials and the dhis2-python-client:

# Client configuration
cfg = ClientSettings(
  base_url="https://play.im.dhis2.org/stable-2-42-3-1",
  username="admin",
  password="district")

client = DHIS2Client(settings=cfg)
info = client.get_system_info()

# Check if everything is working.
# You should see your current DHIS2 version info.
print("Current DHIS2 version:", info["version"])

Current DHIS2 version: 2.42.3.1

2) Construct payload¶

With a connection to DHIS2 established, the next step is to construct the data payload to be imported. As shown in the previous aggregation notebook, climate data aggregated to organisation units often produces outputs such as a pandas.DataFrame. To import these aggregated data into DHIS2, the DataFrame must first be converted to the JSON format expected by the api/dataValueSets endpoint of the DHIS2 REST API.

For this tutorial, we assume the data have already been aggregated and provided as a CSV file, which we load into a pandas.DataFrame below:

df = pd.read_csv('../data/sample_payload.csv')
df

To construct a valid JSON payload for importing a pandas.DataFrame into DHIS2, we can use the dhsi2eo utility function dataframe_to_dhis2_json:

payload = dataframe_to_dhis2_json(
    df = df,                        # pandas.DataFrame
    org_unit_col = 'orgUnit',            # column containing the org unit id
    period_col = 'period',      # column containing the period
    value_col = 'value',              # column containing the value
    data_element_id = 'BXgDHhPdFVU' # id of the DHIS2 data element
)
payload

{'dataValues': [{'orgUnit': 'O6uvpzGd5pu',
   'period': '202501',
   'value': '23.68',
   'dataElement': 'BXgDHhPdFVU'},
  {'orgUnit': 'fdc6uOvgoji',
   'period': '202501',
   'value': '23.96',
   'dataElement': 'BXgDHhPdFVU'},
  {'orgUnit': 'lc3eMKXaEfw',
   'period': '202501',
   'value': '24.52',
   'dataElement': 'BXgDHhPdFVU'},
  {'orgUnit': 'jUb8gELQApl',
   'period': '202501',
   'value': '23.06',
   'dataElement': 'BXgDHhPdFVU'},
  {'orgUnit': 'PMa2VCrupOd',
   'period': '202501',
   'value': '24.45',
   'dataElement': 'BXgDHhPdFVU'},
  {'orgUnit': 'kJq2mPyFEHo',
   'period': '202501',
   'value': '23.27',
   'dataElement': 'BXgDHhPdFVU'}]}

If your data does not come from a Pandas DataFrame, it’s also possible to manually create the data payload to send to the API endpoint by following the same format as above. See the DHIS2 API documentation for a full description of the JSON format expected by DHIS2.

3) Send your payload¶

Once we are done preparing our payload we can proceed to send it to DHIS2 using the dhis2-python-client. For convenience, the client provides a helper method, post_data_value_set(), which handles the POST request and returns the API response:

res = client.post_data_value_set(payload)
res['response']['importCount']

{'imported': 0, 'updated': 0, 'ignored': 6, 'deleted': 0}

The same result can also be achieved by using the more generic post() method to submit raw DHIS2 API POST requests, which provides additional flexibility, such as customizing the import parameters:

res = client.post("/api/dataValueSets", json=payload, params={'dryRun': 'true'})
res['response']['importOptions']

{'idSchemes': {},
 'dryRun': True,
 'async': False,
 'importStrategy': 'CREATE_AND_UPDATE',
 'mergeMode': 'REPLACE',
 'reportMode': 'FULL',
 'skipExistingCheck': False,
 'sharing': False,
 'skipNotifications': False,
 'skipAudit': False,
 'datasetAllowsPeriods': False,
 'strictPeriods': False,
 'strictDataElements': False,
 'strictCategoryOptionCombos': False,
 'strictAttributeOptionCombos': False,
 'strictOrganisationUnits': False,
 'strictDataSetApproval': False,
 'strictDataSetLocking': False,
 'strictDataSetInputPeriods': False,
 'requireCategoryOptionCombo': False,
 'requireAttributeOptionCombo': False,
 'skipPatternValidation': False,
 'ignoreEmptyCollection': False,
 'force': False,
 'firstRowIsHeader': True,
 'skipLastUpdated': False,
 'mergeDataValues': False,
 'skipCache': False}

For more details on submitting data values to DHIS2, see the dhis2-python-client documentation.

4) Troubleshooting¶

Unauthorized: Check credentials and user permissions.
Not found: Verify data element, org unit, and combos (if you have used non-default ones) exist.
Conflicts: Ensure dataset assignments and period are correct.
Locked periods: Unlock dataset period if needed.
Value types: Match the data element value type.