How to Build a Binary Classifier on Afrobarometer Survey Data

Binary classification is one of the most practical machine learning techniques for public policy, governance research, and social science analytics. 




With survey datasets such as Afrobarometer, you can predict outcomes like:

  • Whether a citizen trusts government institutions

  • Whether a respondent supports democracy

  • Whether a household has access to electricity

  • Whether a person believes the country is moving in the right direction


In this tutorial, we will build a binary classifier using Afrobarometer survey data with Python and scikit-learn.


Why Afrobarometer Data Is Ideal for Classification

Afrobarometer provides structured survey responses across African countries covering:

  • Governance

  • Democracy

  • Corruption

  • Public services

  • Economic conditions

  • Trust in institutions

  • Civic participation


Most variables are categorical, making the dataset excellent for:

  • Logistic regression

  • Decision trees

  • Random forests

  • Gradient boosting

  • Explainable AI for policymaking

The challenge is converting raw survey responses into machine-learning-ready features.


Step 1: Install the Required Libraries

pip install pandas scikit-learn matplotlib seaborn



Step 2: Load the Afrobarometer Dataset

Assume you downloaded Round 9 survey data as a CSV. 

If not, the data is converted as below:

import pandas as pd
from google.colab import files

# Install pyreadstat, as it's a missing dependency for pandas.read_spss
!pip install pyreadstat

# Upload the .sav file
uploaded = files.upload()

# Get uploaded filename
file_name = list(uploaded.keys())[0]

# Read the SPSS (.sav) file
df = pd.read_spss(
    file_name,
    convert_categoricals=True
)

# Preview dataset
print(df.head())

# Convert to CSV
csv_file_name = "afrobarometer_round9.csv"

df.to_csv(
    csv_file_name,
    index=False
)

print("Conversion complete.")

# Download the CSV file
files.download(csv_file_name)





We will create a dataset to use to illustrate this.



Inspect the dataset:

print(df.head())
print(df.columns)


Step 3: Define a Binary Target Variable

Suppose we want to predict whether a respondent trusts the president.

Original survey responses may look like:

ResponseMeaning
0Not at all
1Just a little
2Somewhat
3A lot

We can convert this into binary form:

  • 1 = High trust

  • 0 = Low trust

trust_mapping = {
    "Not at all": 0,
    "Just a little": 1,
    "Somewhat": 2,
    "A lot": 3,
    "Don't know/Haven't heard": 0  # Treat 'Don't know' as not high trust
}

df_raw["Q11_TRUST_PRESIDENT_NUM"] = df_raw["Q11_TRUST_PRESIDENT"].map(trust_mapping)

df_raw["high_trust_president"] = df_raw["Q11_TRUST_PRESIDENT_NUM"].apply(
    lambda x: 1 if x >= 2 else 0
)

This becomes the classification target.


Step 4: Select Predictor Variables

Choose variables related to economic conditions and demographics.

features = [
    "Q1_AGE_GROUP",
    "Q4_EDUCATION",
    "Q5_EMPLOYMENT",
    "Q6_RESIDENCE",
    "Q19_ELECTRICITY_ACCESS",
    "Q7_ECON_CONDITION" # Using Q7_ECON_CONDITION as a proxy for 'lived_poverty_index'
]

X = df_raw[features]
y = df_raw["high_trust_president"]


Step 5: Handle Missing Data

Survey datasets almost always contain missing responses.

X = X.dropna()
y = y.loc[X.index]

For production systems, consider imputation instead of dropping rows.


Step 6: Encode Categorical Variables

Machine learning models cannot directly process text categories.

X_encoded = pd.get_dummies(X, drop_first=True)

This transforms categories like:

employment_status→
Employed
Unemployed
Student

into numerical indicator columns.


Step 7: Split the Dataset

Separate training and testing data.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X_encoded,
    y,
    test_size=0.2,
    random_state=42
)

Typical policy analytics workflows use:

  • 80% training

  • 20% testing


Step 8: Train a Logistic Regression Classifier

Logistic regression is highly interpretable for survey data.

from sklearn.linear_model import LogisticRegression

model = LogisticRegression(max_iter=1000)

model.fit(X_train, y_train)

The model learns relationships between demographic/economic variables and institutional trust.


Step 9: Generate Predictions

y_pred = model.predict(X_test)

You can also get probabilities:

y_prob = model.predict_proba(X_test)[:, 1]

These probabilities are useful for policymaker-facing dashboards.



Step 10: Evaluate the Classifier

Use multiple metrics.

from sklearn.metrics import (
    accuracy_score,
    precision_score,
    recall_score,
    f1_score,
    confusion_matrix
)

print("Accuracy:", accuracy_score(y_test, y_pred))
print("Precision:", precision_score(y_test, y_pred))
print("Recall:", recall_score(y_test, y_pred))
print("F1 Score:", f1_score(y_test, y_pred))




Understanding Classification Metrics

Accuracy alone can be misleading. The core classification formulas are:

Accuracy = (TP + TN) / (TP + TN + FP + FN)

Precision = TP / (TP + FP)

Recall = TP / (TP + FN)

F1 Score = 2 × (Precision × Recall) / (Precision + Recall)


Where:

  • TP = True Positives

  • TN = True Negatives

  • FP = False Positives

  • FN = False Negatives

For governance research, recall may matter more if policymakers want to identify vulnerable populations.


Step 11: Visualize the Confusion Matrix

import matplotlib.pyplot as plt
from sklearn.metrics import ConfusionMatrixDisplay

ConfusionMatrixDisplay.from_estimator(
    model,
    X_test,
    y_test
)

plt.show()

This helps analysts understand where the model makes mistakes.


Step 12: Interpret Feature Importance

For logistic regression:

importance = pd.DataFrame({
    "Feature": X_encoded.columns,
    "Coefficient": model.coef_[0]
})

print(importance.sort_values(
    by="Coefficient",
    ascending=False
))

Positive coefficients increase the probability of high trust.

Negative coefficients decrease it.

This is extremely valuable in public policy analysis because the model becomes explainable.


Example Research Questions You Can Answer

Using Afrobarometer classification models, you can predict:

  • Trust in parliament

  • Satisfaction with democracy

  • Likelihood of voting

  • Perception of corruption

  • Access to public services

  • Confidence in elections

  • Support for opposition parties


This transforms survey data into actionable governance intelligence.


Moving Beyond Logistic Regression

After building a baseline model, experiment with:

  • Decision Trees

  • Random Forests

  • XGBoost

  • LightGBM

Tree-based models often improve predictive performance on complex survey interactions.

However, logistic regression remains the most interpretable model for policymakers and governance researchers.


Binary classification on Afrobarometer survey data combines machine learning with governance analytics. 

The process involves:

  1. Cleaning survey data

  2. Defining a binary outcome

  3. Encoding categorical variables

  4. Training a classifier

  5. Evaluating predictive performance

  6. Explaining the drivers behind predictions


For African public policy teams, NGOs, think tanks, and researchers, this approach enables evidence-based decision-making using real citizen sentiment data.


Machine learning is no longer limited to technology companies. Structured African survey data can now power predictive governance systems, democratic analysis, and public-sector intelligence at scale.


Build a Job‑Ready Portfolio in 16 Python Projects — Proven, Practical, and Profitable for $288.



Comments

Post a Comment

Popular posts from this blog

How to Filter Rows Using Boolean Indexing in Pandas (Afrobarometer Kenya Dataset)

Image
Learn how to filter rows in Pandas using boolean indexing with real Afrobarometer Kenya survey data. Build clean, ML-ready features with practical examples. Why Boolean Indexing Matters in Feature Engineering In Pandas, boolean indexing is how you select the exact slices of data that become features .  In Module 04 of the course, you’ll no longer exploring—you’ll be deciding what the model sees. In this case, with a dataset like Afrobarometer Kenya Round 10 , filtering is how you: Remove invalid survey responses Isolate specific demographic groups Prepare clean subsets for encoding Load and Inspect the Dataset This assumes that you’ve already converted the .sav file and loaded it: import pandas as pd df = pd.read_csv("afrobarometer_kenya_r10.csv") df.head() 1. Filter Valid Survey Responses Survey datasets contain codes like -1 , 8 , 9 for missing or “Don’t know”. df = df[df["trust_president"] >= 0] This keeps only valid numeric responses for trust in the pres...
Read more

How to Build a Pivot Table From Our World in Data Demographics

Image
Demographic datasets from Our World in Data are excellent for learning data analysis because they contain real-world statistics on population, life expectancy, fertility rates, migration, and age distribution across countries. One of the fastest ways to summarize this data is with a pivot table. In this tutorial, you will use Python and Pandas in Google Colab to create a demographic pivot table from an Our World in Data dataset. A pivot table is a data summarization tool used to reorganize, group, and analyze large datasets quickly. It allows you to turn raw rows of data into meaningful summaries without changing the original dataset. Why It’s Called a “Pivot” Table The word “pivot” means you can rotate or rearrange the structure of the data: Rows can become columns Columns can become grouped categories Values can be summarized differently You are essentially “pivoting” the dataset around specific fields. Think of a pivot table as: “A fast way to turn messy rows into orga...
Read more

How to Decide Whether to Drop or Fill Missing Value

Image
Learn how to decide whether to drop or fill missing values in datasets using practical, production-ready strategies. Understand trade-offs, bias risks, and data engineering best practices. Missing values are not just a data cleaning nuisance—they’re a signal . The decision to drop or fill them has direct consequences on model performance, statistical validity, and downstream business decisions. If you treat missing data casually, you risk introducing bias, noise, or false confidence into your analysis. This guide gives you a structured, decision-oriented framework you can use in real-world data workflows. 1. First Principle: Understand Why Data Is Missing Before deciding what to do , you need to understand why it’s missing . In data science, this is formalized into three categories: 1. Missing Completely at Random (MCAR) Missingness has no relationship to any variable. Example: A sensor randomly fails. Implication: Safe to drop rows without introducing bias. 2. Missing at Random (MAR...
Read more