Reproducible Data Science + Python + Real-World Data

Reproducible Data Science with Python

About the textbook

Textbook outline
What is Reproducible Data Science?
Learning objectives
Learning resources
Software stack
Data sets
How to get the most of the open learning resource?
Acknowledgment
Code availability

End-to-End Data Science Project

Human mobility during UK lockdowns
Let’s formulate our research question
Data to address the question
Get started with Jupyter/Colab notebook
Let’s get coding
Python tools for data analysis
Load your data
View your data
Data Design: How are data generated?
Data Ethics, Privacy, and Fairness Risks
Describe your data
Exploratory data analysis
Data wrangling
Visualising mobility trends
Mobility changes across lockdowns
Zooming in: Regional mobility trends
Re-cap

Python Data Science on the Cloud

Python for Data Analysis on the Cloud
Get Started with Jupyter & Colab notebooks
Python Coding for Data Analysis
Descriptive statistics with NumPy
Getting help
Readability of your Python code
Running command-line commands in Jupyter/Colab notebook
Installing packages with pip
Jupyter magic commands
Loading real-world data with pandas
Hands-on exercise: Let’s practice your Python skills
Markdown Text Cells
Hands-on exercise: Let’s practice your Markdown skills

Open Reproducible Workflow

Open Reproducible Data Science Workflow
What is computational reproducibility and why it matters?
Open Reproducible Workflow in Jupyter/Colab Notebooks
Rules for Reproducible Workflow
Reproducible research report
Recording dependencies

data design & data wrangling

Data Design & Data Wrangling
Get started with pandas
Importing pandas
Loading your data
Loading Covid-19 data sets via URL
Viewing, Describing, and Accessing your Data
Accessing columns and rows in your data
Hands-on exercise
Handling Missing Values
Compare mobility data formats
Recording dependencies

Data Exploration & Visualisation

Exploratory Data Analysis and Visualisation
Learning resources
Addressing simple questions via Python EDA
Code reuse: creating your own functions
RQ1: How do countries differ in mobility trends?
Data visualisation with Seaborn
Changing figure appearance
Wide and long data format
Multi-plot visualisations
Hands-on exercise
RQ2: How do mobility trends change over time?
Visualising time series data
Bootstrapping for inference
RQ3: How have mobility trends differed across UK lockdowns?
Visualising lockdowns’ mobility as barplots
RQ4: How have mobility trends differed across UK counties during the third lockdown?
Multi-plot visualisations
Split-apply-combine
Summarising data by groups
RQ5: How do mobility categories relate to each other?
Visualising distributions
Plotting pairwise relationships
RQ6: How correlated are mobility categories across UK counties?
Visualising correlation
Visualising linear regression
Recording dependencies

Pattern Discovery using Unsupervised Learning

Discovering patterns in data
Machine Learning: What is it? What is it good for?
Research problem: clustering counties by mobility
k -means clustering
Let’s get coding with scikit-learn
Data preprocessing
Data standardisation
Choosing the optimal number of clusters
Performing for loop
Elbow plot
Dimensionality Reduction via PCA
Visualising mobility clusters
Related approaches: Hierarchical clustering
Hands-on exercise

Prediction using Supervised Learning

Learning from Data to Predict
The Prediction task
Prediction tasks in social sciences
Predictions gone wrong
Supervised learning
Research problem: vaccine hesitancy
Data: Understanding Society COVID-19
Defining Output and Input variables
Data wrangling
Overfitting and data splitting
Preprocessing the training data set
Unbalance class problem
Train models on training data
Model evaluation using Cross-validation
Testing model accuracy on new data
Preprocessing the test data set
Predicting vaccine hesitancy
Model evaluation on test data

Causal Inference

Introduction to Causal Inference
What causes what?
Confounding—basic definitions
Our example of collider bias
Drawing causal graphs
Statistical Models in Python
Collider confounder
Using weights to adjust for complex sample design
Additional material
Gender discrimination and collider bias
References

Network Analysis

Network analysis with NetworkX
Open network data repositories
Network representations
Creating an undirected network
Computing network diagnostics
Assortative mixing
Creating a directed network
A fun example—Star Wars network
Global network diagnostics
Meso-scale network diagnostics
Local network diagnostics
References

Data ethics

Data Ethics, Bias, and Fairness
Discussion: A-level results in 2020 England
Learning resources about the A-level controversy
Bias and Fairness in Data Science Systems

Powered by Jupyter Book

Index

By Valentin Danchev
© Copyright 2021.