NumPy Standard Deviation (ddof and Axes)
NumPy’s std() function calculates standard deviation, but it has defaults that can trip you up. By default, NumPy computes population standard deviation (dividing by N), while pandas defaults to sample standard deviation (dividing by N-1). The ddof parameter controls this, and understanding axis behavior is essential for working with multidimensional data. If you’re new to NumPy arrays, read the indexing and slicing guide first. (NumPy 2.5.3 and pandas 2.3.3 used in examples below.)
What is Standard Deviation?
Standard deviation measures how spread out values are from the mean. A small standard deviation means values cluster near the mean. A large one means they’re scattered.
There are two formulas:
- Population std: √(Σ(x - mean)² / N)
- Sample std: √(Σ(x - mean)² / (N-1))
Use population when your data is the entire population. Use sample when your data is a sample from a larger population. The sample version uses N-1 (Bessel’s correction) to give an unbiased estimate of the population variance.
Basic Usage
Start with a simple array:
import numpy as np
myList = [10.5, 12.2, 4.2, 5.5, 4.2, 7.7, 9]
myStandardDeviation = np.std(myList)
print(myStandardDeviation)
Output:
2.9048868044966527
This is the population std with the default ddof=0.
The ddof Parameter
The ddof parameter stands for Delta Degrees of Freedom. It changes the divisor:
standard_deviation = sqrt(sum((x - mean)^2) / (N - ddof))
The default is ddof=0, giving population std. Set ddof=1 for sample std.
data = np.array([1, 2, 3, 4, 5])
print("np.std(data, ddof=0) - population:", np.std(data, ddof=0))
print("np.std(data, ddof=1) - sample:", np.std(data, ddof=1))
Output:
np.std(data, ddof=0) - population: 1.4142135623730951
np.std(data, ddof=1) - sample: 1.5811388300841898
Sample std is slightly larger because it divides by a smaller number (N-1 instead of N).
The NumPy vs pandas Gotcha
NumPy defaults to ddof=0 (population), but pandas defaults to ddof=1 (sample). The same data gives different answers:
import pandas as pd
data = [10, 12, 15, 18, 20]
print("np.std(data):", np.std(data))
print("pd.Series(data).std():", pd.Series(data).std())
Output:
np.std(data): 3.687817782917155
pd.Series(data).std(): 4.123105625617661
If you’re comparing results, check which version both libraries are using. Also note that the Python statistics module has pstdev() for population and stdev() for sample, matching these formulas.
Manual Computation
Here’s the formula in code, verified against np.std():
data = np.array([2, 4, 6, 8])
mean = np.mean(data)
# Population std (ddof=0)
var_pop = np.sum((data - mean) ** 2) / len(data)
std_pop = np.sqrt(var_pop)
print("Manual population std:", std_pop)
print("np.std(data, ddof=0):", np.std(data, ddof=0))
# Sample std (ddof=1)
var_sample = np.sum((data - mean) ** 2) / (len(data) - 1)
std_sample = np.sqrt(var_sample)
print("Manual sample std:", std_sample)
print("np.std(data, ddof=1):", np.std(data, ddof=1))
Output:
Manual population std: 2.23606797749979
np.std(data, ddof=0): 2.23606797749979
Manual sample std: 2.581988897471611
np.std(data, ddof=1): 2.581988897471611
Working with Multidimensional Arrays
Use the axis parameter to compute std along a specific dimension. axis=0 computes std down the rows (one value per column). axis=1 computes std across the columns (one value per row).
myArray = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
print("Array:")
print(myArray)
print("\nStandard deviation by column (axis=0):")
print(np.std(myArray, axis=0))
print("\nStandard deviation by row (axis=1):")
print(np.std(myArray, axis=1))
Output:
Array:
[[1 2 3]
[4 5 6]
[7 8 9]]
Standard deviation by column (axis=0):
[2.44948974 2.44948974 2.44948974]
Standard deviation by row (axis=1):
[0.81649658 0.81649658 0.81649658]
Handling Missing Values
Use np.nanstd() to ignore NaN values:
data = np.array([1.0, 2.0, np.nan, 4.0, 5.0])
print("Data:", data)
print("np.nanstd(data):", np.nanstd(data))
Output:
Data: [ 1. 2. nan 4. 5.]
np.nanstd(data): 1.5811388300841898
Without nanstd(), regular std() would return NaN.
Standard Deviation in Practice
When analyzing data, know whether your data is a sample or population. If you’re working with a sample (most real-world cases), set ddof=1. If you’re computing properties of the entire dataset you have, use ddof=0. Pay attention to axes when working with matrices or DataFrames—one axis means one value per column or row, while no axis means one value for the entire array.
For detailed parameter documentation, see the NumPy std() reference.
Which default catches you more often—the ddof difference or the axis direction?