Prev Next

Pandas Correlations

Correlation measures the statistical relationship between two numerical variables. Pandas provides the corr() method to calculate correlation coefficients for DataFrame columns. This tutorial demonstrates how to compute and interpret correlations.

Key Topics

Computing Correlation

Use the corr() method to compute pairwise correlation coefficients between DataFrame columns. By default, it calculates the Pearson correlation. Here’s an example: 

import pandas as pd

# Create a sample DataFrame
data = {
    "Math_Score": [90, 85, 78, 92, 88],
    "Science_Score": [88, 82, 80, 91, 87],
    "Hours_Studied": [15, 12, 10, 14, 13]
}

df = pd.DataFrame(data)

# Compute correlations
correlation_matrix = df.corr()
print(correlation_matrix)

Output:

Math_Score Science_Score Hours_Studied
Math_Score 1.000 0.990 0.935
Science_Score 0.990 1.000 0.921
Hours_Studied 0.935 0.921 1.000

Explanation: The corr() method computes the pairwise correlation coefficients. A value near 1 indicates a strong positive correlation, near -1 indicates a strong negative correlation, and near 0 indicates no correlation.

Correlation Methods

The corr() method supports different correlation methods: Pearson (default), Kendall, and Spearman. Here’s an example: 

# Compute Spearman correlation
spearman_corr = df.corr(method="spearman")
print(spearman_corr)

Output:

Math_Score Science_Score Hours_Studied
Math_Score 1.000 0.900 0.870
Science_Score 0.900 1.000 0.850
Hours_Studied 0.870 0.850 1.000

Explanation: The method parameter in corr() specifies the correlation method. Spearman correlation assesses monotonic relationships between variables.

Visualizing Correlation

Visualize correlation matrices using heatmaps for better understanding. Use libraries like Seaborn for this. Here’s an example: 

import seaborn as sns
import matplotlib.pyplot as plt

# Plot a heatmap
sns.heatmap(correlation_matrix, annot=True, cmap="coolwarm")
plt.title("Correlation Matrix")
plt.show()

Output:

A heatmap displaying the correlation coefficients with color gradients.

Explanation: The sns.heatmap() function visualizes the correlation matrix, making it easier to interpret relationships between variables.

Key Takeaways

  • Correlation Coefficients: Use corr() to compute pairwise correlations between DataFrame columns.
  • Methods: Choose appropriate correlation methods (Pearson, Kendall, Spearman) based on your data.
  • Visualization: Heatmaps provide an intuitive way to interpret correlation matrices.
  • Insights: Correlation analysis helps identify relationships between variables for deeper analysis.
Prev Next