Prev Next

String Operations in Pandas

Cleaning and transforming text data is a common requirement in data analysis. Pandas provides numerous string methods through the .str accessor, enabling operations such as removing unwanted characters, splitting, replacing, and formatting text. This tutorial covers key string operations for data cleaning.

Removing Unwanted Characters

Use the str.replace() method to remove or replace unwanted characters in a string column. Here’s an example: 

import pandas as pd

# Create a sample DataFrame
data = {
    "Names": ["Mr. Karthick", "Ms. Durai", "Dr. Praveen"],
    "Ages": [25, 30, 22]
}

df = pd.DataFrame(data)

# Remove prefixes like 'Mr.', 'Ms.', 'Dr.'
df["Cleaned_Names"] = df["Names"].str.replace(r"^(Mr\.|Ms\.|Dr\.)\s", "", regex=True)
print(df)

Output:

Names Ages Cleaned_Names
Mr. Karthick 25 Karthick
Ms. Durai 30 Durai
Dr. Praveen 22 Praveen

Explanation: The str.replace() method uses a regular expression to remove titles like Mr., Ms., and Dr., leaving only the names.

Splitting and Extracting

Use str.split() to split text into multiple parts and str.extract() to extract specific patterns. Here’s an example: 

# Split names into first and last
name_parts = df["Cleaned_Names"].str.split(expand=True)
df["First_Name"] = name_parts[0]
df["Last_Name"] = name_parts[1]

# Extract names starting with 'K'
df["Starts_with_K"] = df["Cleaned_Names"].str.extract(r"^(K\w+)")
print(df)

Output:

Names Ages Cleaned_Names First_Name Last_Name Starts_with_K
Mr. Karthick 25 Karthick Karthick NaN Karthick
Ms. Durai 30 Durai NaN Durai Durai

Explanation: Splitting and extracting provide flexible ways to parse and transform text. This example splits names into first and last names and extracts names starting with the letter "K."

Key Takeaways

  • String Cleaning: Use str.replace() for removing unwanted characters.
  • Splitting: Split text into components using str.split().
  • Extracting: Extract patterns with str.extract() for advanced text transformations.
  • Efficiency: String methods in Pandas are vectorized for faster operations on large datasets.
Prev Next