标签：#dataanalytics

Introduction Every data science, analytics, business intelligence, and any other data-related report depends on a clean dataset. However, almost 90% of the time, we can't clean and process to begin with. In most cases, the datasets are characterized by inconsistent formatting, varying levels of missing values, and other structural anomalies. While there are different tools for data preprocessing, Python's pandas is often utilized. Thus, to provide a detailed understanding of the pandas library's capabilities for data preparation, this project focused on HR data. The primary steps in data preprocessing were as follows. Phase 1: Imported the Required Libraries and Loaded the Dataset Required libraries were imported as pandas (as pd) and numpy (as np). Read the raw dataset using pd.read_excel #Import required libraries import pandas as pd import numpy as np #Load Dataset hrdata = pd . read_excel ( r " C:\Users\HP\OneDrive\Desktop\LuxDev Tutorials\Python Projects\HR Data Analysis Project.ipynb\HR_Dirty_Data.xlsx " ) Phase 2: Initial Data Quality Assessment Constructed an initial quality_report DataFrame to evaluate: Column names and total column count Data types per column Total row count Missing value counts and missing percentages Unique value counts per column quality_report = pd . DataFrame ({ " Column " : hrdata . columns , " Total Columns " : len ( hrdata . columns ), " Data_Type " : hrdata . dtypes . astype ( str ), " Total_Rows " : len ( hrdata ), " Missing_Values " : hrdata . isna (). sum (). values , " Missing_Percentage " : ( hrdata . isna (). mean () * 100 ). round ( 2 ). values , " Unique_Values " : hrdata . nunique (). values , }) quality_report Phase 3: Data Cleaning Steps Employed Step 1: Renamed and Standardized Column Heads Explored existing column headings using print ( f " Current Column Names Heading \n " , hrdata . columns ) Cleaned headers automatically using the created clean_column_names() function to: Convert letters to lowercase (.str.lower())