Business analysis glossary
Data Profiling
What it is
Data profiling is examining a data set to understand its structure, content and quality: counts of rows, empty values, distinct values, ranges, formats, duplicates and patterns. It is usually done with SQL or profiling tools before using data for a migration, integration or report.
Why it matters
Assumptions about data are often wrong. Profiling replaces them with facts before design decisions are made, for example discovering that a mandatory field is empty in a third of records. Analysts profile data early in a project so that data problems become requirements and risks instead of surprises in testing.
Example
Before designing a new customer screen, the analyst profiles the old customer table with a few SQL queries. She finds twelve percent of mobile numbers missing, two thousand duplicate customers with the same PAN and dates of birth in three formats. Each finding becomes a cleansing task or a validation rule.