The data understanding phase of CRISP-DM involves taking a closer look at the data available for mining. The deck states its purpose in one sentence: this step is critical in preventing problems during data preparation, which is typically the longest part of a project. Since preparation eats 50 to 70 percent of the effort (Chapter 2), an hour spent here is worth several later.
The phase involves four steps:
Data collection is the process of gathering information on targeted variables in an established system. The goal is to capture evidence that allows data analysis to formulate credible answers to the questions that have been posed.
The analyst then proceeds to:
"Report problems and solutions to aid with future replications" is the same reproducibility concern that returns twice more in this course: as the random_state seed in Chapter 8, and as the reason universities release their datasets publicly. If nobody can rerun your collection, nobody can check your result.
The deck opens this block with a picture and a question: red dots stand for places where surviving planes were shot. How would you reinforce the planes?
The deck gives two definitions:
During data collection and analysis, several biases can occur:
| Bias | Definition from the deck |
|---|---|
| Selection | The sample used for data collection is not representative of the population being studied |
| Sampling | Certain segments of the population are more likely to be included or excluded from the sample |
| Response | Participants in a survey or study provide inaccurate or misleading responses |
| Confirmation | The tendency to favor information that confirms pre-existing beliefs or hypotheses while ignoring contradictory evidence |
| Cultural | Data collection methods, survey questions or study designs are culturally insensitive or fail to account for cultural differences |
| Time-Interval | The timing of data collection influences the results |
| … and many others | |
Quoted on the slide, from Ronald H. Coase: "If you torture the data long enough, it will confess to anything." The deck pairs it with a 2023 Italian press case about data "tortured" to support a political position.
The everyday version: a person who believes in a specific political ideology might seek out news articles or social media posts that reinforce their beliefs, while ignoring or dismissing information that contradicts their views. This selective exposure strengthens the pre-existing opinion.
The "survivors" get studied, while the failures are excluded, leading to potentially flawed conclusions. Three examples from the deck:
When shopping for a new car, a person may see a car priced at $30,000, and then a second one priced at $25,000. Even if the second car is not objectively better or a great deal, the first price "anchors" the perception of value, leading them to think it is a better deal simply because it is cheaper than the first option.
After watching several news reports about airplane crashes, a person might overestimate the risk of flying and avoid it — despite it being statistically safer than driving — because the images and stories of crashes are more readily available in their memory.
Each scenario is one of the biases catalogued above. Pick the one that fits.
Fairness refers to the various attempts to correct algorithmic bias in automated decision processes based on ML models. Decisions made by such models after a learning process may be considered unfair if they were based on variables considered sensitive — e.g. gender, ethnicity, sexual orientation, or disability.
Two real cases are put on the table, and the class is asked how could we approach this problem?:
Both cases are downstream symptoms of upstream data decisions. A recruiting model trained on a decade of hiring outcomes inherits whatever selection bias produced those outcomes — the model is not wrong about the data, the data is a record of the bias. This is why the phase that catches it is data understanding, not evaluation. Chapter 5 continues the thread with the legal side: the GDPR list of sensitive personal data and the risk classes of the AI Act.
The more data we have, the more analysis we can do — however, more data != smarter data. Several disciplines focus on data (Data Science, Data Mining, Big Data, Business Intelligence). In Europe, and now in many areas of the world, there can be problems related to privacy: when is it "right" to protect privacy, and when does it become a limit?
Acquiring data is a time-consuming, investment- and knowledge-intensive process. So: how much data is enough?
Quoted from Chowdhury and Turin (2020): there is no set rule as to the number of variables to include in a prediction model, as it often depends on several factors. The one in ten rule, a rule that stipulates how many variables/parameters can be estimated from a data set, is quite popular in traditional clinical prediction modeling strategies (e.g., logistic regression and survival models). According to this rule, one variable can be considered in a model for every 10 events.
Apply it backwards to the brown spot case of Chapter 2: 100 attributes would call for on the order of 1000 events, and the project had 100 rows. The rule would have killed the collection design on paper, before an employee spent months driving across Emilia-Romagna.
Data collection is usually a costly process: an investment of time and money for
It is therefore necessary to carefully consider whether it is appropriate to acquire a new dataset: the considerations are not only engineering but also management and economic, and future needs must be foreseen in advance.
Sometimes companies have a lot of "old" data in their databases or information systems: they do not know what to do with it, and they re-collect data that already exists because it must be clean or something similar. In many business processes it is unclear which data is possible to collect (also due to privacy issues) and which data is really useful for the business.
| Route | Why | Watch out for |
|---|---|---|
| Publicly available data (datasets or databases) | Someone collected them for us. They can be free or for a fee. | The quality of the data made available must be checked. |
| Acquire a new set of data | It is not certain that public data well represents the problem we want to solve | Cost, time, hardware, and the bugs you notice at the end |
| We want specific data and thus generate specific expertise for the company (know-how) | ||
| We are forced to acquire data that, due to its sensitive nature, would not otherwise be available (privacy issues) | ||
| The company we work for already has a collection process that we can use |
Many universities publicly release their datasets: there are no requirements related to profit or non-disclosure agreements; it is the basis of the scientific method, in particular for the reproducibility of the results obtained — I release my data so that others can conduct my experiments and verify my results. The example given is ImageNet. Some platforms make datasets available for competitions, such as Kaggle.
The key question of the description step: does the data acquired satisfy the relevant requirements? This step also provides a basic understanding of the data on which subsequent steps will be built. For instance, if age is important and the data does not reflect the entire age range, it may be wise to collect a different dataset.
The data analyst examines the "surface" properties of the acquired data:
The public dataset used throughout the module: Iris, used in R. A. Fisher's classic 1936 paper, also found on the UCI Machine Learning Repository. It includes 3 iris species (Setosa, Virginica, Versicolor) with 50 samples each, and characterizes flowers with five columns: SepalLengthCm, SepalWidthCm, PetalLengthCm, PetalWidthCm, Species.
| Id | SepalLengthCm | SepalWidthCm | PetalLengthCm | PetalWidthCm | Species |
|---|---|---|---|---|---|
| 1 | 5.1 | 3.5 | 1.4 | 0.2 | Iris-setosa |
| 2 | 4.9 | 3 | 1.4 | 0.2 | Iris-setosa |
| 3 | 4.7 | 3.2 | 1.3 | 0.2 | Iris-setosa |
| 4 | 4.6 | 3.1 | 1.5 | 0.2 | Iris-setosa |
| 5 | 5 | 3.6 | 1.4 | 0.2 | Iris-setosa |
| … | … | … | … | … | … |
RangeIndex: 150 entries, 0 to 149
Data columns (total 5 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 sepal length (cm) 150 non-null float64
1 sepal width (cm) 150 non-null float64
2 petal length (cm) 150 non-null float64
3 petal width (cm) 150 non-null float64
4 species 150 non-null object
dtypes: float64(4), object(1)
memory usage: 6.0+ KBThree facts are readable at a glance and all three matter: 150 entries, 150 non-null in every column (no missing values — Chapter 5 would otherwise have work to do), and the dtypes: four floats and one object, i.e. the species is categorical and will need encoding (Chapter 6) for algorithms that require numeric input.
| Id | SepalLengthCm | SepalWidthCm | PetalLengthCm | PetalWidthCm | |
|---|---|---|---|---|---|
| count | 150 | 150 | 150 | 150 | 150 |
| mean | 75.5 | 5.84333 | 3.054 | 3.75867 | 1.19867 |
| std | 43.4454 | 0.828066 | 0.433594 | 1.76442 | 0.763161 |
| min | 1 | 4.3 | 2 | 1 | 0.1 |
| 25% | 38.25 | 5.1 | 2.8 | 1.6 | 0.3 |
| 50% | 75.5 | 5.8 | 3 | 4.35 | 1.3 |
| 75% | 112.75 | 6.4 | 3.3 | 5.1 | 1.8 |
| max | 150 | 7.9 | 4.4 | 6.9 | 2.5 |
Look at PetalLengthCm: mean 3.76 but median 4.35, and the 25% quartile at 1.6 while the 75% is at 5.1. The distribution is not a single hump — it is the signature of the three species sitting in different places, which is exactly what the exploration step will show. Note also the Id column, dutifully profiled with mean 75.5: a reminder that a profiling tool describes whatever you give it, including columns that carry no information (Chapter 6 removes them with a variance threshold, or by simply not selecting them).
In descriptive statistics, a box plot shows graphically the locality, spread, and skewness of numerical data. It is a standardized way of displaying the dataset based on the five-number summary:
The interquartile range is IQR = Q3 − Q1. The graphical elements:
Pick a column: the five numbers are the ones printed in the profiling table above; IQR, fences and the outlier verdict are computed live.
This task tackles the data mining questions, which can be addressed using querying, visualization, and reporting. Two deliverables:
On Iris the deck does exactly that: value distributions per species (petal_length vs species, sepal_length vs species) and a 2D visualization of the dataset.
At this point the analyst examines the quality of the data. The common issues to check:
| Check | Example from the deck |
|---|---|
| Missing values or attributes | Unknown date of death — is it missing? is the person alive? |
| Whether all possible values are represented | Are all age groups contained in the dataset? |
| Plausibility of values: review any attributes that may give answers conflicting with common sense | Teenagers with high income |
| Whether attributes with different values have similar meanings | low fat and diet |
| The spelling of values | law fat or low fat? |
Applied to the running example, the verdict is short: nothing to worry about in Iris — 150 non-null values in all five columns, four plausible float measurements and a clean categorical.
The deck closes with two references that formalise all of this: Dimensions of Data Quality (Sidi et al. 2012) and Metrics of Data Quality (Batini et al. 2009) — the literature that turns the checklist above into measurable dimensions.
Be able to recite the four steps and to give one concrete example of each quality check. The subtle one is the fourth: attributes with different values that have similar meanings. It is not a typo problem — low fat and diet are both spelled correctly and mean nearly the same thing, so a naive one-hot encoding (Chapter 6) will create two columns for one concept and split the evidence between them.
(1) Collection of initial data, (2) description of data, (3) exploration of data, (4) verification of data quality. It is critical in preventing problems during data preparation, which is typically the longest part of a project — 50 to 70 percent of the effort. Data understanding itself takes 20 to 30 percent.
The red dots mark where surviving planes were shot. Reinforcing those areas is exactly wrong: those are hits a plane could take and still come home. The planes hit in the unmarked areas — engines, cockpit — never returned, so they are absent from the sample. Generally: the survivors get studied while the failures are excluded, leading to potentially flawed conclusions. The same structure appears in start-up success stories, fitness testimonials and portfolios of top-performing stocks.
(1) Inclination or prejudice for or against one person or group, especially in a way considered to be unfair. (2) A systematic distortion of a statistical result due to a factor not allowed for in its derivation.
Selection: the sample used for data collection is not representative of the population being studied. Sampling: certain segments of the population are more likely to be included or excluded from the sample. Response: participants in a survey or study provide inaccurate or misleading responses — the brown spot questions about legally constrained pesticides are a textbook case.
Fairness refers to the various attempts to correct algorithmic bias in automated decision processes based on ML models. Decisions may be considered unfair if they were based on sensitive variables, e.g. gender, ethnicity, sexual orientation or disability. The cases cited: Amazon scrapping a secret AI recruiting tool biased against women (2018), and a Google AI tool producing offensive historical images (2024).
According to the one in ten rule (Chowdhury and Turin 2020), one variable can be considered in a model for every 10 events. There is no set rule for the number of variables to include, but this one is quite popular in traditional clinical prediction modeling (logistic regression, survival models). Applied to the brown spot project: 100 attributes would need on the order of 1000 events, and the dataset had 100 rows — hence "not statistically relevant".
Minimum (Q0, 0th percentile), first quartile (Q1, 25th), median (Q2, 50th), third quartile (Q3, 75th), maximum (Q4, 100th). IQR = Q3 − Q1. The box is drawn from Q1 to Q3; the whiskers are based on 1.5 · IQR and are drawn up to the largest/lowest observed data point that falls within that distance, so their lengths can look unequal; all points outside are plotted as outliers.
A box plot shows graphically the locality, spread and skewness of numerical data. The Iris profiling: 150 entries, all columns 150 non-null (no missing values), four float64 and one object column (the species will need encoding), and a petal length whose mean (3.76) sits well below its median (4.35) with quartiles at 1.6 and 5.1 — the fingerprint of three species occupying different ranges.
Missing values (unknown date of death: missing, or the person is alive?); whether all possible values are represented (all age groups present?); plausibility (teenagers with high income); attributes with different values but similar meanings (low fat and diet); spelling (law fat or low fat?).
Four reasons: public data may not well represent the problem we want to solve; we want specific data and thus generate specific know-how for the company; we are forced to collect data that, being sensitive, would not otherwise be available; the company already has a collection process we can use. The cost side: programming or learning an acquisition tool, handling large amounts of data, testing for bugs (often noticed at the end of the process), and new hardware — with management and economic considerations, not only engineering ones.
Because there are no profit or NDA requirements and it is the basis of the scientific method, in particular for the reproducibility of results: I release my data so that others can conduct my experiments and verify my results. ImageNet is the example given; Kaggle is cited as a platform releasing datasets for competitions.
Public data used in R. A. Fisher's classic 1936 paper, available from the UCI Machine Learning Repository: 3 species (Setosa, Virginica, Versicolor), 50 samples each (150 rows), four measurements per flower plus the species. It is small, clean ("nothing to worry about in Iris"), and just structured enough to demonstrate description, exploration, PCA, decision trees, k-NN and neural networks without any data quality noise in the way.