Business understanding is the first phase of CRISP-DM and, in the words of the deck, perhaps the most important and hard phase of any data mining project (Shearer 2000). It focuses on four things:
It is essential to understand which data should later be analyzed and how. Everything that happens in the following four chapters — collecting, describing, cleaning, encoding, modelling — is downstream of decisions taken here, which is why a mistake in this phase is the most expensive mistake available.
Business understanding is not a preamble to the technical work: it is the phase that decides what would count as a correct answer. The deck states the risk in one line — ensure that the project does not produce the right answers to the wrong questions.
A company wants to track the spreading of the brown spot of European pear in Emilia-Romagna. Brown spots are caused by a fungus (Stemphylium vesicarium) that damages pear fruits and causes economic loss for producers. The company also wants to find out whether there are environmental factors related to the spreading of the fungus and to the level of damage.
Ideally, the goal is to build a dataset like this one, and to learn relationships between environmental features and Level of Damage — the target of the analysis:
| Field | AVG. Temperature | Presence of Water Source | … | Level of Damage |
|---|---|---|---|---|
| Field#1 | 14 | Close distance | … | High |
| Field#2 | 14.5 | Medium distance | … | Medium |
| Field#3 | 13.8 | Medium distance | … | Low |
| Field#4 | … | … | … | … |
In other words: can we identify features that allow us to predict an accurate level of damage? The metric proposed is to predict the level of damage with high accuracy — which immediately raises the question the lecturer poses to the class: how would you define accuracy? And then: what features would you collect? How do we translate this problem into a quantitative metric? How can we collect the data?
The company prepared a questionnaire on paper with around 100 questions, and one of the employees moved across Emilia-Romagna to ask questions to the farmers and collect their answers. Once the employee had collected almost 100 questionnaires, the company asked the analysts to apply machine learning techniques to understand which environmental conditions could be related to the spreading of the fungus.
This project was missing a proper plan (business + data understanding), and turned out to be a failure for the company. Even applying machine learning techniques to the data, the results obtained would not be statistically relevant.
Advance the company and the analysts one step at a time and watch the state of the project. The verdict is computed from the same numbers the deck reports.
This case study is the canonical answer to "why is business understanding the hardest phase?". Everything that went wrong is a business-understanding failure: the target was never turned into a measurable metric, the collection instrument (paper) was never checked against the analysis requirement (a digital table), and nobody asked in advance how many rows do we need for 100 features. Chapter 4 gives the rule of thumb that would have caught it: one in ten — one variable per ten events.
Understanding a client's true goal is critical to uncovering the important factors involved in the project. The deck lists four rules:
The worked example: the business goal could be to retain current customers by predicting when they are prone to move to a competitor. The related questions look like this:
Notice the grammar of these questions: each names a measurable input (primary channel, ATM fees), a measurable outcome (stay or go, number of high-value customers who leave) and a population (bank customers, high-value customers). That is already halfway to a feature matrix X and a target y.
Outline the available resources to accomplish the project, from personnel to software. The core question is: discover what data is available to meet the primary business goal.
Example given: to address the business question, a minimum number of customers over age 50 is necessary. Availability is not only "do we have a table", it is "do we have enough of the right rows".
The rest of the assessment is project management:
The deck flags this block as "the added value of DTM": in a real organisation, assessing the situation also means mapping people and politics.
Without this step you cannot claim an improvement: an accuracy figure means nothing if nobody wrote down what the company was doing before.
Formulate the project objectives in business terms, then ask and refine sharp questions that are relevant, specific, and unambiguous. Data science is a process that uses numbers to answer such questions, and you typically use data science or machine learning to answer five types of questions:
| # | Question | Task family |
|---|---|---|
| 1 | How much or how many? | Regression |
| 2 | Which category? | Classification |
| 3 | Which group? | Clustering |
| 4 | Is this unusual? | Anomaly detection |
| 5 | Which option should be taken? | Recommendation |
Read the question and pick the family. The questions are the ones used across the decks.
The metrics must be SMART:
| Letter | Meaning | Question it forces you to answer |
|---|---|---|
| S | Specific | Which population, which outcome, which period? |
| M | Measurable | With which number, from which table? |
| A | Achievable | Is there enough data and enough signal for this? |
| R | Relevant | Does it relate to at least one business objective? |
| T | Time-bound | By when, and over which window is it measured? |
Two consequences stated on the slide:
"Predict the level of damage with high accuracy" — the metric proposed in the brown spot case — fails S, M and T. High compared to what? Accuracy measured on which held-out fields? Within which season? The lecturer asks the class exactly this: how would you define accuracy? Answering that question before collecting the data is the whole point of the phase.
Describe the intended plan for achieving the goals, including:
The generally accepted industry timeline standards (Shearer 2000) are the single most quoted numbers of this course:
| Phase | Share of time and effort |
|---|---|
| Data Preparation | 50 to 70 percent |
| Data Understanding | 20 to 30 percent |
| Modeling | 10 to 20 percent |
| Evaluation | 10 to 20 percent |
| Business Understanding | 10 to 20 percent |
| Deployment planning | 5 to 10 percent |
Distribute 100 points of effort and compare your plan with the industry standard. The widget flags any phase outside its published range.
Identifying and delving into the problem to solve is an interdisciplinary phase. Even if you have good knowledge in the fields of computer and data science, you may still miss the domain knowledge that is fundamental to understanding and modeling the problem.
This is why "create a glossary of business and data mining terms" appears in the assessment checklist, and why the next chapter is made of three case studies from three unrelated domains — pear orchards, urban mobility, stock indices and endurance sport. In each one, the hard part is not the algorithm: it is knowing what a staypoint, a market cap weight or a normalized power means before you compute it.
The following chapter takes exactly this route: three complete business-understanding exercises where the domain knowledge is the deliverable.
(1) Determining the business objectives; (2) assessing the context of the analysis; (3) determining and translating objectives into data mining goals; (4) producing the project plan. Its purpose is to understand which data should later be analyzed, and how.
A company wanted to track the spreading of the brown spot of European pear (Stemphylium vesicarium) in Emilia-Romagna and to find environmental factors related to the spreading and to the level of damage. Data was collected with a paper questionnaire of about 100 questions, carried around the region by one employee, producing about 100 questionnaires. The analysts could not apply ML to data on paper, so a Google Form was built and digitalization took about a month. Then came four issues: answers inconsistent with the provided options; open questions; untrustworthy answers on legally constrained pesticides and fertilizers; and a dataset of 100 features and 100 rows. The project was missing a proper plan (business + data understanding) and failed: even applying ML, the results would not be statistically relevant.
That a technically flawless analysis can still be worthless if the objective was not the client's true goal. The countermeasure listed on the slide is procedural: uncover the primary business objective and the related questions, beware of unattainable goals, and make sure that each success criterion relates to at least one specified business objective.
1. How much or how many? → regression. 2. Which category? → classification. 3. Which group? → clustering. 4. Is this unusual? → anomaly detection. 5. Which option should be taken? → recommendation.
Specific, Measurable, Achievable, Relevant, Time-bound. If the business goal cannot be effectively translated into a data mining goal, it may be wise to consider redefining the problem. Success can be defined as reducing lost customers by 10%, or as a better understanding of the customer base, or as achieving a certain level of predictive accuracy.
50 to 70 percent of the time and effort goes into Data Preparation; 20 to 30 percent into Data Understanding; only 10 to 20 percent into each of Modeling, Evaluation and Business Understanding; and 5 to 10 percent into Deployment planning (Shearer 2000).
Whether you have data relevant to the question: measures of the target and features related to the target, and whether those measures are accurate. An existing system might not have the data needed, in which case you find external sources or update the systems to collect new data. The example given is that a minimum number of customers over age 50 is necessary to answer the business question. The assessment also lists risks, assumptions, a glossary of business and data mining terms, and a cost-benefit analysis.
Because the phase is interdisciplinary: the data scientist typically lacks the domain knowledge that is fundamental to understanding and modeling the problem, and the domain experts lack the data mining vocabulary. In an interdisciplinary team a shared glossary is described as essential.
Specific steps with a proposed timeline, an assessment of potential risks, and an initial assessment of the tools and techniques needed to support the project.
You need to fix, at least: the population (pear fields in Emilia-Romagna with the given cultivars), the target encoding (Level of Damage as an ordinal Low/Medium/High), the metric and the evaluation protocol (accuracy measured on fields never used for training), a threshold that counts as success, and the window (one growing season). The deck does not hand you a reformulation — it hands you the question "how would you define accuracy?", which is the exam question.
Because assessing the situation asks whether an accurate measure of target and features is available, and whether the existing system can produce it. A paper instrument produces answers inconsistent with the provided options and free text from open questions, i.e. no reliable encoding; a single employee travelling the region caps the number of rows at about a hundred, i.e. no statistical relevance for 100 attributes; and questions about legally constrained pesticides invite untrustworthy answers, i.e. a systematic distortion — a bias, in the vocabulary of Chapter 4.