Part II — Understanding the problem and the data · Chapter 2

Business Understanding

~28 min read4 interactive widgets3 plates

In this chapter

  1. What the phase is for
  2. Case study: brown spot of European pear
  3. Determine the business objectives
  4. Assess the situation
  5. Determine the data mining goals: the five questions
  6. SMART metrics
  7. Produce a project plan (and where the effort goes)
  8. Interdisciplinarity
  9. Check your understanding

1. What the phase is for

Business understanding is the first phase of CRISP-DM and, in the words of the deck, perhaps the most important and hard phase of any data mining project (Shearer 2000). It focuses on four things:

  1. Determining the business objectives
  2. Assessing the context of the analysis
  3. Determining and translating objectives into data mining goals
  4. Producing the project plan

It is essential to understand which data should later be analyzed and how. Everything that happens in the following four chapters — collecting, describing, cleaning, encoding, modelling — is downstream of decisions taken here, which is why a mistake in this phase is the most expensive mistake available.

Key idea

Business understanding is not a preamble to the technical work: it is the phase that decides what would count as a correct answer. The deck states the risk in one line — ensure that the project does not produce the right answers to the wrong questions.

2. Case study: brown spot of European pear

The request

A company wants to track the spreading of the brown spot of European pear in Emilia-Romagna. Brown spots are caused by a fungus (Stemphylium vesicarium) that damages pear fruits and causes economic loss for producers. The company also wants to find out whether there are environmental factors related to the spreading of the fungus and to the level of damage.

The ideal dataset

Ideally, the goal is to build a dataset like this one, and to learn relationships between environmental features and Level of Damage — the target of the analysis:

FieldAVG. TemperaturePresence of Water SourceLevel of Damage
Field#114Close distanceHigh
Field#214.5Medium distanceMedium
Field#313.8Medium distanceLow
Field#4

In other words: can we identify features that allow us to predict an accurate level of damage? The metric proposed is to predict the level of damage with high accuracy — which immediately raises the question the lecturer poses to the class: how would you define accuracy? And then: what features would you collect? How do we translate this problem into a quantitative metric? How can we collect the data?

How the data was actually collected

The company prepared a questionnaire on paper with around 100 questions, and one of the employees moved across Emilia-Romagna to ask questions to the farmers and collect their answers. Once the employee had collected almost 100 questionnaires, the company asked the analysts to apply machine learning techniques to understand which environmental conditions could be related to the spreading of the fungus.

What the analysts found

Outcome

This project was missing a proper plan (business + data understanding), and turned out to be a failure for the company. Even applying machine learning techniques to the data, the results obtained would not be statistically relevant.

WHAT WAS NEEDED many fields observed, few well-chosen features features (few) rows >> columns enough events per variable WHAT ARRIVED one attribute per question, one row per questionnaire 100 features 100 rows × 100 columns not statistically relevant + paper questionnaires, ~1 month to digitalize + answers inconsistent with the provided options + open questions with no encoding + answers on legally constrained pesticides: trustworthy? the plan was missing, not the algorithm
Plate 2.1 — The shape of the failure. The project did not fail on the choice of model: it failed because the business and data understanding phases never happened, so the dataset that arrived had as many columns as rows.

Widget — replay the project

Advance the company and the analysts one step at a time and watch the state of the project. The verdict is computed from the same numbers the deck reports.

For the exam

This case study is the canonical answer to "why is business understanding the hardest phase?". Everything that went wrong is a business-understanding failure: the target was never turned into a measurable metric, the collection instrument (paper) was never checked against the analysis requirement (a digital table), and nobody asked in advance how many rows do we need for 100 features. Chapter 4 gives the rule of thumb that would have caught it: one in ten — one variable per ten events.

3. Determine the business objectives

Understanding a client's true goal is critical to uncovering the important factors involved in the project. The deck lists four rules:

The worked example: the business goal could be to retain current customers by predicting when they are prone to move to a competitor. The related questions look like this:

Editor's note

Notice the grammar of these questions: each names a measurable input (primary channel, ATM fees), a measurable outcome (stay or go, number of high-value customers who leave) and a population (bank customers, high-value customers). That is already halfway to a feature matrix X and a target y.

4. Assess the situation

Outline the available resources to accomplish the project, from personnel to software. The core question is: discover what data is available to meet the primary business goal.

Example given: to address the business question, a minimum number of customers over age 50 is necessary. Availability is not only "do we have a table", it is "do we have enough of the right rows".

The rest of the assessment is project management:

Things are more complex in a company

The deck flags this block as "the added value of DTM": in a real organisation, assessing the situation also means mapping people and politics.

  1. Identify key individuals in the organization — also to provide domain expertise.
  2. Identify the business units that will be affected by the data mining project.
  1. Identify the problem area, such as marketing, customer care, or business development.
  2. Describe the problem in general terms.
  3. Clarify the prerequisites of the project: what are the motivations behind it? Does the business already use data mining?
  4. Check on the status of the project within the business group: has the effort been approved, or does data mining need to be "advertised" as a key technology for the group?
  5. If necessary, prepare informational presentations on data mining for your organization.
  1. Describe any solutions currently used to address the business problem.
  2. Describe the advantages and disadvantages of the current solution.

Without this step you cannot claim an improvement: an accuracy figure means nothing if nobody wrote down what the company was doing before.

5. Determine the data mining goals: the five questions

Formulate the project objectives in business terms, then ask and refine sharp questions that are relevant, specific, and unambiguous. Data science is a process that uses numbers to answer such questions, and you typically use data science or machine learning to answer five types of questions:

#QuestionTask family
1How much or how many?Regression
2Which category?Classification
3Which group?Clustering
4Is this unusual?Anomaly detection
5Which option should be taken?Recommendation
a sharp business question How much / how many? Regression answer: a real number Which category? Classification answer: a label Which group? Clustering answer: a partition, no labels Is this unusual? Anomaly detection answer: a score, few positives Which option? Recommendation answer: a ranked list
Plate 2.2 — The translation table of the phase. A business question becomes a data mining goal when you can name which of the five families it belongs to — because the family fixes the shape of the answer, and therefore the metric.

Widget — classify the question

Read the question and pick the family. The questions are the ones used across the decks.

6. SMART metrics

The metrics must be SMART:

LetterMeaningQuestion it forces you to answer
SSpecificWhich population, which outcome, which period?
MMeasurableWith which number, from which table?
AAchievableIs there enough data and enough signal for this?
RRelevantDoes it relate to at least one business objective?
TTime-boundBy when, and over which window is it measured?

Two consequences stated on the slide:

Careful

"Predict the level of damage with high accuracy" — the metric proposed in the brown spot case — fails S, M and T. High compared to what? Accuracy measured on which held-out fields? Within which season? The lecturer asks the class exactly this: how would you define accuracy? Answering that question before collecting the data is the whole point of the phase.

7. Produce a project plan (and where the effort goes)

Describe the intended plan for achieving the goals, including:

The generally accepted industry timeline standards (Shearer 2000) are the single most quoted numbers of this course:

PhaseShare of time and effort
Data Preparation50 to 70 percent
Data Understanding20 to 30 percent
Modeling10 to 20 percent
Evaluation10 to 20 percent
Business Understanding10 to 20 percent
Deployment planning5 to 10 percent
20% 40% 60% 80% SHARE OF TIME AND EFFORT (SHEARER 2000) Data Preparation 50-70% Data Understanding 20-30% Modeling 10-20% Evaluation 10-20% Business Understanding 10-20% Deployment planning 5-10% pale bar = lower bound of the range, solid bar = the rest up to the upper bound
Plate 2.3 — Where a data mining project actually spends itself. Preparing the data costs more than everything else together; modeling — the part everyone wants to do — is a tenth to a fifth of the work. Chapters 5 and 6 of this site are long for the same reason.

Widget — plan your own budget

Distribute 100 points of effort and compare your plan with the industry standard. The widget flags any phase outside its published range.

8. Interdisciplinarity

Identifying and delving into the problem to solve is an interdisciplinary phase. Even if you have good knowledge in the fields of computer and data science, you may still miss the domain knowledge that is fundamental to understanding and modeling the problem.

Key idea

This is why "create a glossary of business and data mining terms" appears in the assessment checklist, and why the next chapter is made of three case studies from three unrelated domains — pear orchards, urban mobility, stock indices and endurance sport. In each one, the hard part is not the algorithm: it is knowing what a staypoint, a market cap weight or a normalized power means before you compute it.

The following chapter takes exactly this route: three complete business-understanding exercises where the domain knowledge is the deliverable.

Check your understanding

What are the four focuses of the business understanding phase?

(1) Determining the business objectives; (2) assessing the context of the analysis; (3) determining and translating objectives into data mining goals; (4) producing the project plan. Its purpose is to understand which data should later be analyzed, and how.

Tell the brown spot story and say where it went wrong.

A company wanted to track the spreading of the brown spot of European pear (Stemphylium vesicarium) in Emilia-Romagna and to find environmental factors related to the spreading and to the level of damage. Data was collected with a paper questionnaire of about 100 questions, carried around the region by one employee, producing about 100 questionnaires. The analysts could not apply ML to data on paper, so a Google Form was built and digitalization took about a month. Then came four issues: answers inconsistent with the provided options; open questions; untrustworthy answers on legally constrained pesticides and fertilizers; and a dataset of 100 features and 100 rows. The project was missing a proper plan (business + data understanding) and failed: even applying ML, the results would not be statistically relevant.

What does "do not produce the right answers to the wrong questions" mean in practice?

That a technically flawless analysis can still be worthless if the objective was not the client's true goal. The countermeasure listed on the slide is procedural: uncover the primary business objective and the related questions, beware of unattainable goals, and make sure that each success criterion relates to at least one specified business objective.

List the five types of question data science answers, and their task families.

1. How much or how many? → regression. 2. Which category? → classification. 3. Which group? → clustering. 4. Is this unusual? → anomaly detection. 5. Which option should be taken? → recommendation.

What does SMART stand for, and what happens if a business goal cannot be made SMART?

Specific, Measurable, Achievable, Relevant, Time-bound. If the business goal cannot be effectively translated into a data mining goal, it may be wise to consider redefining the problem. Success can be defined as reducing lost customers by 10%, or as a better understanding of the customer base, or as achieving a certain level of predictive accuracy.

Quote the industry timeline standards for a data mining project.

50 to 70 percent of the time and effort goes into Data Preparation; 20 to 30 percent into Data Understanding; only 10 to 20 percent into each of Modeling, Evaluation and Business Understanding; and 5 to 10 percent into Deployment planning (Shearer 2000).

In "assess the situation", what has to be checked about the data?

Whether you have data relevant to the question: measures of the target and features related to the target, and whether those measures are accurate. An existing system might not have the data needed, in which case you find external sources or update the systems to collect new data. The example given is that a minimum number of customers over age 50 is necessary to answer the business question. The assessment also lists risks, assumptions, a glossary of business and data mining terms, and a cost-benefit analysis.

Why does the deck insist on a glossary of business and data mining terms?

Because the phase is interdisciplinary: the data scientist typically lacks the domain knowledge that is fundamental to understanding and modeling the problem, and the domain experts lack the data mining vocabulary. In an interdisciplinary team a shared glossary is described as essential.

What three things go into the project plan?

Specific steps with a proposed timeline, an assessment of potential risks, and an initial assessment of the tools and techniques needed to support the project.

Reformulate "predict the level of damage with high accuracy" as a SMART goal.

You need to fix, at least: the population (pear fields in Emilia-Romagna with the given cultivars), the target encoding (Level of Damage as an ordinal Low/Medium/High), the metric and the evaluation protocol (accuracy measured on fields never used for training), a threshold that counts as success, and the window (one growing season). The deck does not hand you a reformulation — it hands you the question "how would you define accuracy?", which is the exam question.

Why does the "one employee with paper questionnaires" method fail the assess-the-situation test even before the data arrives?

Because assessing the situation asks whether an accurate measure of target and features is available, and whether the existing system can produce it. A paper instrument produces answers inconsistent with the provided options and free text from open questions, i.e. no reliable encoding; a single employee travelling the region caps the number of rows at about a hundred, i.e. no statistical relevance for 100 attributes; and questions about legally constrained pesticides invite untrustworthy answers, i.e. a systematic distortion — a bias, in the vocabulary of Chapter 4.