Part II — Understanding the problem and the data · Chapter 3

Three case studies: mobility, stocks, sport KPIs

~35 min read5 interactive widgets3 plates

In this chapter

  1. Personal gazetteer: the question behind La Città Intorno
  2. Location intelligence and mobility data
  3. Staypoints, trajectories and flows
  4. Measuring the attractiveness of a neighbourhood
  5. Exploiting trajectory data
  6. Stocks: what is the weight of a company in an index?
  7. Sport KPIs: training load, normalized power, intensity factor
  8. Fitness, fatigue and performance readiness
  9. Check your understanding

Three business-understanding exercises in three unrelated domains. In each one the algorithm is almost an afterthought: the work is turning a vague request — who are these people, how big is this company, am I ready to race — into a quantity you can compute from data you can actually get.

1. Personal gazetteer: the question behind La Città Intorno

Lacittàintorno is a programme for urban regeneration promoted by Fondazione Cariplo:

The business question is one line long: what is the profile of users in these neighbourhoods? Translating it is the whole exercise — "profile" is not a column of any table.

2. Location intelligence and mobility data

More than 80% of companies take decisions on the basis of information characterized by a spatial component. Two families of spatial data:

KindDefinitionCharacter
Geo-referenced dataData that contain a spatial positionTypically static
Trajectory dataData describing objects moving in the environmentDynamic, ordered in time

The main providers of trajectory data are: apps collecting data from GPS; telcos that trace connections to Base Transceiver Stations; fleets of trucks, cars and ships; and others.

Location Intelligence is a set of tools that allow a geographic dimension to be integrated within a BI platform. The goal is to increase the monitoring ability and the capability of understanding business events; location intelligence supports data visualization and interaction with maps in BI contexts. The slides use Google Timeline (which shows both single locations and trajectories) and OpenStreetMap (Forlì, London) as the everyday examples.

3. Staypoints, trajectories and flows

A personal gazetteer is a personalized, digital record of significant locations — stay points — for an individual, similar to a geographical fingerprint. Staypoints are places of interest for a user. The design question of the case study is: how can we extract staypoints with a given semantics — frequented places, home, work?

The two lenses

Trajectories describe mobility behaviors and patterns.

Use case: understand where people going to an event come from.

Frequent trajectories are an end-to-end cluster of trajectories that are:

  • compact enough,
  • frequent enough,
  • long enough.

All three conditions are thresholds you must set — which is a business-understanding decision, not a technical one.

Staypoints identify places of interest for a user, each with an explicit semantic definition:

SemanticDefinition used in the case study
HomeThe place where a user frequently stops between 1 and 5
WorkThe place where a user frequently stops between 9 and 17
AttendA place where the user stands at least 20 minutes and where at least one commercial activity is situated

These three definitions are the reason the analysis is possible at all: they turn "profile of the users" into three computable predicates over anonymous trajectory data.

A flow is defined by a sequence of overlapping trajectories that are not necessarily end-to-end compact.

Use case: identify crowded routes. The slides show top flows overall, and separately top morning flows and top evening flows — the same city, two different pictures, because commuting is directional.

Compare with frequent trajectories: those must be compact end-to-end, a flow only needs the overlaps. A flow is the shape of the traffic; a frequent trajectory is a route many people travel in full.

FROM A RAW TRACE TO A PERSONAL GAZETTEER HOME stops between 1 and 5 WORK stops between 9 and 17 ATTEND ≥ 20 min + a commercial activity on the spot the dashed line is the trajectory: it describes behaviour, not places the circles are staypoints: places, each with a semantic
Plate 3.1 — The same raw GPS points read twice. As a trajectory they describe a mobility pattern; clustered into staypoints and filtered by the time-window and dwell rules they become Home, Work and Attend — the geographical fingerprint of one user.
Key idea

Notice what has happened here: a fuzzy word ("profile") was replaced by three operational definitions written in terms of quantities the data actually contains (a time of day, a dwell time, the presence of a commercial activity). That substitution is the business understanding phase; everything after it is data preparation.

4. Measuring the attractiveness of a neighbourhood

Colouring maps by the LiveIn semantic (LiveIn vs WorksIn, LiveIn vs Attend, heatmaps of who lives in Adriano and works elsewhere and vice versa) gives qualitative results. The next question is quantitative: how can we measure the attractiveness of a neighbourhood?

Two measures are proposed (Francia, Gallinucci, and Golfarelli 2024):

Attractiveness(n)  = ( LiveIn(n) + WorkIn(n) + Frequent(n) ) / LiveIn(n)

Attractiveness'(n) = MovementPattern(n) / SUM over i of MovementPattern(i)

The first is a ratio local to the neighbourhood: how many people the place pulls in, per resident. The second normalizes over the whole city: what share of all movement patterns this neighbourhood accounts for. They answer different questions, and the published table shows they disagree.

NeighbourhoodDistinct usersWorkIn + FrequentLiveInAttractivenessAttractiveness′
Brera27792396440545%83%
Duomo46074203464906%83%
Buenos Aires - Venezia384026441484178%82%
XXII Marzo17901202681177%82%
Gallaratese1508929752124%5%
Lodi - Corvetto1627993795125%4%
Padova1628996753132%3%
Adriano63631638582%1%
Editor's note

If you recompute the fifth column from the third and fourth, you get exactly (WorkIn + Frequent) / LiveIn: Brera 2396/440 = 545%, Duomo 4203/464 = 906%, Adriano 316/385 = 82%. Adding LiveIn to the numerator, as the formula on the slide is typeset, would give 644% for Brera. The widget below computes both so you can see the difference; the values printed in the published table are the ones without LiveIn in the numerator.

Widget — recompute the attractiveness

For the exam

Read the two extremes out loud. Duomo attracts nine visitors for every resident, and accounts for a large share of the city movement: it is attractive on both measures. Adriano — one of the four target neighbourhoods of the programme — is below 100%: fewer people come in than live there, and it accounts for 1% of the movement patterns. That number, not a map colour, is what a 10-million-euro regeneration programme can be measured against.

5. Exploiting trajectory data

The catalogue of what this data supports, as listed in the deck:

Reality mining: the synthetic profile of a community

Ubiquitous user profiling means creating a synthetic profile of the community that transits at a point or place, starting from anonymous trajectory data. The examples given are the deliverable of such an analysis — statements a business can act on:

QuestionProfile extracted
Who are the customers of the shopping center?50% live within 5 km from the shopping center
20% attend the shopping center X too
On Saturday, 20% of customers go to the cinema after being at the shopping center
On Saturday, 30% of customers go to restaurant Y after being at the shopping center
Who comes on vacation to Cesenatico?In July, 70% of the customers come from Emilia-Romagna
27% of customers live in neighbourhoods with an average income above 30K €
In August, 25% of customers come from Milan
40% of the tourists have children
How do people using the Freccia Rossa Milan-Rome behave?30%, once in Rome, move less than 10 km from the station
20%, once in Rome, take the subway

The last application shown is operational rather than commercial: garbage collection paths for HERA, with overall statistics and the trips of a single truck — the same trajectory machinery applied to a fleet.

Careful

Every one of these profiles is derived from anonymous trajectory data, and yet a personal gazetteer is explicitly described as a geographical fingerprint: home is where you sleep, work is where you spend the day. Chapter 5 returns to this with the GDPR list of sensitive data and the AI Act risk classes — real-time remote biometric identification in public spaces and social scoring are banned, not merely regulated.

6. Stocks: what is the weight of a company in an index?

The Standard and Poor's 500 index tracks the stock performance of 500 of the largest companies in the United States. The vocabulary first:

The business question — how would you define the weight of a company in the index? — has two standard answers, and they are not equivalent.

Market cap weight (e.g., S&P 500)
  MarketCap(C)       = SharesOut(C) x StockPrice(C)
  MarketCapWeight(C) = MarketCap(C) / SUM over C' in I of MarketCap(C')

Price weight index (e.g., Dow Jones Industrial Average)
  PriceWeight(C)     = StockPrice(C) / SUM over C' in I of StockPrice(C')

The five companies given, with their published weights in the two indices:

TickerCloseShares outstandingPWI (%)Market Cap (%)
AMZN222.131.0515e+103.202534.53745
AAPL242.71.51158e+103.49917.12683
GS580.023.1391e+088.362370.353707
MSFT424.567.43488e+096.121056.13209
NVDA140.112.449e+102.020026.66582
THE SAME FIVE COMPANIES, TWO DEFINITIONS OF WEIGHT price weight (DJIA) market cap weight (S&P) 0% 2% 4% 6% 8% AMZN AAPL GS 8.36% vs 0.35% MSFT NVDA
Plate 3.2 — Goldman Sachs is the heaviest of the five in a price-weighted index (8.36%) and almost invisible in a capitalization-weighted one (0.35%), because a high share price says nothing about the size of a company. Microsoft, by coincidence, weighs nearly the same in both.

Widget — compute both weights

Change a closing price and watch the two rankings move independently. The percentages here are computed within this five-company sample; the two rightmost columns of the table above are the published weights within the full indices.

Key idea

Same data, two defensible definitions of the same word — weight — and completely different conclusions about which company matters. This is the business-understanding lesson of the stock case study: the metric is a modelling choice, and it must be made explicit before anything is computed. A price-weighted index is moved by an expensive share; a capitalization-weighted one is moved by a big company.

7. Sport KPIs: training load, normalized power, intensity factor

The third case study: we are preparing for a race and we need to create a proper plan. The risks of poor planning are overtraining, underperformance and increased risk of injury. How can we exploit data-driven planning?

Key Performance Indicators (KPIs) are measurable values that track performance and progress toward a goal. They let you objectively measure improvements, balance training intensity and recovery, and prevent overtraining or undertraining. The objective is to optimize performance readiness, achieved when fitness is high and fatigue is low: minimize fatigue (short-term stress from recent training) and maximize fitness (long-term training adaptations).

Training Load

Training Load (TL) is a measure of the stress placed on the body during exercise:

TL = Duration x Intensity        units: TSS (Training Stress Score)

Duration is easy. Intensity is the hard half — and it is measured, in cycling, with a power meter: power meters measure torque using strain gauges, which measure a rider's force on a given component (pedal, crank, etc.); this torque is later converted to power by multiplying it by that component's cadence (angular velocity).

Normalized Power

Normalized Power (NP) quantifies the physiological demands of a cycling activity with variable intensity:

NP = fourth_root( SUM over i of (P_30sec,i)^4 / n )
  1. Divide the ride into 30-second moving averages and calculate the average power for every 30-second interval — this smooths out short-term power fluctuations.
  2. Raise each 30-second average to the fourth power — this emphasizes higher power outputs, as intense efforts contribute disproportionately to physiological stress.
  3. Take the average of all fourth powers.
  4. Take the fourth root of the result.

Unlike average power, NP accounts for the variability in effort during a ride, reflecting how the body responds to the stress: high-intensity intervals are more stressful than steady-state efforts, even if the average power is the same. The eight intervals given in the deck:

TimeIntervalAvg PowerAvg HR
228528242130
231332103134
234527253135
23723297136
240428278139
243232104147
246428290145
249271107140

Given these 8 intervals: average power = 184, normalized power = 226.

Editor's note — the arithmetic, step by step

Average power: (242 + 103 + 253 + 97 + 278 + 104 + 290 + 107) / 8 = 1474 / 8 = 184.25 ≈ 184.
Normalized power: raise each value to the fourth power (2424 = 3 429 742 096; 1034 = 112 550 881; 2534 = 4 097 152 081; 974 = 88 529 281; 2784 = 5 972 816 656; 1044 = 116 985 856; 2904 = 7 072 810 000; 1074 = 131 079 601), sum = 21 021 666 452, divide by 8 = 2 627 708 306.5, take the fourth root = 226.4 ≈ 226. The 42-watt gap between 184 and 226 is exactly the cost of riding in surges instead of steadily.

From absolute to relative: the Intensity Factor

Is NP = 200 W a hard or an easy ride? An easy ride for Mathieu van der Poel, an impossible ride for most people. Normalized Power is an absolute measurement: absolute measurements provide information on volume, size, or magnitude, while relative measurements provide a comparative gauge of how well the work has been executed.

IF = NP / FTP

The Intensity Factor (IF) measures workout intensity relative to the Functional Threshold Power (FTP): the highest average power (in watts, measured with sensors) a cyclist can sustain for 60 minutes. Testing methods: a 20-minute time trial, where FTP = 95% of the average power; or ramp / step tests with smart trainers.

IF rangeMeaning
IF < 0.65Easy recovery
IF ∈ [0.65, 0.85]Moderate endurance
IF > 0.85High-intensity intervals

Use IF to adjust the training load: too high means risk of overtraining, too low means insufficient stimulus for improvement.

Widget — normalized power and intensity factor

The eight intervals from the deck are loaded. Edit a power value, or flatten the ride, and watch NP separate from the plain average.

8. Fitness, fatigue and performance readiness

Training load is defined. Fitness and fatigue are harder, and the deck lists why:

The trick is to separate them by time scale — the same quantity, averaged over a short and a long window:

KPIDefinitionReads as
ATL — Acute Training LoadSum of daily TL over 7 days, divided by 7Short-term training stress: fatigue
CTL — Chronic Training LoadSum of daily TL over 42 days, divided by 42Long-term training stress: fitness
TSB — Training Stress BalanceTSB = CTL − ATL = Fitness − FatiguePositive: readiness. Negative: fatigue

Training adjustments follow mechanically: if CTL is low, focus on increasing training volume or intensity; if ATL is high, incorporate rest days or low-intensity sessions.

Planning to optimize performance readiness

Monitor the key metrics — CSS (single training load), ATL (short-term fatigue: lower it during the taper to avoid carrying excessive stress), CTL (long-term fitness: maintain it high relative to past races), TSB (recovery and readiness: aim for a positive balance, e.g. +10 to +20 before race day). Adjust based on feedback: reduce intensity or volume if fatigue accumulates, and focus on recovery activities such as yoga or massage.

Tapering before the race: a gradual reduction in training volume to reduce fatigue while maintaining fitness — reduce training volume by 40–60% over 1–3 weeks, and maintain intensity to preserve fitness.

A TRAINING BLOCK, READ THROUGH THREE KPIs 0 time (days) CTL = fitness (42-day mean) ATL = fatigue (7-day mean) TSB = CTL - ATL TAPER volume -40 to -60% over 1-3 weeks, intensity kept race day TSB +10..+20 negative TSB: fatigue. Training works here, racing does not.
Plate 3.3 — Fitness rises slowly and falls slowly (42-day window); fatigue rises and falls fast (7-day window). During the taper, volume drops, ATL collapses, CTL barely moves, and TSB crosses into the positive band the plan aims for on race day.

Widget — build a training block

Add days and watch the two rolling means diverge. This is genuine arithmetic: ATL is the mean of the last 7 daily loads, CTL the mean of the last 42, TSB their difference.

For the exam

The transferable structure of this case study, and the reason it is in a data mining course: an unmeasurable business concept (am I ready?) is decomposed into one raw measurement (power from a strain gauge), one derived feature that respects the physics (NP, with its fourth power), one normalisation that makes it comparable across athletes (IF = NP/FTP), and two aggregations at different time scales whose difference is the answer (TSB = CTL − ATL). Feature engineering — Chapter 6 — is exactly this, done to a table.

Check your understanding

What is a personal gazetteer, and how are Home, Work and Attend defined?

A personal gazetteer is a personalized, digital record of significant locations (stay points) for an individual, similar to a geographical fingerprint. The three semantics: Home is the place where a user frequently stops between 1 and 5; Work is the place where a user frequently stops between 9 and 17; Attend is a place where the user stands at least 20 minutes and where at least one commercial activity is situated.

Distinguish trajectories, frequent trajectories and flows.

Trajectories describe mobility behaviors and patterns (use case: where do people going to an event come from). Frequent trajectories are an end-to-end cluster of trajectories that are compact enough, frequent enough and long enough. A flow is a sequence of overlapping trajectories that are not necessarily end-to-end compact (use case: identify crowded routes; the deck shows top morning and top evening flows separately).

Give the two definitions of attractiveness of a neighbourhood, and say what they measure.

Attractiveness(n) = (LiveIn + WorkIn + Frequent) / LiveIn — a local ratio: how many people the neighbourhood pulls in per resident. Attractiveness′(n) = MovementPattern(n) / SUM_i MovementPattern(i) — a share of the whole city. In the published table, Duomo reaches 906% on the first and 83% on the second, while Adriano is at 82% and 1%: below 100% means fewer people come in than live there.

What are geo-referenced data and trajectory data, and who provides the latter?

Geo-referenced data contain a spatial position and are typically static. Trajectory data describe objects moving in the environment. The main providers are apps collecting GPS data, telcos tracing connections to Base Transceiver Stations, and fleets of trucks, cars and ships. More than 80% of companies take decisions on the basis of information with a spatial component.

Define reverse geocoding and layer integration.

Reverse geocoding is the process by which an address or the name of a place is associated with a spatial datum (latitude, longitude). Layer integration is the process by which several information layers are superimposed and interrogated.

Write the market cap weight and the price weight of a company, and explain when they diverge.

MarketCap(C) = SharesOut(C) x StockPrice(C) and MarketCapWeight(C) = MarketCap(C) / SUM MarketCap(C') — used by the S&P 500. PriceWeight(C) = StockPrice(C) / SUM StockPrice(C') — used by the Dow Jones Industrial Average. They diverge whenever share price and company size are unrelated: GS, at 580.02 per share but only 3.1391e+08 shares outstanding, weighs 8.36% in the price-weighted index and 0.354% by capitalization.

Why is Normalized Power computed with a fourth power?

Because intense efforts contribute disproportionately to physiological stress. The four steps are: (1) split the ride into 30-second moving averages, to smooth out short-term fluctuations; (2) raise each 30-second average to the fourth power, to emphasize higher power outputs; (3) average all fourth powers; (4) take the fourth root. On the eight intervals of the deck, the plain average is 184 while NP is 226 — the same ride, weighted by how hard it actually was.

What is FTP, how is it tested, and what is the Intensity Factor?

FTP (Functional Threshold Power) is the highest average power a cyclist can sustain for 60 minutes. Testing methods: a 20-minute time trial, where FTP is 95% of the average power, or ramp/step tests with smart trainers. IF = NP / FTP makes an absolute wattage relative to the individual: below 0.65 is easy recovery, 0.65–0.85 moderate endurance, above 0.85 high-intensity intervals.

Define ATL, CTL and TSB, and explain why the two windows differ.

ATL = sum of daily TL over 7 days / 7: short-term training stress, i.e. fatigue. CTL = sum of daily TL over 42 days / 42: long-term training stress, i.e. fitness. TSB = CTL − ATL: positive indicates readiness, negative indicates fatigue. The windows differ because the same session has two effects on different time scales — yesterday's activity provokes fatigue but will increase fitness in the long run — so the only way to separate them from one signal is to average it twice.

What is a taper, quantitatively?

A gradual reduction of training volume to reduce fatigue while maintaining fitness: reduce volume by 40–60% over 1–3 weeks and maintain intensity to preserve fitness. The target is a positive TSB, e.g. +10 to +20, on race day. In terms of the KPIs: ATL falls quickly because its window is 7 days, CTL barely moves because its window is 42.

Why is quantifying fatigue harder than quantifying training load?

Three reasons given: subjectivity (perceived exertion varies by individual), external factors (sleep, nutrition and stress influence fatigue but are hard to measure precisely), and delayed effects (fatigue may appear days after intense training). Training load, by contrast, is a product of two measured quantities: duration and intensity, the latter from a power meter that converts strain-gauge torque into watts by multiplying by cadence.

What do all three case studies have in common, methodologically?

Each replaces a word with a formula. "Profile of the users" becomes Home/Work/Attend predicates over trajectories and an attractiveness ratio; "weight of a company" becomes either a price ratio or a capitalization ratio, with different winners; "am I ready to race" becomes TSB = CTL − ATL over a 42-day and a 7-day window. In all three the decisive work happens before any model is trained, and it requires domain knowledge — which is why business understanding is described as an interdisciplinary phase.