← Course home · Sprint 9A brief · Scrum Scoreboard

Sprint 9B: Did our change work?

Sprint 9B runs from Tue 10 Nov to the Sprint Review in class on Mon 16 Nov. Sprint 10 repeats the same test on a second week and ends at Demo Day, Mon 23 Nov.

Sprint 9B goal: by Monday 16 Nov, every team can show, with its own locked numbers, whether the change it wrote down in Sprint 9A made its prediction better, worse or no different, and what it will do about it.

The idea in one minute

In Sprint 9A you measured how wrong your past predictions were and wrote down what you would change. Now you test that change.

The obvious test is: “Last week we were 3% off. This week, with the change, we were 1% off. It worked.” That test tells you almost nothing. This week might simply have been an easier week to predict, and you would have been 1% off anyway.

Think of a farmer testing a new fertiliser. Comparing this year’s harvest with last year’s proves nothing, because the weather was different. Planting two halves of the same field, one with the fertiliser and one without, does.

So this week you lock two predictions for the same Friday:

All three face the same market on Friday 13 Nov. The change is the only difference between A and B, so if B lands closer to the close than A, the change gets the credit. The baseline asks the honest question every forecaster has to face: did all our analysis beat simply repeating the last close?

Where Sprint 9B fits: the Architecture Deep Dive

In the Architecture Deep Dive, Sprint 9A built Stage 6 (Delta Measurement): how far off you were. Sprint 9B completes Stage 7 (Prescriptions): a change, applied and tested. By Demo Day your system runs the whole loop: prediction → error → change → tested result.

Before you start: three things to fix

Sprint 9A results stay as they are on the scoreboard. These fixes are not re-scored, but Sprint 9B cannot work without them, because your vW10 errors are added to the same files.

• Every row in data/errors.csv uses the Friday close of its own week. Check your vW9 rows against Friday 6 Nov, not Thursday 5 Nov.
• Your old predictions table is at exactly data/old_predictions.csv, with a blank cell (not “DOWN”, not “NaN”) where a week had no number.
• Your sealed prediction files predictions/vW4.json to vW9.json are exactly as they were when locked. If any were edited, restore them from your Git history. From now on they are read-only; any later change to them is shown on the scoreboard.

Sprint 9B: your five milestones

Each milestone is one item in a fixed place by a fixed time, on main. The Scrum Scoreboard shows every team’s result and its actual work, exactly as in Sprint 9A.

Mon 16 Nov: Sprint Review in class, using the Sprint 9B presentation template.

1. Sprint 9B backlog (FigJam + GitHub)

One card for each of milestones 2–5, each with an owner and a one-line “done” statement, in your team’s FigJam section. Copy the same cards into docs/backlog-9b.md and merge it into main by the same deadline. FigJam is where you work; the GitHub copy is what records the time.

2. Your change, as rules

In Sprint 9A each market owner named a change. Now write it as one rule per market that someone outside your team could apply without asking you. One rule only per market, so the A/B test can show what that rule did.

docs/change.md

## SPX
Rule: New-method SPX = old-method SPX reduced by our median SPX error
over vW4–vW9 (in %), because we were too high in 4 of those 6 weeks.
Uses only: Tuesday 10 Nov close, data/errors.csv (vW4–vW9).
Comes from: our SPX rows in data/errors.csv, too high in vW6–vW9.
Would stop working if: the market turns and rises for several weeks.

Each rule must:

• Use only information available before Wed 11 Nov 23:59 SGT. At that moment the latest completed US close is Tuesday’s. Wednesday’s and Thursday’s closes do not exist yet, so a rule like “start from Thursday’s close” cannot be applied at the lock.
• Produce a number. “Consider recent trends” is not a rule; “subtract 1%” or “use Tuesday’s close if it is below the 20-day average” is.
• Point to the error rows it comes from, and say when it would stop working.

If a market keeps its old method, write Rule: no change for it, and say why.

3. Two vW10 locks

Both files use exactly the Sprint 9A format: numbers only, no % signs, no actual closes. Both are locked by the same deadline, and the old-method file must not borrow anything from the new rule.

predictions/vW10_old.json                predictions/vW10.json
{                                        {
  "week": "vW10",                          "week": "vW10",
  "target_date": "2026-11-13",             "target_date": "2026-11-13",
  "SPX": 7400,                             "SPX": 7330,
  "NDX": 27100,                            "NDX": 26800,
  "GOLD": 4080,                            "GOLD": 4080,
  "TNX": 4.88                              "TNX": 4.84
}                                        }

Example numbers only. Here GOLD is the same in both files because that market’s rule was “no change”.

4. A/B errors and chart

Extend engine/check_error.py so that data/errors.csv gets three vW10 rows per market, one for each method, with a method column:

week,target_date,instrument,method,predicted,actual,error,unit
vW10,2026-11-13,SPX,old,7400,[Fri close],[..],pct
vW10,2026-11-13,SPX,new,7330,[Fri close],[..],pct
vW10,2026-11-13,SPX,baseline,[Tue close],[Fri close],[..],pct

Same rules as Sprint 9A: actual closes from Yahoo Finance (^GSPC, ^NDX, GCZ26.CMX, ^TNX), error = (predicted − actual) ÷ actual × 100 for SPX, NDX and GOLD, and in basis points for TNX. Rows vW4–vW9 have old in the method column.

On ui/index.html, add one chart per market showing the three vW10 errors side by side (old, new, baseline) with a zero line, next to your Sprint 9A error charts.

5. Did it work?

In docs/did_it_work.md, one short paragraph per market (SPX, NDX, GOLD, TNX), each written by that market’s owner. Each paragraph must:

• quote the three vW10 error rows from data/errors.csv;
• say whether the new method beat the old one, and whether either beat the Tuesday-close baseline;
• give a verdict for Sprint 10: keep the rule, adjust it once (say exactly how), or drop it.

An honest “the change did not help” is a good result. The point is that your team can now tell.

How results are shown

Exactly as in Sprint 9A. Only main counts, at exactly the paths above, and arrival times come from GitHub’s own server record. Each milestone shows one result: ✅ On time, ⚠️ Late (with the time it arrived), or ❌ No submission. A result is fixed at its deadline and is not changed afterwards. The scoreboard updates shortly after each deadline: early Wed 11 Nov (milestone 1), early Thu 12 Nov (milestones 2–3) and early Sun 15 Nov (milestones 4–5). Your numbers appear only after the Wednesday lock has passed.

Path checklist (check on GitHub before each deadline)

Open each file on GitHub, on the main branch, before the deadline. A file in the right folder with the wrong name, or on another branch, does not count.

Who does what

After Sprint 9B

Sprint 10 (16–23 Nov) repeats the same A/B test on vW11, using your keep/adjust/drop verdict, so Demo Day on Mon 23 Nov shows two weeks of results for every market.