# Scheduler Performance Milestone Ship Summary

Date: 2026-05-17

## Commit Ladder

| Step | Commit | Summary |
|---|---|---|
| Step 1 | `5777a6f` | Full-cycle profiling instrumentation and diagnosis. |
| Step 2 | `50dcfb2` | Named scheduler cycles plus Windows Task Scheduler split. |
| Step 3 | This commit | Verification and ship summary. |

## Diagnosis Recap

The monolithic scheduler was not failing because tasks were broken. It was failing because one sequential forced cycle exceeded common shell / Windows Task Scheduler timeout windows.

Step 1 forced-cycle findings:

- 50 task subprocesses completed successfully.
- Task failures: 0.
- Task-level timeouts: 0.
- Sequential task wall-clock sum: 1,225.4s.
- End-to-end task span: 20m 25.8s.
- The prior 5-minute timeout was therefore guaranteed to produce false scheduler failures.

Top bottlenecks from the monolithic run:

| Rank | Task | Seconds | Diagnosis |
|---:|---|---:|---|
| 1 | `run_season_projection_pipeline` | 485.6 | Projection recompute; pre-existing slow path. |
| 2 | `run_daily_mithrandir` | 310.5 | Card/postgame pipeline; not website-critical. |
| 3 | `fetch_statcast_seasonal_current` | 84.3 | Network/API + parquet write. |
| 4 | `run_home_run_daily_production` | 69.7 | Existing HR production/matchups work. |
| 5 | `run_strikeout_edge_refresh` | 48.2 | K board production and market joins. |
| 6 | `compute_sample_flags` | 38.1 | MLB career sample-size computation. |
| 7 | `fetch_daily_odds_fanduel` | 35.5 | Network/API. |
| 8 | `run_total_base_daily_production` | 25.2 | TB board production. |

## Fix Strategy

The scheduler now supports named cycles via `--cycle`:

- `morning_core`
- `calibration`
- `betting_market`
- `cards_postgame`

This was chosen over parallel execution because it solves the immediate timeout problem with much lower risk. Parallel execution is still possible later, but today it would introduce shared-resource questions around SQLite writes, sportsbook/API rate limits, and generated-artifact ordering.

The website-critical refresh path is now separated from slower maintenance and card generation work. `run_daily_mithrandir` is isolated in `cards_postgame`, so card generation can continue for Twitter/social use without blocking current site data.

Cycle documentation lives at:

- `docs/codex_context/scheduler_cycle_architecture.md`

## Windows Task Scheduler

Four active Windows scheduled tasks were created, all pointing at `C:\Projects\mithrandir-metrics`:

| Task | Time | Command cycle |
|---|---:|---|
| `Mithrandir Metrics Morning Core` | 06:00 ET | `--cycle morning_core` |
| `Mithrandir Metrics Calibration` | 07:30 ET | `--cycle calibration` |
| `Mithrandir Metrics Betting Market` | 09:00 ET | `--cycle betting_market` |
| `Mithrandir Metrics Cards Postgame` | 23:30 ET | `--cycle cards_postgame` |

The old monolithic `Mithrandir Metrics Scheduler Cycle` task was disabled to avoid duplicate work.

Carried operational note: older disabled OneDrive-pointed tasks may still appear in Windows Task Scheduler. They are disabled, but the user should delete them manually in the Task Scheduler GUI when convenient to reduce confusion.

## Before / After Timing

Before:

- Monolithic full forced cycle: 20m 25.8s.
- Website-critical and postgame/card work shared the same sequential timeout window.

After forced named-cycle verification on 2026-05-17:

| Cycle | Tasks | Wall-clock | Status | Slowest task |
|---|---:|---:|---|---|
| `morning_core` | 16 | 128.4s | success | `fetch_statcast_seasonal_current` 56.6s |
| `calibration` | 17 | 389.8s | success | `run_season_projection_pipeline` 315.7s |
| `betting_market` | 16 | 163.4s | success | `run_home_run_daily_production` 46.1s |
| `cards_postgame` | 1 | 572.5s | success | `run_daily_mithrandir` 572.2s |

All cycles completed under 10 minutes in verification. `cards_postgame` is close to the ceiling but is no longer on the website-critical path.

## Verification

Commands run:

```powershell
$env:PYTHONPATH='src'
python scripts\ops\schedule.py --once --force --cycle morning_core --profile-cycle --skip-health-check --task-timeout-minutes 15
python scripts\ops\schedule.py --once --force --cycle calibration --profile-cycle --skip-health-check --task-timeout-minutes 15
python scripts\ops\schedule.py --once --force --cycle betting_market --profile-cycle --skip-health-check --task-timeout-minutes 15
python scripts\ops\schedule.py --once --force --cycle cards_postgame --profile-cycle --skip-health-check --task-timeout-minutes 15
python scripts\shared\profile_web_routes.py
python -m unittest discover tests
```

Results:

- Named cycles: all successful.
- Tests: 87 passed.
- OneDrive runtime-path scan: zero hits in Python/JSON runtime files.
- Chalk discipline: template-level chalk remains isolated to `daily.html`; shared CSS token definitions remain in the design-system stylesheets.

Route timing smoke test:

| Route | Status | Seconds |
|---|---:|---:|
| `/` | 200 | 2.331 |
| `/daily` | 200 | 0.497 |
| `/projections` | 200 | 1.179 |
| `/leaderboards` | 200 | 0.070 |
| `/methods` | 200 | 3.051 |
| `/betting/consensus` | 200 | 0.787 |
| `/design-system` | 200 | 0.017 |

## Carried Debts

- `run_daily_mithrandir` remains heavy at 572.2s in the latest verification. It is isolated from site refresh now, but still close to the 10-minute target.
- `run_season_projection_pipeline` remains the largest maintenance task at 315.7s. It fits inside `calibration`, but it is the obvious future optimization target.
- `psutil` is not installed locally, so peak memory profiling fields are currently unavailable. Installing it would make the scheduler profiler more useful.
- Parallel execution remains deferred. It may be useful later for independent recalibration or ingestion groups, but cycle splitting solved the current issue with less risk.
- Older disabled OneDrive scheduled tasks should be manually deleted from Windows Task Scheduler when convenient.

## Explicit Non-Goals

- No data-model changes.
- No web template redesigns.
- No new ML training logic.
- No parallel scheduler execution yet.
- No push to GitHub from this milestone without user approval.