# Stratum tutorial datasets

This folder holds the **generator scripts** for the datasets used in the Stratum
video/blog series. Small and medium CSVs are written to the parent `datasets/`
directory; the very large one is generated on demand (not committed).

All generators are **deterministic** (fixed seed) so screenshots stay reproducible.

## Synthetic datasets (generated here)

| Script | Output | Used in lessons | Notes |
|---|---|---|---|
| *(prebuilt)* | `housing_market.csv` | 1, 2, 3, 5, 6, 11, 12, 13, 17, 23 | 1,500 listings × 15 columns. Copied from the app's `TestData/`. |
| `gen_retail.py` | `retail_sales.csv` + `stores.csv` | 4, 18 | Two related tables sharing `StoreID`; 24 months of seasonal sales. |
| `gen_process.py` | `process_measurements.csv` | 7, 8, 9, 15, 16, 19, 20, 21 | ~2,000-row semiconductor process log (factors, measurements, defect counts, time-ordered with drift). |
| `gen_clinical.py` | `clinical_trial.csv` | 14 | ~220-patient trial (Treatment/Control, pre/post scores) for two-sample tests. |
| `gen_survey.py` | `survey_responses.csv` | 10, 16 | ~1,600-respondent survey (Plan, Satisfaction, Region…) for categorical charts & contingency. |
| `gen_region_satisfaction.py` | `region_satisfaction.csv` | 10 | 8 regions × paired 2024/2025 satisfaction scores — wide, two-endpoint table for the dumbbell chart & dot plot. |
| `gen_iot_5m.py` | `iot_events_5m.csv` | 25 | ~5,000,000-row IoT event log for the scale demo. **Not committed** (~400 MB). |

### Running them

```bash
cd Website/datasets/generators
python3 gen_retail.py          # writes retail_sales.csv + stores.csv
python3 gen_iot_5m.py          # writes iot_events_5m.csv (~5M rows, ~400 MB)
python3 gen_iot_5m.py --rows 1000000   # smaller test run
```

No third-party packages required — standard library only.

## Public datasets (download separately, for credibility)

These are referenced in later lessons. Download them yourself and drop the CSV in
`datasets/`. Check each source's license/terms before redistributing.

| Dataset | Used in | Where to get it |
|---|---|---|
| **UCI Wine** (178 × 13, 3 cultivars) | 22, 24 | UCI ML Repository — "Wine" dataset (`wine.data`). Add a header row before importing. |
| **NYC Yellow Taxi** (millions of trips) | 25 (optional real alternative to the synthetic 5M set) | NYC TLC Trip Record Data — monthly Parquet/CSV. Convert one month to CSV if using the CSV importer. |