Bootstrap uncertainty in real measurements
Resample an observed Iris group, compare percentile intervals and standard-error formulas, and state the sampling assumptions.
Goal and how to run
This lesson assumes basic Python knowledge. Read each method and caption before changing parameters. AI exercises are optional; no AI account is needed for the core workflow. Data lives in notebook metadata, so there is no companion data download. Keep the notebook saved in the active working folder.
Your goal is to understand the research question, execute transparent calculations, check the results, and state the limits of the conclusion.
Setup and parameters
from pathlib import Path
import base64, gzip, hashlib, json, platform
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from IPython.display import display, HTML
from IPython.display import display as _display
def display(value):
if isinstance(value,pd.DataFrame):
table=value.to_html(max_rows=12,escape=True)
style="<style>.vibeit-readable-table{max-width:100%;overflow-x:auto}.vibeit-readable-table table{width:max-content;max-width:none;border-collapse:collapse}.vibeit-readable-table td,.vibeit-readable-table th{white-space:nowrap!important;word-break:normal!important;overflow-wrap:normal!important}</style>"
return _display(HTML(style+'<div class="vibeit-readable-table">'+table+'</div>'))
return _display(value)
RECIPE_ID='ma05-bootstrap'
OUTPUT_DIR=Path.cwd()/"results"/RECIPE_ID
plt.rcParams.update({"figure.dpi":120,"font.size":10,"axes.spines.top":False,"axes.spines.right":False,
"axes.prop_cycle":plt.cycler(color=["#18785f","#416aa6","#bc6541","#9974a5"])})
print("Python:",platform.python_version(),"; NumPy:",np.__version__,"; Pandas:",pd.__version__)
print("Working folder:",Path.cwd())
print("Core data mode: embedded snapshot; no network requests")
Python: 3.13.13 ; NumPy: 2.5.3 ; Pandas: 3.0.6 Working folder: /Users/zhiluo/Developer/vibeit-cookbook/cookbook/downloads Core data mode: embedded snapshot; no network requests
Load the embedded snapshot
def load_snapshot(recipe_id, expected_sha256, snapshot_key):
"""Read embedded data from this notebook, including a renamed copy."""
for path in sorted(Path.cwd().glob("*.ipynb")):
try:
notebook = json.loads(path.read_text(encoding="utf-8"))
metadata = notebook.get("metadata", {}).get("vibeit_cookbook", {})
blob = metadata.get("snapshots", {}).get(snapshot_key)
if metadata.get("recipe_id") != recipe_id or not blob:
continue
if blob.get("sha256") != expected_sha256:
continue
raw = gzip.decompress(base64.b64decode(blob["gzip_base64"], validate=True))
if hashlib.sha256(raw).hexdigest() != expected_sha256:
raise ValueError("Embedded snapshot checksum mismatch: " + snapshot_key)
return json.loads(raw)
except (OSError, json.JSONDecodeError):
continue
raise FileNotFoundError(
"Save/open the complete .ipynb in the active working folder. "
"The embedded snapshot could not be located for " + recipe_id)
SNAPSHOT_HASHES={'iris': 'cb2512f34911979eef3a42b7a9e6fbc6ca566fd2c0eaa7bd86daefd7ac1cf5ad'}
def snapshot(key):
return load_snapshot(RECIPE_ID,SNAPSHOT_HASHES[key],key)
print("Embedded snapshots:",list(SNAPSHOT_HASHES))
Embedded snapshots: ['iris']
Read the method functions
def audit_numeric_table(frame,columns):
"""Audit observed rows; never silently impute or remove them."""
if frame.empty or frame.index.has_duplicates:
raise ValueError('Expected a nonempty table with unique row IDs')
missing=[name for name in columns if name not in frame.columns]
if missing:raise ValueError('Missing required columns: '+str(missing))
values=frame[columns].to_numpy(dtype=float)
if not np.isfinite(values).all():raise ValueError('Non-finite measurements; inspect missing values explicitly')
return {'rows':len(frame),'features':len(columns),'finite':True,'duplicate_measurement_rows':int(frame[columns].duplicated().sum())}
print("Method ready: audit_numeric_table")
Method ready: audit_numeric_table
def bootstrap_mean(values,repetitions=2000,seed=42):
"""Ordinary iid bootstrap; exchangeability is a statistical assumption."""
values=np.asarray(values,dtype=float)
if values.ndim!=1 or len(values)<2 or not np.isfinite(values).all() or repetitions<2:
raise ValueError('Need finite observations and at least two resamples')
rng=np.random.default_rng(seed)
resamples=rng.choice(values,size=(repetitions,len(values)),replace=True)
means=resamples.mean(axis=1)
return dict(estimate=float(values.mean()),means=means,percentile_95=np.quantile(means,[.025,.975]),standard_error=float(means.std(ddof=1)))
print("Method ready: bootstrap_mean")
Method ready: bootstrap_mean
Data and model boundary
Analytic functions and Monte Carlo points are mathematical teaching models, not experimental measurements. When Iris observations are used, row exchangeability and the sampling design remain assumptions; convenience records alone do not establish population representativeness. Numerical discretization error is distinct from sampling uncertainty.
Steps
1. Define a within-group measurement target
Use the 50 source setosa petal lengths in centimetres. The target is their mean under an iid resampling model. Bootstrap uncertainty does not correct sampling bias, duplicated plants or an unrepresentative source collection.
iris=snapshot("iris")
frame=pd.DataFrame(iris["rows"],columns=iris["columns"])
features=list(frame.columns[:4])
audit=audit_numeric_table(frame,features)
assert len(frame)==150 and frame["species"].nunique()==3
assert (frame[features]>0).all().all()
print("Audit:",audit,"; measurement unit: cm")
print("Source variant:",iris["variant"])
display(frame.head(8))
observed=frame.loc[frame.species.eq("Iris-setosa"),"petal_length_cm"].to_numpy(float)
assert len(observed)==50
bootstrap=bootstrap_mean(observed,repetitions=2000,seed=42)
print("Observed mean (cm):",bootstrap["estimate"],"; percentile interval:",bootstrap["percentile_95"])
fig,ax=plt.subplots(figsize=(7,4));ax.hist(observed,bins=np.arange(1,2.11,.1),edgecolor="white")
ax.set(xlabel="Observed petal length (cm)",ylabel="Source observations",title="Empirical group measurements");plt.tight_layout();plt.show()
Audit: {'rows': 150, 'features': 4, 'finite': True, 'duplicate_measurement_rows': 3} ; measurement unit: cm
Source variant: Original UCI iris.data; no silent correction of the two documented transcription discrepancies
| sepal_length_cm | sepal_width_cm | petal_length_cm | petal_width_cm | species | |
|---|---|---|---|---|---|
| 0 | 5.1 | 3.5 | 1.4 | 0.2 | Iris-setosa |
| 1 | 4.9 | 3.0 | 1.4 | 0.2 | Iris-setosa |
| 2 | 4.7 | 3.2 | 1.3 | 0.2 | Iris-setosa |
| 3 | 4.6 | 3.1 | 1.5 | 0.2 | Iris-setosa |
| 4 | 5.0 | 3.6 | 1.4 | 0.2 | Iris-setosa |
| 5 | 5.4 | 3.9 | 1.7 | 0.4 | Iris-setosa |
| 6 | 4.6 | 3.4 | 1.4 | 0.3 | Iris-setosa |
| 7 | 5.0 | 3.4 | 1.5 | 0.2 | Iris-setosa |
Observed mean (cm): 1.464 ; percentile interval: [1.416 1.51 ]
2. Read the resampled mean distribution
Every bootstrap sample has 50 draws with replacement from the same 50 measurements. The histogram shows 2000 means, not 2000 newly measured plants. The shaded range marks percentile endpoints, conditional on this resampling model.
fig,ax=plt.subplots(figsize=(7,4));ax.hist(bootstrap["means"],bins=35,edgecolor="white")
ax.axvspan(*bootstrap["percentile_95"],alpha=.2,color="orange",label="Percentile 95% range")
ax.axvline(bootstrap["estimate"],color="black",linestyle="--",label="Observed mean")
ax.set(xlabel="Bootstrap mean (cm)",ylabel="Resamples",title="Conditional bootstrap mean distribution");ax.legend();plt.tight_layout();plt.show()
3. Inspect dependence on resample count
Resampling Monte Carlo error changes as the number of resamples grows, while the original 50-plant dataset stays fixed. More resamples improve numerical stability of this conditional distribution; they do not enlarge the observational sample.
counts=[100,300,1000,2000];intervals=np.array([np.quantile(bootstrap["means"][:n],[.025,.975]) for n in counts])
fig,ax=plt.subplots(figsize=(7,4));ax.plot(counts,intervals[:,0],"o-",label="2.5th percentile");ax.plot(counts,intervals[:,1],"o-",label="97.5th percentile")
ax.set(xlabel="Resamples used",ylabel="Percentile endpoint (cm)",title="Monte Carlo stability with fixed source data");ax.legend();plt.tight_layout();plt.show()
4. Check resampling variability and export
For the empirical distribution, the theoretical bootstrap standard error of the mean is population-SD/sqrt(n). Compare that with the observed resample SD, allowing finite-resampling noise. The textbook sample-SD/sqrt(n) is a different finite-n estimate.
empirical_se=observed.std(ddof=0)/np.sqrt(len(observed));sample_se=observed.std(ddof=1)/np.sqrt(len(observed))
assert np.isclose(bootstrap["standard_error"],empirical_se,rtol=.08)
assert bootstrap["percentile_95"][0]<=bootstrap["percentile_95"][1]
print("Empirical-distribution SE:",empirical_se,"; sample-SD SE:",sample_se,"; resample SE:",bootstrap["standard_error"])
OUTPUT_DIR.mkdir(parents=True,exist_ok=True)
pd.DataFrame({"bootstrap_mean_cm":bootstrap["means"]}).to_csv(OUTPUT_DIR/"bootstrap-means.csv",index=False)
(OUTPUT_DIR/"bootstrap-summary.json").write_text(json.dumps({"source_group":"Iris-setosa","n":50,"resamples":2000,"seed":42,"mean_cm":bootstrap["estimate"],"percentile_95_cm":bootstrap["percentile_95"].tolist(),"empirical_distribution_SE_cm":empirical_se,"resample_SE_cm":bootstrap["standard_error"],"assumption":"iid exchangeable observations; source sampling bias not corrected"},indent=2))
print("Bootstrap variance check passed; exports:",OUTPUT_DIR)
Empirical-distribution SE: 0.024291562321102363 ; sample-SD SE: 0.02453818348981016 ; resample SE: 0.023742351476355786 Bootstrap variance check passed; exports: /Users/zhiluo/Developer/vibeit-cookbook/cookbook/downloads/results/ma05-bootstrap
Exercises and reference explanation
- Why are resampled means not new plants?
- Why use population SD for the empirical bootstrap variance identity?
Reference explanation. Resampling reuses the observed empirical distribution. Its variance uses denominator n, while a sample estimate conventionally uses n−1. Neither removes sampling bias.
Keep a copy before changing parameters. Compare old and new figures and record which inputs, calculations and conclusions changed.
Optional AI coding exercises
Prompt 1
Using NumPy/Pandas/Matplotlib, repeat the bootstrap for another declared species group and retain cm units and source assumptions.
Prompt 2
Using the same packages, plot interval endpoints across three independent seeds without claiming extra observed sample size.
Human acceptance. Keep n=50 fixed, distinguish resample count from observations and compare empirical-distribution SE.
Review the edited source, then manually run affected cells and subsequent checks. Ask the assistant to explain the purpose, packages and data provenance of its changes, and reconcile that explanation with actual output.
Optional online extension
RUN_ONLINE_UPDATE=False
if RUN_ONLINE_UPDATE:
import requests
try:
response=requests.get('https://archive.ics.uci.edu/dataset/53/iris',timeout=20)
response.raise_for_status()
print("Source:",response.url)
print(response.text[:700])
except requests.RequestException as error:
print("Online extension unavailable:",type(error).__name__,str(error)[:160])
else:
print("Online extension skipped; embedded core data unchanged")
Online extension skipped; embedded core data unchanged
Sources, snapshots and next steps
- UCI iris — CC BY 4.0; credit UCI and original dataset authors; retrieved 2026-10-09. SHA-256
d11fe30213d36434a0879aab7cb00ce3c812eb7ba2495874438abff7b7b762e9. Extract iris.data from original download; preserve all rows and declared units; no imputation. Fisher, R. (1936). Iris. UCI Machine Learning Repository. DOI: 10.24432/C56C76.
Next steps. Transfer these methods to your own public or authorized data while retaining input checks, recorded parameters and result validation. Document the new experimental design and method limits; this example does not validate a new experiment. Full data licenses and generation instructions accompany the cookbook.