Reconcile empirical tail risk and block resampling
Define inverse-CDF VaR and fractional-tail ES for a hypothetical currency basket, then inspect conditional resampling variability.
Goal and how to run
This lesson assumes basic Python knowledge. Read each method and caption before changing parameters. AI exercises are optional; no AI account is needed for the core workflow. Data lives in notebook metadata, so there is no companion data download. Keep the notebook saved in the active working folder.
Your goal is to understand the research question, execute transparent calculations, check the results, and state the limits of the conclusion.
Setup and parameters
from pathlib import Path
import base64, gzip, hashlib, json, platform
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from IPython.display import display, HTML
from datetime import datetime
from IPython.display import display as _display
def display(value):
if isinstance(value,pd.DataFrame):
table=value.to_html(max_rows=12,escape=True)
style="<style>.vibeit-readable-table{max-width:100%;overflow-x:auto}.vibeit-readable-table table{width:max-content;max-width:none;border-collapse:collapse}.vibeit-readable-table td,.vibeit-readable-table th{white-space:nowrap!important;word-break:normal!important;overflow-wrap:normal!important}</style>"
return _display(HTML(style+'<div class="vibeit-readable-table">'+table+'</div>'))
return _display(value)
RECIPE_ID='ec06-tail-risk'
OUTPUT_DIR=Path.cwd()/"results"/RECIPE_ID
plt.rcParams.update({"figure.dpi":120,"font.size":10,"axes.spines.top":False,"axes.spines.right":False,
"axes.prop_cycle":plt.cycler(color=["#18785f","#416aa6","#bc6541","#9974a5"])})
print("Python:",platform.python_version(),"; NumPy:",np.__version__,"; Pandas:",pd.__version__)
print("Working folder:",Path.cwd())
print("Core data mode: embedded snapshot; no network requests")
Python: 3.13.13 ; NumPy: 2.5.3 ; Pandas: 3.0.6 Working folder: /Users/zhiluo/Developer/vibeit-cookbook/cookbook/downloads Core data mode: embedded snapshot; no network requests
Load the embedded snapshot
def load_snapshot(recipe_id, expected_sha256, snapshot_key):
"""Read embedded data from this notebook, including a renamed copy."""
for path in sorted(Path.cwd().glob("*.ipynb")):
try:
notebook = json.loads(path.read_text(encoding="utf-8"))
metadata = notebook.get("metadata", {}).get("vibeit_cookbook", {})
blob = metadata.get("snapshots", {}).get(snapshot_key)
if metadata.get("recipe_id") != recipe_id or not blob:
continue
if blob.get("sha256") != expected_sha256:
continue
raw = gzip.decompress(base64.b64decode(blob["gzip_base64"], validate=True))
if hashlib.sha256(raw).hexdigest() != expected_sha256:
raise ValueError("Embedded snapshot checksum mismatch: " + snapshot_key)
return json.loads(raw)
except (OSError, json.JSONDecodeError):
continue
raise FileNotFoundError(
"Save/open the complete .ipynb in the active working folder. "
"The embedded snapshot could not be located for " + recipe_id)
SNAPSHOT_HASHES={'ecb': '7afd344000313ec2b4f6fb2548894e2a4817dc3687c14f8de47937ebf7bba0ad'}
def snapshot(key):
return load_snapshot(RECIPE_ID,SNAPSHOT_HASHES[key],key)
print("Embedded snapshots:",list(SNAPSHOT_HASHES))
Embedded snapshots: ['ecb']
Read the method functions
def empirical_tail_risk(losses,confidence=.95):
"""Inverse empirical-CDF VaR and fractional upper-tail expected shortfall."""
losses=np.asarray(losses,dtype=float)
if losses.ndim!=1 or len(losses)<2 or not np.isfinite(losses).all() or not 0<confidence<1:
raise ValueError('Need finite losses and confidence strictly between zero and one')
ordered=np.sort(losses)[::-1];mass=(1-confidence)*len(ordered);full=int(np.floor(mass));fraction=mass-full
tail_sum=float(ordered[:full].sum())
if fraction>0:tail_sum+=fraction*ordered[full]
return dict(VaR=float(np.quantile(losses,confidence,method='inverted_cdf')),expected_shortfall=tail_sum/mass,tail_sample_mass=mass)
print("Method ready: empirical_tail_risk")
Method ready: empirical_tail_risk
def circular_block_resample(values,block_length=5,seed=42):
"""One circular block bootstrap draw; stationary resampling is assumed."""
values=np.asarray(values,dtype=float)
if values.ndim!=1 or len(values)<2 or not np.isfinite(values).all() or not isinstance(block_length,(int,np.integer)) or not 1<=block_length<=len(values):
raise ValueError('Invalid sequence or block length')
rng=np.random.default_rng(seed);count=int(np.ceil(len(values)/block_length))
starts=rng.integers(0,len(values),count)
indices=(starts[:,None]+np.arange(block_length)[None,:])%len(values)
return values[indices.ravel()[:len(values)]]
print("Method ready: circular_block_resample")
Method ready: circular_block_resample
def iso_datetimes(values):
"""Parse source ISO dates with the standard library; keep tables as ISO strings."""
return [datetime.fromisoformat(str(value)) for value in values]
print("Method ready: iso_datetimes")
Method ready: iso_datetimes
Data and model boundary
Historical price-only currency examples use hypothetical daily rebalancing, excluding interest/carry, fees, execution costs and tax. They do not establish future performance or an investment recommendation. Source: ECB; original reference rates are available free of charge at the ECB. Inversions, returns and risk statistics are derived course calculations, not ECB statistics.
Steps
1. Declare a hypothetical loss sequence
Loss is minus the equal-weight simple return of inverted currency prices. Negative loss represents a gain. The price-only basket does not include actual trades, interest/carry, fees or tax.
ecb=snapshot("ecb");rates=pd.DataFrame(ecb["rows"])
rates["date"]=rates["Date"].astype(str);rates=rates.set_index("date").sort_index()
currencies=ecb["currency_order"];original_dates=len(rates);rates=rates[currencies].dropna()
assert rates.index.is_unique and rates.index.is_monotonic_increasing and (rates>0).all().all()
prices=1/rates;returns=prices.pct_change(fill_method=None).iloc[1:]
print(ecb["source_notice"])
print("Quote:",ecb["quote"],"; common complete dates:",len(rates),"of",original_dates)
display(rates.head(6))
losses=-(returns.to_numpy()@np.full(4,.25));risk=empirical_tail_risk(losses,.95)
fig,ax=plt.subplots(figsize=(8,5));ax.hist(100*losses,bins=60,edgecolor="white")
ax.axvline(100*risk["VaR"],color="red",label="Empirical 95% VaR");ax.axvline(100*risk["expected_shortfall"],color="orange",label="Fractional-tail ES")
ax.set(xlabel="Hypothetical price-return loss (%)",ylabel="Intervals",title="Frozen-period empirical tail");ax.legend();plt.tight_layout();plt.show()
Source: European Central Bank. The original reference rates are available free from the ECB. Course returns, inversions and risk estimates are derived calculations, not ECB statistics. Quote: Foreign currency units per 1 EUR ; common complete dates: 1538 of 1538
| USD | GBP | JPY | CNY | |
|---|---|---|---|---|
| date | ||||
| 2019-01-02 | 1.1397 | 0.90165 | 124.28 | 7.8165 |
| 2019-01-03 | 1.1348 | 0.90312 | 122.21 | 7.8019 |
| 2019-01-04 | 1.1403 | 0.89988 | 123.20 | 7.8280 |
| 2019-01-07 | 1.1445 | 0.89720 | 123.90 | 7.8421 |
| 2019-01-08 | 1.1440 | 0.89743 | 124.46 | 7.8405 |
| 2019-01-09 | 1.1455 | 0.89913 | 124.70 | 7.8226 |
2. State the finite-sample conventions
VaR uses the inverse empirical-CDF order statistic. ES averages the worst (1−alpha)n sample mass, including a fractional next observation. A mean above an interpolated quantile can implement a different convention.
confidence=[.90,.95,.99]
risk_table=pd.DataFrame([{"confidence":level,**empirical_tail_risk(losses,level)} for level in confidence]);display(risk_table)
fig,ax=plt.subplots(figsize=(8,4));positions=np.arange(3)
ax.bar(positions-.18,100*risk_table.VaR,width=.36,label="VaR");ax.bar(positions+.18,100*risk_table.expected_shortfall,width=.36,label="Fractional-tail ES")
ax.set_xticks(positions,[f"{q:.0%}" for q in confidence]);ax.set(xlabel="Confidence level",ylabel="Historical loss (%)",title="Declared empirical tail conventions");ax.legend();plt.tight_layout();plt.show()
| confidence | VaR | expected_shortfall | tail_sample_mass | |
|---|---|---|---|---|
| 0 | 0.90 | 0.003717 | 0.005646 | 153.70 |
| 1 | 0.95 | 0.005020 | 0.006980 | 76.85 |
| 2 | 0.99 | 0.008298 | 0.010010 | 15.37 |
3. Inspect conditional circular-block variation
Five-observation circular blocks retain within-block order but assume stationary resampling. Wrapping the boundary is artificial, not a true adjacent date. The interval describes conditional historical variability, not a tomorrow-loss guarantee.
draws=[]
for seed in range(1000):
sampled=circular_block_resample(losses,block_length=5,seed=seed);value=empirical_tail_risk(sampled,.95)
draws.append([value["VaR"],value["expected_shortfall"]])
draws=np.asarray(draws);interval=np.quantile(draws[:,1],[.025,.975])
fig,ax=plt.subplots(figsize=(8,4));ax.hist(100*draws[:,1],bins=40,edgecolor="white")
ax.axvspan(*(100*interval),color="orange",alpha=.2,label="Conditional percentile range");ax.axvline(100*risk["expected_shortfall"],color="black",linestyle="--",label="Original ES")
ax.set(xlabel="Resampled historical ES (%)",ylabel="Block resamples",title="Conditional variability, not a future bound");ax.legend();plt.tight_layout();plt.show()
4. Check a hand-calculated fractional tail
For [0,1,2,3], alpha=.625 gives tail mass 1.5 and ES=(3+0.5·2)/1.5. Verify this convention, ES≥VaR and rejected confidence endpoints, then export full assumptions.
fixture=empirical_tail_risk(np.array([0,1,2,3]),.625)
assert np.isclose(fixture["expected_shortfall"],(3+.5*2)/1.5)
assert np.all(risk_table.expected_shortfall>=risk_table.VaR-1e-15)
try:empirical_tail_risk(losses,1)
except ValueError:print("Empty-tail confidence endpoint rejected")
else:raise AssertionError("Invalid confidence accepted")
OUTPUT_DIR.mkdir(parents=True,exist_ok=True);risk_table.to_csv(OUTPUT_DIR/"empirical-tail-risk.csv",index=False)
pd.DataFrame(draws,columns=["VaR","fractional_ES"]).to_csv(OUTPUT_DIR/"conditional-block-risk.csv",index=False)
pd.DataFrame({"date":returns.index,"hypothetical_loss":losses}).to_csv(OUTPUT_DIR/"hypothetical-loss-sequence.csv",index=False)
print("Hand-calculated tail check passed; conditional ES range:",interval)
Empty-tail confidence endpoint rejected Hand-calculated tail check passed; conditional ES range: [0.00637013 0.00758114]
Exercises and reference explanation
- Why state quantile and tail-mass conventions?
- Which assumptions support circular blocks?
Reference explanation. Empirical interpolation and inverse-CDF tail integration differ. Circular blocks assume stationarity and make an artificial boundary; intervals are conditional.
Keep a copy before changing parameters. Compare old and new figures and record which inputs, calculations and conclusions changed.
Optional AI coding exercises
Prompt 1
Using NumPy/Pandas/Matplotlib, compare block lengths 1, 5, 10 with recorded seeds.
Prompt 2
With the same packages, export 90%, 95%, 99% risk with the exact empirical convention.
Human acceptance. Hand examples pass, ES≥VaR, endpoint confidence fails and no future-loss guarantee is claimed.
Review the edited source, then manually run affected cells and subsequent checks. Ask the assistant to explain the purpose, packages and data provenance of its changes, and reconcile that explanation with actual output.
Optional online extension
RUN_ONLINE_UPDATE=False
if RUN_ONLINE_UPDATE:
import requests
try:
response=requests.get('https://data-api.ecb.europa.eu/service/data/EXR/D.USD.EUR.SP00.A?startPeriod=2024-01-01&endPeriod=2024-12-31&format=csvdata',timeout=20)
response.raise_for_status()
print("Source:",response.url)
print(response.text[:700])
except requests.RequestException as error:
print("Online extension unavailable:",type(error).__name__,str(error)[:160])
else:
print("Online extension skipped; embedded core data unchanged")
Online extension skipped; embedded core data unchanged
Sources, snapshots and next steps
- ECB SDMX euro reference exchange rates — ECB copyright / free reuse with accurate source attribution; data obtained free from ECB; derived returns explicitly labelled; retrieved 2026-10-09. SHA-256
5f39fc39f60606abafae26c00492d05388c7bdf29cce1922f173f99bd0273ce8. Retain 2019-01-01 through 2024-12-31 and USD, GBP, JPY, CNY; preserve missing values and quotation direction.
Next steps. Transfer these methods to your own public or authorized data while retaining input checks, recorded parameters and result validation. Document the new experimental design and method limits; this example does not validate a new experiment. Full data licenses and generation instructions accompany the cookbook.