Measuring Causal Uncertainty with Cluster Bootstrapping | Humanitarians AI

Recapping Causal Couture's week 6 foundations

[0:00]I'm Ashes Vi Rachel. Week six established causal couture's first working causal intelligence foundation, observational effect estimation, row-level bootstrap uncertainty, placebo testing, treatment variation diagnostics, stockout analysis, robustness testing, and observational evidence reporting. Week seven focuses on making that foundation more reliable. This week does not introduce a new causal question. Instead, it addresses

The risk of ignoring repeated product patterns across dates

[0:24]One methodological weakness in the existing analysis. How uncertainty is measured around the estimated effect of social engagement on subsequent product sales. Existing validation workflow uses bootstrap resampling. Eligible observations are sampled. The engagement effect is reestimated and this process is repeated to create an empirical distribution of estimates. From that distribution, the system can summarize

Building the cluster bootstrap algorithm in Python

[0:46]The mean effect, median effect, bootstrap standard error, and 95% interval. The analytical grain is date by SKU. The same product can appear repeatedly across different dates. Treating every day TKU row as an independently resampled unit may overlook the fact that multiple observations belong to the same product over time. Those repeated observations may share persistent characteristics, baseline demand, historical popularity,

Using unique bootstrap cluster identifiers to avoid fixed-effect collapse

[1:11]Engagement behavior, inventory patterns, or other product specific factors. These shared features can make observations within one SKU related to one another. This does not make the original rowle bootstrap useless or automatically wrong. It means that relying only on independent row resampling may provide an incomplete picture of uncertainty when observations are naturally clustered within products. To address this limitation, I implemented a new

Integrating both bootstrap methods into the validation response API

[1:36]Skule cluster bootstrap procedure in app/appi/causal_cluster_ootstrap.py. The key change is the resampling unit from an individual date SKU observation to an entire SKU cluster. Procedure first identifies the unique SKUs in the eligible causal analysis dare set. It then samples those SKU clusters with replacement. All eligible observations

Documenting method comparisons, limitations, and wider intervals

[1:58]Belonging to each sampled SKU are retained and a new bootstrap dare set is reconstructed. The existing causal estimator is used to reestimate the engagement effect. This repeats across bootstrap iterations producing an empirical distribution of estimated effects. In the rowle bootstrap, individual rows are sampled independently. In the SKU level cluster bootstrap, entire product clusters are sampled together. The underlying observations remain the same. The resampling unit changes, so repeated product structure is better preserved because sampling occurs with replacement. The same SKU can be selected more than once in a single bootstrap iteration. Each sampled copy receives a unique bootstrap cluster identifier. This prevents duplicate copies from collapsing into one SKU category when the estimator constructs SKU fixed effects. Cluster bootstrap output reports available SKU clusters, requested iterations, and successful estimations. It also reports the mean engagement effect, median effect,

Bootstrap standard error, and 95% interval. Checks for insufficient clusters or successful samples allow the system to fail transparently instead of returning misleading uncertainty estimates. New method was integrated into get/phase 5/causal validation. The original rowle uncertainty output was deliberately preserved. The response now distinguishes uncertainty, meaning the original rowle bootstrap from cluster underscore uncertainty, meaning the new SKU level method. The rest of the phase 5 validation workflow remains intact. Keeping both methods makes comparison possible. The goal is not to replace one approach automatically, but to examine how sensitive the uncertainty assessment is to the choice of resampling unit. This gives the validation framework an additional reliability check. If the rowle and SKU cluster estimates and intervals are reasonably similar, the effect appears less sensitive to the resampling unit. If the cluster bootstrap produces substantially wider uncertainty within SKU dependence

May matter and independent row resampling may understate uncertainty. A wider interval does not automatically mean the estimate is wrong. This comparison is a robustness diagnostic. It does not remove unobserved confounding, prove the causal DAG, guarantee treatment positivity, or establish causality. I documented the method comparison guidance and these analytical limits in doc/phase 5_cluster bootstrap_refinement. MD. After integration, the fast API backend loaded successfully. The updated validation workflow executed correctly. Cluster uncertainty appeared alongside the original analysis and the broader phase 5 pipeline remained intact. At the end of week seven, causal coutur has two complimentary uncertainty approaches. Rowlevel bootstrap and skule level cluster bootstrap. New method better respects repeated product observations and makes sensitivity to within product dependence more transparent. This weak improved causal reliability by refining uncertainty estimation. But improved uncertainty estimation is not the same as improved causal identification.