Chapter 3 Randomized Inference

Adapted from Cunningham (2021)

We will use a placebo-based test to calculate exact p-values. We need every single permutation of treatment to calculate the exact p-values from the ATE.

3.1 Set up the Dataset

First we will set the seed so we can replicate our randomized results

set seed 1234

Next we will bring in our

cd "/Users/Sam/Desktop/Econ 672/Data"
use ri.dta, clear
tempfile ri
gen id = _n
save "`ri'", replace

Randomized Inference Example Next we will create all combinations from our eight observations. We will need to install the percom package into stata.

ssc install percom

Out of 8 observations, we get 70 permutations. We can use the combin command to get these permutations. From the help combin, the combin command generates \(k\) unique combinations from a set of \(n\) objects, where \(k\) is not greater than \(5 (2 =< k <= 5)\). Program first arranges \(k\) possible permutations and then retains unique combinations. It supports both string and numeric variables.

combination: \(\frac{n!}{k!(n-k)!}\)

We will tell the combin command that we want 4 combinations from our 8 observations.

combin id, k(4)
         k =          4
         N =          8
Combinations Formed: 70

Permutations Next we will label the permutations 1 to 70.

gen permutation = _n
tempfile combo
save "`combo'", replace

forvalue i =1/4 {
    rename id_`i' treated`i'
}

Next we will merge our permutations with our dataset of potential outcomes

destring treated*, replace
cross using `ri'
Merge Dataset with Permutations
Merge Dataset with Permutations

3.2 Randomize Assignment to Treatment

Next we will randomized treatment and calculate other test statistics.

First, we create a fake treatment variable \(d2\)

gen d2 = .
replace d2 = 1 if id == treated1 | id == treated2 | id == treated3 | id == treated4
replace d2 = 0 if ~(id == treated1 | id == treated2 | id == treated3 | id == treated4)
gen check = d - d2
Set up False Treatment
Set up False Treatment

Next, we will calculate true effect using absolute value of simple difference in outcomes \((SDO)\).

egen te1 = mean(y) if d2==1, by(permutation)
egen te0 = mean(y) if d2==0, by(permutation)
Get the average treatment effects by permutations
Get the average treatment effects by permutations

3.3 Calculate the exact p-value

Get average treatment effects \((ATE)\)

collapse (mean) te1 te0, by(permutation)
gen     ate = te1 - te0
keep    ate permutation
Collapse the treatment effects by permutation
Collapse the treatment effects by permutation

Next, we rank the permutations

gsort -ate
gen rank = _n
sum rank if permutation==1
Sort and Rank by ATE
Sort and Rank by ATE

Finally, we calculate Exact \(p\)-value

gen pvalue = (`r(mean)'/70)
list pvalue if permutation==1
     |    pvalue |
     |-----------|
 29. | .41428571 |
     +-----------+