The Steps of Statistical Modeling

1. What is your question?

2. What model of the world matches your question?

3. Is your model valid?

4. Query your model to answer your question.

Our question of the day: What is the relationship between inbreeding coefficient and litter size in wolves?

Roll that beautiful linear regression with 95% CI footage

Regression to Be Mean

1. What is regression?

2. What do regression coefficients mean?

3. What do the error coefficients of a regression mean?

4. Correlation and Regression

5. Transformation and Model Structure for More Sensible Coefficients

What is a regression?

y = a + bx + error

This is 90% of the modeling you will ever do because...

Everything is a linear model!

• multiple parameters (x1, x2, etc...)

• nonlinear transformations of y or x

• multiplicative terms (b x1 x2) are still additive

• generalized linear models with non-normal error

• and so much more....

Linear Regression

$\Large y_i = \beta_0 + \beta_1 x_i + \epsilon_i$

$\Large \epsilon_i \sim^{i.i.d.} N(0, \sigma)$

Regressions You Have Seen

---

---

These All Are Equation-Forms of This Relationship

Regression to Be Mean

1. What is regression?

2. What do regression coefficients mean?

3. What do the error coefficients of a regression mean?

4. Correlation and Regression

5. Transformation and Model Structure for More Sensible Coefficients

What are we doing with regression?

Goals:

1. Association

   • What is the strength of a relationship between two quantities
   • Not causal

2. Prediction

   • If we have two groups that differ in their X value by 1 unit, what is the average difference in their Y unit?
   • Not causal

3. Counterfactual

   • What would happen to an individual if their value of X increased by one unit?
   • Causal reasoning!

What Can We Say About This?

Model Coefficients: Slope

term	estimate	std.error
(Intercept)	6.567	0.791
inbreeding.coefficient	-11.447	3.189

1. Association: A one unit increase in inbreeding coefficient is associated with ~11 fewer pups, on average.

2. Prediction: A new wolf with an inbreeding coefficient 1 unit greater than a second new wolf will have ~11 fewer pups, on average.

3. Counterfactual: If an individual wolf had had its inbreeding coefficient 1 unit higher, it would have ~11 fewer pups.

Which of these is the correct thing to say? When?

1. Association: A one unit increase in inbreeding coefficient is associated with ~11 fewer pups, on average.

2. Prediction: A new wolf with an inbreeding coefficient 1 unit greater than a second new wolf will have ~11 fewer pups, on average.

3. Counterfactual: If an individual wolf had had its inbreeding coefficient 1 unit higher, it would have ~11 fewer pups.

11 Fewer Pups? What would be, then, a Better Way to Talk About this Slope?

Model Coefficients: Intercept

term	estimate	std.error
(Intercept)	6.567	0.791
inbreeding.coefficient	-11.447	3.189

When the inbreeding coefficient is 0, a wolves will have ~6.6 pups, on average.

Intercept Has Direct Interpretation on the Visualization

Regression to Be Mean

1. What is regression?

2. What do regression coefficients mean?

3. What do the error coefficients of a regression mean?

4. Correlation and Regression

5. Transformation and Model Structure for More Sensible Coefficients

Two kinds of error

1. Fit error - error due to lack of precision in estimates

   • Coefficient SE
   • Precision of estimates

2. Residual error - error due to variability not explained by X.

   • Residual SD (from $\epsilon_i$)

Precision: coefficient SEs

term	estimate	std.error
(Intercept)	6.567	0.791
inbreeding.coefficient	-11.447	3.189

• Shows precision of ability to estimate coefficients

• Gets smaller with bigger sample size!

• Remember, ~ 2 SE covered 95% CI

• Comes from likelihood surface...but we'll get there

Visualizing Precision: 95% CI (~2 SE)

Visualizing Precision with Simulation from your Model

Residual Error

r.squared	sigma
0.369	1.523

• Sigma is the SD of the residual

$$\Large \epsilon_i \sim N(0,\sigma)$$

• How much does does # of pups vary beyond the relationship with inbreeding coefficient?

• For any number of pups estimated on average, ~68% of the # of pups observed will fall within ~1.5 of that number

Visualizing Residual Error's Implications

Residual Error -> Variance Explained

r.squared	sigma
0.369	1.523

• $\large{R^2 = 1 - \frac{\sigma^2_{residual}}{\sigma^2_y}}$

  • Fraction of the variation in Y related to X.

  • Here, 36.9% of the variation in pups is related to variation in Inbreeding Coefficient

  • Relates to r, the Pearson correlation coefficient

Regression to Be Mean

1. What is regression?

2. What do regression coefficients mean?

3. What do the error coefficients of a regression mean?

4. Correlation and Regression

5. Transformation and Model Structure for More Sensible Coefficients

What is Correlation?

• The change in standard deviations of variable x per change in 1 SD of variable y
  • Clear, right?

• Assesses the degree of association between two variables

• But, unitless (sort of)
  • Between -1 and 1

Calculating Correlation: Start with Covariance

Describes the relationship between two variables. Not scaled.

$\sigma_{xy}$ = population level covariance
$s_{xy}$ = covariance in your sample

Pearson Correlation

Describes the relationship between two variables.
Scaled between -1 and 1.


$\large \rho_{xy}$ = population level correlation, $\large r_{xy}$ = correlation in your sample

Assumptions of Pearson Correlation

The meaning of r

Y is perfectly predicted by X if r = -1 or 1.


$R^2$ = the porportion of variation in y explained by x

Get r in your bones...

http://guessthecorrelation.com/

Example: Wolf Breeding and Litter Size

Example: Wolf Inbreeding and Litter Size

Covariance Matrix:

                       inbreeding.coefficient  pups
inbreeding.coefficient                   0.01 -0.11
pups                                    -0.11  3.52

Correlation Matrix:

                       inbreeding.coefficient  pups
inbreeding.coefficient                   1.00 -0.61
pups                                    -0.61  1.00

Yes, you can estimate a SE (cor.test() or bootstrapping)

Wait, so, how does Correlation relate to Regression? Slope versus r...

$\LARGE b=\frac{s_{xy}}{s_{x}^2}$ $= \frac{cov(x,y)}{var(x)}$

$\LARGE = r_{xy}\frac{s_{y}}{s_{x}}$

Correlation v. Regression Coefficients

Or really, r is just the coefficient of a fit lm with a z-transform of our predictors

$$\Large z_i = \frac{x_i - \bar{x}}{\sigma_x}$$

Correlation versus Standardized Regression: It's the Same Picture

$$z(y_i) = \beta_0 + \beta_1 z(x_i) + \epsilon_i$$

term	estimate	std.error
(Intercept)	0.000	0.166
inbreeding_std	-0.608	0.169

versus correlation: -0.608

Regression to Be Mean

1. What is regression?

2. What do regression coefficients mean?

3. What do the error coefficients of a regression mean?

4. Correlation and Regression

5. Transformation and Model Structure for More Sensible Coefficients

Modifying (transformating) Your Regression: Centering you X

• Many times X = 0 is silly

• E.g., if you use year, are you going to regress back to 0?

• Centering X allows you to evaluate a meaningful intercept

  • what is Y at the mean of X

Centering X to generate a meaningful intercept

$$x_{i \space centered} = x_i - mean(x)$$

term	estimate	std.error
(Intercept)	3.958	0.311
inbreeding.centered	-11.447	3.189

Intercept implies wolves with the average level of inbreeding in this study have ~4 pups. Wolves with higher inbreeding have fewer pups, wolves with lower inbreeding have more.

Centering X to generate a meaningful intercept

Modifying (transformating) Your Regression: Log Transform of Y

• Often, Y cannot be negative

• And/or the process generating Y is multiplicative

• Log(Y) can fix this and other sins.

• VERY common, but, what do the coefficients mean?

  • $exp(\beta_1) - 1 \approx$ percent change in Y for chance in 1 unit of X

Other Ways of Looking at This Relationship: Log Transformation of Y

$$log(y_i) = \beta_0 + \beta_1 x_i + \epsilon_i$$

• relationship is now curved
• cannot have negative pups (yay!)

Model Coefficients: Log Slope

term	estimate	std.error
(Intercept)	1.944	0.215
inbreeding.coefficient	-2.994	0.869

To understand the coefficient, remember $$y_i = e^{\beta_0 + \beta_1 x_i + \epsilon_i}$$

exp(-2.994)-1 = -0.95, so, a 1 unit increase in x causes y to lose 95% of its value, so...

Association: A one unit increase in inbreeding coefficient is associated with having 95% fewer pups, on average.

You are now a Statistical Wizard. Be Careful. Your Model is a Golem.

(sensu Richard McElreath)