The world isn't additive

• Until now, we have assumed predictors combine additively
  • the effect of one is not dependent on the effect of the other

• BUT - what if the effect of one variable depends on the level of another?

• This is an INTERACTION and is quite common

  • Heck, a squared term is the interaction of a variable with itself!

• Biology: The science of "It depends..."

• This is challenging to think about and visualize, but if you can master it, you will go far!

We Have Explored Nonliearities Via Transforming our Response

But We Have Also Always Tested for Non-Additivity of Predictors

The Linear Model Can Accomodate Many Flavors of Nonlinearity

$$\hat{y_i} = \beta_0 + \beta_1 x_{1i} + \beta_2 x_{2i}$$ $$y_i \sim N(\hat{y_i}, \sigma)$$ Could become...

$$\hat{y_i} = \beta_0 + \beta_1 x_{1i} + \beta_2 x_{1i}^2$$ Could be...

$$\hat{y_i} = \beta_0 + \beta_1 x_{1i} + \beta_2 x_{2i} + \beta_3 x_{1i}x_{2i}$$ It is ALL additive terms

A Non-Additive World

1. Replicating Categorical Variable Combinations: Factorial Models

2. Evaluating Interaction Effects

3. How to Look at Means and Differences with an Interaction Effect

4. Continuous Variables and Interaction Effects

Intertidal Grazing!

Experiment Replicated on Two Ends of a gradient

Factorial Experiment

Factorial Design

Note: You can have as many treatment types or observed category combinations as you want (and then 3-way, 4-way, etc. interactions)

The Data: See the dependency of one treatment on another?

If we had fit y ~ a + b, residuals look weird

A Factorial Model

$$\large y_{ijk} = \beta_{0} + \sum \beta_{i}x_{i} + \sum \beta_{j}x_{j} + \sum \beta_{ij}x_{ij} + \epsilon_{ijk}$$

$$\large \epsilon_{ijk} \sim N(0, \sigma^{2} )$$ $$\large x_{i} = 0,1, x_{j} = 0,1, x_{ij} = 0,1$$

• Note the new last term

• Deviation due to combination of categories i and j

$$\Large \boldsymbol{Y} = \boldsymbol{\beta X} + \boldsymbol{\epsilon}$$

The Data (Four Rows)

The Dummy-Coded Treatment Contrasts

Fitting with Least Squares

graze_int <- lm(sqrtarea ~ height + herbivores +
                  height:herbivores,
                data=algae)
## OR
graze_int <- lm(sqrtarea ~ height*herbivores,
                data=algae)

Now We Are Linear/Additive

Residuals A-OK

HOV Good!

No Outliers

Collinearity is Tricky - unimportant for interaction

A Non-Additive World

1. Replicating Categorical Variable Combinations: Factorial Models

2. Evaluating Interaction Effects

3. How to Look at Means and Differences with an Interaction Effect

4. Continuous Variables and Interaction Effects

What do the Coefficients Mean?

• Intercept chosen as basal condition (low, herbivores -)

• Changing height to mid is associated with a loss of 10 units of algae relative to low/-

• Adding herbivores is associated with a loss of 22 units of algae relative to low/-

• BUT - if you add herbivores and mid, that's also associated with an additional increase of 25 units of algae relative to mid and + alone

  • 25.5 - 22.5 - 10.4 = only a loss of 7.4 relative to low/-

What do the Coefficients Mean?

NEVER TRY AND INTERPRET ADDITIVE EFFECTS ALONE WHEN AN INTERACTION IS PRESENT
that way lies madness

This view is intuitive

This view is also intuitive

We Can Still Look at R^2

# R2 for Linear Regression
       R2: 0.228
  adj. R2: 0.190

Eh, not great, not bad...

• Note: adding more interaction effects will always increase the R² so only add if warranted - NO FISHING!

A Non-Additive World

1. Replicating Categorical Variable Combinations: Factorial Models

2. Evaluating Interaction Effects

3. How to Look at Means and Differences with an Interaction Effect

4. Continuous Variables and Interaction Effects

Posthoc Estimated Means and Interactions with Categorical Variables

• Must look at simple effects first in the presence of an interaction
  • The effects of individual treatment combinations
  • If you have an interaction, this is what you do!

• Main effects describe effects of one variable in the absence of an interaction
  • Useful only if there is no interaction
  • Or useful if one categorical variable can be absent

Estimated Means with No Interaction - Misleading!

 herbivores emmean   SE df lower.CL upper.CL
 minus        27.7 2.73 60     22.2     33.2
 plus         18.0 2.73 60     12.5     23.4
Results are averaged over the levels of: height 
Confidence level used: 0.95

 height emmean   SE df lower.CL upper.CL
 low      21.7 2.73 60     16.2     27.1
 mid      24.0 2.73 60     18.6     29.5
Results are averaged over the levels of: herbivores 
Confidence level used: 0.95

Posthoc Comparisons Averaging Over Blocks - Misleading!

 contrast     estimate   SE df lower.CL upper.CL
 minus - plus     9.72 3.86 60     2.01     17.4
Results are averaged over the levels of: height 
Confidence level used: 0.95

Simple Effects Means

 height herbivores emmean   SE df lower.CL upper.CL
 low    minus        32.9 3.86 60    25.20     40.6
 mid    minus        22.5 3.86 60    14.77     30.2
 low    plus         10.4 3.86 60     2.69     18.1
 mid    plus         25.6 3.86 60    17.84     33.3
Confidence level used: 0.95

Posthoc with Simple Effects

Might be easier visually

We are often interested in looking at differences within levels...

height = low:
 herbivores emmean   SE df lower.CL upper.CL
 minus        32.9 3.86 60    25.20     40.6
 plus         10.4 3.86 60     2.69     18.1
height = mid:
 herbivores emmean   SE df lower.CL upper.CL
 minus        22.5 3.86 60    14.77     30.2
 plus         25.6 3.86 60    17.84     33.3
Confidence level used: 0.95

We Can Then Look at Something Simpler...

Why think about interactions?

• It Depends is a rule in biology

• Context dependent interactions everywhere

• Using categorical predictors in a factorial design is an elegant way to see interactions without worrying about shapes of relationships

• BUT - it all comes down to a general linear model! And the same inferential frameworks we have been dealing with since linear regression!

To Blow Your Mind - You can have 2, 3, and more-way interactions!

A Non-Additive World

1. Replicating Categorical Variable Combinations: Factorial Models

2. Evaluating Interaction Effects

3. How to Look at Means and Differences with an Interaction Effect.

4. Continuous Variables and Interaction Effects

Problem: What if Continuous Predictors are Not Additive?

Problem: What if Continuous Predictors are Not Additive?

Problem: What if Continuous Predictors are Not Additive?

Model For Age Interacting with Elevation to Influence Fire Severity

$$y_i = \beta_0 + \beta_{1}x_{1i} + \beta_{2}x_{2i}+ \beta_{3}x_{1i}x_{2i} + \epsilon_{i}$$

$$\epsilon_{i} \sim \mathcal{N}(0,\sigma^2)$$

Or in code:

keeley_lm_int <- lm(firesev ~ age * elev, data=keeley)

Assumption Tests as Usual!

Examine Residuals With Respect to Each Predictor

Interactions, VIF, and Centering

# Check for Multicollinearity
Low Correlation
 Term  VIF    VIF 95% CI Increased SE Tolerance Tolerance 95% CI
  age 3.20 [2.37,  4.52]         1.79      0.31     [0.22, 0.42]
Moderate Correlation
     Term  VIF    VIF 95% CI Increased SE Tolerance Tolerance 95% CI
     elev 5.52 [3.95,  7.92]         2.35      0.18     [0.13, 0.25]
 age:elev 8.29 [5.83, 11.98]         2.88      0.12     [0.08, 0.17]

• Collinearities between additive predictors and interaction effects are not problematic.

• However, you should make sure your ADDITIVE predictors do not have VIF problems in a model with no interactions.

• If you are worried, center your predictors - i.e., $X_i - mean(X)$
  • This can fix issues with models not converging

Interpretation of Centered Coefficients

$$\huge X_i - \bar{X}$$

• Additive coefficients are the effect of a predictor at the mean value of the other predictors

• Intercepts are at the mean value of all predictors

• Visualization will keep you from getting confused!

Interactions, VIF, and Centering

$$y = \beta_0 + \beta_{1}(x_{1}-\bar{x_{1}}) + \beta_{2}(x_{2}-\bar{x_{2}})+ \beta_{3}(x_{1}-\bar{x_{1}})(x_{2}-\bar{x_{2}})$$

Variance Inflation Factors for Centered Model:

# Check for Multicollinearity
Low Correlation
         Term  VIF      VIF 95% CI Increased SE Tolerance Tolerance 95% CI
        age_c 1.02 [1.00, 2376.13]         1.01      0.98     [0.00, 1.00]
       elev_c 1.04 [1.00,    7.04]         1.02      0.96     [0.14, 1.00]
 age_c:elev_c 1.04 [1.00,    9.78]         1.02      0.96     [0.10, 1.00]

Coefficients!

R² = 0.3235187.

Note that additive coefficients signify the effect of one predictor in the abscence of all others.

Centered Coefficients!

R² = 0.3235187

Note that additive coefficients signify the effect of one predictor at the average level of all others.

Interpretation

• What the heck does a continuous interaction effect mean?

• We can look at the effect of one variable at different levels of the other

• We can look at a surface

• We can construct counterfactual plots showing how changing both variables influences our outcome

Age at Different Levels of Elevation

Elevation at Different Levels of Age

Or all in one plot

Without Data and Including CIs

A Heatmap Approach

Surfaces and Other 3d Objects