class: center, middle <!-- make the interaction plots make more sense and relate to a question --> # When The Effect of One Predictor Depends on Another ![](images/anova/yoda_factorial.jpg) --- class: center, middle # Etherpad <br><br> <center><h3>https://etherpad.wikimedia.org/p/607-complex-linear-models-2023</h3></center> --- # The world isn't additive - Until now, we have assumed predictors combine additively - the effect of one is not dependent on the effect of the other -- - BUT - what if the effect of one variable depends on the level of another? -- - This is an **INTERACTION** and is quite common - Heck, a squared term is the interaction of a variable with itself! -- - Biology: The science of "It depends..." -- - This is challenging to think about and visualize, but if you can master it, you will go far! --- # We Have Explored Nonliearities Via Transforming our Response <img src="interactions_lm_files/figure-html/loglinear-1.png" style="display: block; margin: auto;" /> --- # But We Have Also Always Tested for Non-Additivity of Predictors <img src="interactions_lm_files/figure-html/unnamed-chunk-1-1.png" style="display: block; margin: auto;" /> --- # The Linear Model Can Accomodate Many Flavors of Nonlinearity `$$\hat{y_i} = \beta_0 + \beta_1 x_{1i} + \beta_2 x_{2i}$$` `$$y_i \sim N(\hat{y_i}, \sigma)$$` -- Could become... `$$\hat{y_i} = \beta_0 + \beta_1 x_{1i} + \beta_2 x_{1i}^2$$` -- Could be... `$$\hat{y_i} = \beta_0 + \beta_1 x_{1i} + \beta_2 x_{2i} + \beta_3 x_{1i}x_{2i}$$` -- .center[**It is ALL additive terms**] --- class: center, middle ![](images/anova/two_way_interaction.jpg) --- # A Non-Additive World 1. Replicating Categorical Variable Combinations: Factorial Models 2. Evaluating Interaction Effects 3. How to Look at Means and Differences with an Interaction Effect 4. Continuous Variables and Interaction Effects --- # Intertidal Grazing! .center[ ![image](./images/22/grazing-expt.jpeg) #### Do grazers reduce algal cover in the intertidal? ] --- # Experiment Replicated on Two Ends of a gradient ![image](./images/22/zonation.jpg) --- # Factorial Experiment ![image](./images/22/factorial_blocks.jpg) --- # Factorial Design ![image](./images/22/factorial_layout.jpg) Note: You can have as many treatment types or observed category combinations as you want (and then 3-way, 4-way, etc. interactions) --- # The Data: See the dependency of one treatment on another? <img src="interactions_lm_files/figure-html/plot_algae-1.png" style="display: block; margin: auto;" /> --- # If we had fit y ~ a + b, residuals look weird <img src="interactions_lm_files/figure-html/graze_assumptions-1.png" style="display: block; margin: auto;" /> --- # A Factorial Model `$$\large y_{ijk} = \beta_{0} + \sum \beta_{i}x_{i} + \sum \beta_{j}x_{j} + \sum \beta_{ij}x_{ij} + \epsilon_{ijk}$$` `$$\large \epsilon_{ijk} \sim N(0, \sigma^{2} )$$` `$$\large x_{i} = 0,1, x_{j} = 0,1, x_{ij} = 0,1$$` -- - Note the new last term -- - Deviation due to *combination of categories i and j* -- <hr> This is still something that is in `$$\Large \boldsymbol{Y} = \boldsymbol{\beta X} + \boldsymbol{\epsilon}$$` --- # The Data (Four Rows) <table class="table table-striped" style="margin-left: auto; margin-right: auto;"> <thead> <tr> <th style="text-align:left;"> height </th> <th style="text-align:left;"> herbivores </th> <th style="text-align:right;"> sqrtarea </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> low </td> <td style="text-align:left;"> minus </td> <td style="text-align:right;"> 9.4055728 </td> </tr> <tr> <td style="text-align:left;"> low </td> <td style="text-align:left;"> plus </td> <td style="text-align:right;"> 11.9767608 </td> </tr> <tr> <td style="text-align:left;"> mid </td> <td style="text-align:left;"> minus </td> <td style="text-align:right;"> 0.7071068 </td> </tr> <tr> <td style="text-align:left;"> mid </td> <td style="text-align:left;"> plus </td> <td style="text-align:right;"> 0.7071068 </td> </tr> </tbody> </table> --- # The Dummy-Coded Treatment Contrasts <table class="table table-striped" style="margin-left: auto; margin-right: auto;"> <thead> <tr> <th style="text-align:right;"> (Intercept) </th> <th style="text-align:right;"> heightmid </th> <th style="text-align:right;"> herbivoresplus </th> <th style="text-align:right;"> heightmid:herbivoresplus </th> </tr> </thead> <tbody> <tr> <td style="text-align:right;"> 1 </td> <td style="text-align:right;"> 0 </td> <td style="text-align:right;"> 0 </td> <td style="text-align:right;"> 0 </td> </tr> <tr> <td style="text-align:right;"> 1 </td> <td style="text-align:right;"> 0 </td> <td style="text-align:right;"> 1 </td> <td style="text-align:right;"> 0 </td> </tr> <tr> <td style="text-align:right;"> 1 </td> <td style="text-align:right;"> 1 </td> <td style="text-align:right;"> 0 </td> <td style="text-align:right;"> 0 </td> </tr> <tr> <td style="text-align:right;"> 1 </td> <td style="text-align:right;"> 1 </td> <td style="text-align:right;"> 1 </td> <td style="text-align:right;"> 1 </td> </tr> </tbody> </table> --- # Fitting with Least Squares ```r graze_int <- lm(sqrtarea ~ height + herbivores + height:herbivores, data=algae) ## OR graze_int <- lm(sqrtarea ~ height*herbivores, data=algae) ``` --- # Now We Are Linear/Additive <img src="interactions_lm_files/figure-html/unnamed-chunk-2-1.png" style="display: block; margin: auto;" /> --- # Residuals A-OK <img src="interactions_lm_files/figure-html/unnamed-chunk-3-1.png" style="display: block; margin: auto;" /> --- # HOV Good! <img src="interactions_lm_files/figure-html/unnamed-chunk-4-1.png" style="display: block; margin: auto;" /> --- # No Outliers <img src="interactions_lm_files/figure-html/unnamed-chunk-5-1.png" style="display: block; margin: auto;" /> --- # Collinearity is Tricky - unimportant for interaction <img src="interactions_lm_files/figure-html/unnamed-chunk-6-1.png" style="display: block; margin: auto;" /> --- # A Non-Additive World 1. Replicating Categorical Variable Combinations: Factorial Models 2. .red[ Evaluating Interaction Effects ] 3. How to Look at Means and Differences with an Interaction Effect 4. Continuous Variables and Interaction Effects --- # What do the Coefficients Mean? <table class="table table-striped" style="margin-left: auto; margin-right: auto;"> <thead> <tr> <th style="text-align:left;"> term </th> <th style="text-align:right;"> estimate </th> <th style="text-align:right;"> std.error </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> (Intercept) </td> <td style="text-align:right;"> 32.91450 </td> <td style="text-align:right;"> 3.855532 </td> </tr> <tr> <td style="text-align:left;"> heightmid </td> <td style="text-align:right;"> -10.43090 </td> <td style="text-align:right;"> 5.452546 </td> </tr> <tr> <td style="text-align:left;"> herbivoresplus </td> <td style="text-align:right;"> -22.51075 </td> <td style="text-align:right;"> 5.452546 </td> </tr> <tr> <td style="text-align:left;"> heightmid:herbivoresplus </td> <td style="text-align:right;"> 25.57809 </td> <td style="text-align:right;"> 7.711064 </td> </tr> </tbody> </table> -- - Intercept chosen as basal condition (low, herbivores -) -- - Changing height to high is associated with a loss of 10 units of algae relative to low/- -- - Adding herbivores is associated with a loss of 22 units of algae relative to low/- -- - BUT - if you add herbivores and mid, that's also associated with an additional increase of 25 units of algae relative to mid and + alone - 25.5 - 22.5 - 10.4 = only a loss of 7.4 relative to low/- -- .center[**NEVER TRY AND INTERPRET ADDITIVE EFFECTS ALONE WHEN AN INTERACTION IS PRESENT**<Br>that way lies madness] --- # This view is intuitive <img src="interactions_lm_files/figure-html/unnamed-chunk-7-1.png" style="display: block; margin: auto;" /> --- # This view is also intuitive <img src="interactions_lm_files/figure-html/unnamed-chunk-8-1.png" style="display: block; margin: auto;" /> --- # We Can Still Look at R^2 ``` # R2 for Linear Regression R2: 0.228 adj. R2: 0.190 ``` Eh, not great, not bad... -- - Note: adding more interaction effects will always increase the R<sup>2</sup> so only add if warranted - NO FISHING! --- # A Non-Additive World 1. Replicating Categorical Variable Combinations: Factorial Models 2. Evaluating Interaction Effects 3. .red[ How to Look at Means and Differences with an Interaction Effect ] 4. Continuous Variables and Interaction Effects --- # Posthoc Estimated Means and Interactions with Categorical Variables - Must look at **simple effects** first in the presence of an interaction - The effects of individual treatment combinations - If you have an interaction, this is what you do! -- - **Main effects describe effects of one variable in the absence of an interaction** - Useful only if there is no interaction - Or useful if one categorical variable can be absent --- # Estimated Means with No Interaction - Misleading! ``` herbivores emmean SE df lower.CL upper.CL minus 27.7 2.73 60 22.2 33.2 plus 18.0 2.73 60 12.5 23.4 Results are averaged over the levels of: height Confidence level used: 0.95 ``` ``` height emmean SE df lower.CL upper.CL low 21.7 2.73 60 16.2 27.1 mid 24.0 2.73 60 18.6 29.5 Results are averaged over the levels of: herbivores Confidence level used: 0.95 ``` --- # Posthoc Comparisons Averaging Over Blocks - Misleading! ``` contrast estimate SE df lower.CL upper.CL minus - plus 9.72 3.86 60 2.01 17.4 Results are averaged over the levels of: height Confidence level used: 0.95 ``` --- # Simple Effects Means ``` height herbivores emmean SE df lower.CL upper.CL low minus 32.9 3.86 60 25.20 40.6 mid minus 22.5 3.86 60 14.77 30.2 low plus 10.4 3.86 60 2.69 18.1 mid plus 25.6 3.86 60 17.84 33.3 Confidence level used: 0.95 ``` --- # Posthoc with Simple Effects <table class="table table-striped" style="margin-left: auto; margin-right: auto;"> <thead> <tr> <th style="text-align:left;"> contrast </th> <th style="text-align:right;"> estimate </th> <th style="text-align:right;"> SE </th> <th style="text-align:right;"> df </th> <th style="text-align:right;"> t.ratio </th> <th style="text-align:right;"> p.value </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> low minus - mid minus </td> <td style="text-align:right;"> 10.430905 </td> <td style="text-align:right;"> 5.452546 </td> <td style="text-align:right;"> 60 </td> <td style="text-align:right;"> 1.913034 </td> <td style="text-align:right;"> 0.0605194 </td> </tr> <tr> <td style="text-align:left;"> low minus - low plus </td> <td style="text-align:right;"> 22.510748 </td> <td style="text-align:right;"> 5.452546 </td> <td style="text-align:right;"> 60 </td> <td style="text-align:right;"> 4.128484 </td> <td style="text-align:right;"> 0.0001146 </td> </tr> <tr> <td style="text-align:left;"> low minus - mid plus </td> <td style="text-align:right;"> 7.363559 </td> <td style="text-align:right;"> 5.452546 </td> <td style="text-align:right;"> 60 </td> <td style="text-align:right;"> 1.350481 </td> <td style="text-align:right;"> 0.1819337 </td> </tr> <tr> <td style="text-align:left;"> mid minus - low plus </td> <td style="text-align:right;"> 12.079843 </td> <td style="text-align:right;"> 5.452546 </td> <td style="text-align:right;"> 60 </td> <td style="text-align:right;"> 2.215450 </td> <td style="text-align:right;"> 0.0305355 </td> </tr> <tr> <td style="text-align:left;"> mid minus - mid plus </td> <td style="text-align:right;"> -3.067346 </td> <td style="text-align:right;"> 5.452546 </td> <td style="text-align:right;"> 60 </td> <td style="text-align:right;"> -0.562553 </td> <td style="text-align:right;"> 0.5758352 </td> </tr> <tr> <td style="text-align:left;"> low plus - mid plus </td> <td style="text-align:right;"> -15.147189 </td> <td style="text-align:right;"> 5.452546 </td> <td style="text-align:right;"> 60 </td> <td style="text-align:right;"> -2.778003 </td> <td style="text-align:right;"> 0.0072896 </td> </tr> </tbody> </table> -- .center[**That's a Lot to Drink In!**] --- # Might be easier visually <img src="interactions_lm_files/figure-html/graze_posthoc_plot-1.png" style="display: block; margin: auto;" /> --- # We are often interested in looking at differences within levels... ``` height = low: herbivores emmean SE df lower.CL upper.CL minus 32.9 3.86 60 25.20 40.6 plus 10.4 3.86 60 2.69 18.1 height = mid: herbivores emmean SE df lower.CL upper.CL minus 22.5 3.86 60 14.77 30.2 plus 25.6 3.86 60 17.84 33.3 Confidence level used: 0.95 ``` --- # We Can Then Look at Something Simpler... <img src="interactions_lm_files/figure-html/graze_posthoc_plot2-1.png" style="display: block; margin: auto;" /> --- # Why think about interactions - It Depends is a rule in biology - Context dependent interactions everywhere - Using categorical predictors in a factorial design is an elegant way to see interactions without worrying about shapes of relationships - BUT - it all comes down to a general linear model! And the same inferential frameworks we have been dealing with since linear regression! --- # To Blow Your Mind - You can have 2, 3, and more-way interactions! .center[.middle[ ![image](./images/22/4_way_interaction.jpg) ]] --- ## A Non-Additive World 1. Replicating Categorical Variable Combinations: Factorial Models 2. Evaluating Interaction Effects 3. How to Look at Means and Differences with an Interaction Effect. 4. .red[ Continuous Variables and Interaction Effects ] --- ## Problem: What if Continuous Predictors are Not Additive?
## Problem: What if Continuous Predictors are Not Additive? <img src="interactions_lm_files/figure-html/keeley_int_plot-1.png" style="display: block; margin: auto;" /> ## Problem: What if Continuous Predictors are Not Additive? <img src="interactions_lm_files/figure-html/keeley_int_plot2-1.png" style="display: block; margin: auto;" /> --- class:center, middle ![](./images/23/regression_depression.jpg) --- ## Model For Age Interacting with Elevation to Influence Fire Severity `$$y_i = \beta_0 + \beta_{1}x_{1i} + \beta_{2}x_{2i}+ \beta_{3}x_{1i}x_{2i} + \epsilon_{i}$$` `$$\epsilon_{i} \sim \mathcal{N}(0,\sigma^2)$$` Or in code: ```r keeley_lm_int <- lm(firesev ~ age * elev, data=keeley) ``` --- ## Assumption Tests as Usual! <img src="interactions_lm_files/figure-html/klm_diag_int-1.png" style="display: block; margin: auto;" /> --- ## Examine Residuals With Respect to Each Predictor <img src="interactions_lm_files/figure-html/klm_diag2_int-1.png" style="display: block; margin: auto;" /> --- ## Interactions, VIF, and Centering ``` # Check for Multicollinearity Low Correlation Term VIF VIF 95% CI Increased SE Tolerance Tolerance 95% CI age 3.20 [2.37, 4.52] 1.79 0.31 [0.22, 0.42] Moderate Correlation Term VIF VIF 95% CI Increased SE Tolerance Tolerance 95% CI elev 5.52 [3.95, 7.92] 2.35 0.18 [0.13, 0.25] age:elev 8.29 [5.83, 11.98] 2.88 0.12 [0.08, 0.17] ``` - Collinearities between additive predictors and interaction effects are not problematic. -- - However, you should make sure your ADDITIVE predictors do not have VIF problems in a model with no interactions. -- - If you are worried, **center** your predictors - i.e., `\(X_i - mean(X)\)` - This can also fix issues with some models not converging --- ## Interpretation of Centered Coefficients `$$\huge X_i - \bar{X}$$` -- - Additive coefficients are the effect of a predictor at the mean value of the other predictors -- - Intercepts are at the mean value of all predictors -- - Visualization will keep you from getting confused! --- ## Interactions, VIF, and Centering `$$y = \beta_0 + \beta_{1}(x_{1}-\bar{x_{1}}) + \beta_{2}(x_{2}-\bar{x_{2}})+ \beta_{3}(x_{1}-\bar{x_{1}})(x_{2}-\bar{x_{2}})$$` Variance Inflation Factors for Centered Model: ``` # Check for Multicollinearity Low Correlation Term VIF VIF 95% CI Increased SE Tolerance Tolerance 95% CI age_c 1.02 [1.00, 2376.13] 1.01 0.98 [0.00, 1.00] elev_c 1.04 [1.00, 7.04] 1.02 0.96 [0.14, 1.00] age_c:elev_c 1.04 [1.00, 9.78] 1.02 0.96 [0.10, 1.00] ``` --- ## Coefficients! <table class="table table-striped" style="margin-left: auto; margin-right: auto;"> <thead> <tr> <th style="text-align:left;"> term </th> <th style="text-align:right;"> estimate </th> <th style="text-align:right;"> std.error </th> <th style="text-align:right;"> statistic </th> <th style="text-align:right;"> p.value </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> (Intercept) </td> <td style="text-align:right;"> 1.8132153 </td> <td style="text-align:right;"> 0.6156070 </td> <td style="text-align:right;"> 2.945411 </td> <td style="text-align:right;"> 0.0041484 </td> </tr> <tr> <td style="text-align:left;"> age </td> <td style="text-align:right;"> 0.1206292 </td> <td style="text-align:right;"> 0.0208618 </td> <td style="text-align:right;"> 5.782298 </td> <td style="text-align:right;"> 0.0000001 </td> </tr> <tr> <td style="text-align:left;"> elev </td> <td style="text-align:right;"> 0.0030852 </td> <td style="text-align:right;"> 0.0013329 </td> <td style="text-align:right;"> 2.314588 </td> <td style="text-align:right;"> 0.0230186 </td> </tr> <tr> <td style="text-align:left;"> age:elev </td> <td style="text-align:right;"> -0.0001472 </td> <td style="text-align:right;"> 0.0000431 </td> <td style="text-align:right;"> -3.416029 </td> <td style="text-align:right;"> 0.0009722 </td> </tr> </tbody> </table> R<sup>2</sup> = 0.3235187. Note that additive coefficients signify the effect of one predictor in the abscence of all others. --- ## Centered Coefficients! <table class="table table-striped" style="margin-left: auto; margin-right: auto;"> <thead> <tr> <th style="text-align:left;"> term </th> <th style="text-align:right;"> estimate </th> <th style="text-align:right;"> std.error </th> <th style="text-align:right;"> statistic </th> <th style="text-align:right;"> p.value </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> (Intercept) </td> <td style="text-align:right;"> 4.6091266 </td> <td style="text-align:right;"> 0.1463029 </td> <td style="text-align:right;"> 31.503991 </td> <td style="text-align:right;"> 0.0000000 </td> </tr> <tr> <td style="text-align:left;"> age_c </td> <td style="text-align:right;"> 0.0581123 </td> <td style="text-align:right;"> 0.0117591 </td> <td style="text-align:right;"> 4.941901 </td> <td style="text-align:right;"> 0.0000038 </td> </tr> <tr> <td style="text-align:left;"> elev_c </td> <td style="text-align:right;"> -0.0006786 </td> <td style="text-align:right;"> 0.0005792 </td> <td style="text-align:right;"> -1.171587 </td> <td style="text-align:right;"> 0.2445985 </td> </tr> <tr> <td style="text-align:left;"> age_c:elev_c </td> <td style="text-align:right;"> -0.0001472 </td> <td style="text-align:right;"> 0.0000431 </td> <td style="text-align:right;"> -3.416029 </td> <td style="text-align:right;"> 0.0009722 </td> </tr> </tbody> </table> R<sup>2</sup> = 0.3235187 Note that additive coefficients signify the effect of one predictor at the average level of all others. --- ## Interpretation - What the heck does a continuous interaction effect mean? -- - We can look at the effect of one variable at different levels of the other -- - We can look at a surface -- - We can construct *counterfactual* plots showing how changing both variables influences our outcome --- ## Age at Different Levels of Elevation <img src="interactions_lm_files/figure-html/int_visreg-1.png" style="display: block; margin: auto;" /> --- ## Elevation at Different Levels of Age <img src="interactions_lm_files/figure-html/int_visreg_2-1.png" style="display: block; margin: auto;" /> --- ## Or all in one plot <img src="interactions_lm_files/figure-html/keeley_int_pred-1.png" style="display: block; margin: auto;" /> --- ## Without Data and Including CIs <img src="interactions_lm_files/figure-html/keeley_int_pred_nodata-1.png" style="display: block; margin: auto;" /> --- ## A Heatmap Approach <img src="interactions_lm_files/figure-html/int_heatmap-1.png" style="display: block; margin: auto;" /> --- ## Surfaces and Other 3d Objects
--- class: center, middle ![](images/interactions/main-effects-without.jpg)