l stoodialinza
← All topics

Step 3 · Methods · Topic 10 of 17 · 8 min

Regression and correlation

How Y depends on X, and how closely the points follow the line.

In short#

  • Linear regression Y = β₀ + β₁X + ε estimates an intercept and slope by least squares. Usual confidence intervals and tests assume a linear relationship, independent observations and constant residual variance. Normally distributed residuals matter for exact inference in small samples.
  • For a linear model with an intercept, R² describes the share of total variation in Y around its mean accounted for by the line. A high R² alone does not validate linearity or the model.
  • Pearson's correlation r describes linear association. Spearman's rank correlation ρ describes monotonic association by ranks, which can be nonlinear.
  • Correlation does not establish causation. Extrapolation beyond the observed X is unreliable.

In plain words#

The slope is the expected average change in Y for a one-unit increase in X. For example, it describes how a sensor signal changes on average as the reference input changes. This describes an association; it does not establish a causal effect.

Pearson correlation describes linear association: r = 1 corresponds to a perfect increasing line, r = −1 to a perfect decreasing line, and r = 0 means no linear correlation. A nonlinear association may still be present.

Inspect the residual plot. A systematic arc means the linear model does not adequately describe the mean relationship, even when R² is high.

In Linza#

  1. Create an XY table: each row holds the paired numeric X and Y values for one observation in one series.
  2. In Analyses, choose linear regression, Pearson, Spearman, logistic regression or ROC/AUC. XY diagnostics show residuals for regression.
  3. Open the sensor calibration example. Its data is already loaded and ready to inspect.
  4. For two or more series, Linza first tests whether their slopes differ. If no difference is detected, it compares the lines' vertical positions under a common slope. Failure to detect a difference does not prove that the slopes are equal.
  5. The comparison assumes a linear relationship within each series, independent observations, common residual variance and no measurement error in X. The sequential test p-values are unadjusted.
Try it in Linza Opens in a new tab with synthetic data. Work in the original tab remains unchanged.

Common mistakes#

  • Treating a high R² as proof of a linear relationship.
  • Inferring cause from correlation in observational data.

Check yourself#

Spearman ρ = 0.95 while Pearson r = 0.70. What does this most likely mean?

Optional. Your answer is not saved.

Practice exercise: What is the linear regression slope? →

Sources#

  • Anscombe F.J. (1973) Graphs in statistical analysis. The American Statistician, 27(1). Open publication Opens in a new tab
  • Altman N., Krzywinski M. (2015) Simple linear regression. Nature Methods, 12. Search for publication Opens in a new tab
  • Spearman C. (1904) The proof and measurement of association between two things. American Journal of Psychology, 15(1). Search for publication Opens in a new tab

References are for further reading. A method’s suitability depends on the study design and its assumptions.