Learn Simple Regression

Click on the graph to create data

The following exercises have been carefully designed to help user's learn regression using this interactive tool. Working through these exercises will give user's a more intuitive feel for regression! Each lesson should take 10-15 minutes.

Line Confidence Interval

Lesson Goal

This lesson is dedicated to better understanding confidence regions and understanding the relationship between confidence regions and statistical significance.

Instructions

Case Study

Imagine you are working for a company X that is deciding whether or not to invest more money in advertising as a means of generating more sales. They are only willing to invest more money into advertising if there is a statistically significant relation between advertising and revenue at the 99% level.

Create a data set with two properties:

1) A horizontal line (the mean line) can be drawn inside the 99% confidence region.

2) A horizontal line can not be drawn inside the 95% confidence region.
Getting Started

For the following exercise, you may use a case study of your own or use the one provided.
- Click on (x, y) graph to create a half-dozen data points with an upward slope.
- Uncheck the “Point” confidence region box.
- Click on “99%” or “95%” to see the corresponding line confidence region.
- Click on “Am I Right” when you think you have the right answer.
- Click on “Start Over” to erase all data and start over.
ANSWER: Denoted by two asterisks after the slope t-stat, statistical significance at a 95% level means two things:

1. You cannot draw a horizontal line inside the 95% confidence interval. In other words, you reject the hypothesis that the line is flat (no relationship between X and Y) at a 95% level of confidence (5% Type I error).

2. You can draw a horizontal line inside the 99% confidence interval. In other words, you cannot reject the hypothesis that the line is flat at a 99% level of confidence (1% Type I error).

t-Statistic

Lesson Goal

Understand the relationship between the t-statistic and statistical significance. Learning how outliers affect statistical significance.

Instructions

Create a data set with two properties:

1) The t-statistic of the slope (green box) has one asterix after it.

2) Use 4-8 observations (number of data points)

Working with the data set:

3) Place a significant outlier into the data set.

4) Oh-No! The t-statistic doesn't have an asterix! Add data points along the original line until the t-statistic slope has one asterix again. Keep track of how many new observations it takes!
ANSWER: A line is statistically significant at the 90% level if 9 out of 10 data point samples from the same population will produce a regression line within the 90% confidence region (shaded area). In other words, A line is statistically significant at the 90% level if you can reject the null hypothesis that the line is flat. With one asterix after the t-stat, the line is statistically significant at a 90% level because the confidence interval does not include a flat line (no relationship between X and Y). It is not significant at the 95% level because you can draw a flat line inside the 95% confidence interval.

One-tail Vs Two-tail

Lesson Goal

To understand the difference between one tail and two tail tests.

Instructions

Case Study

Let's say you are the quality control manager for factories A and B, each of which produces light bulbs. Factory A claims that their light bulbs last longer than their competitors. Factory B claims that there is no difference between the competitions lightbulbs. Your job is to test each of these individual claims. Which claim corresponds to a one-tail test? How about two-tailed?

Create a data set for each of the following conditions:

1) You reject both Factory A + B's claim at the 95% level.

2) You accept Factory A's claim at the 95% level, but reject Factory B's claim 95% level.

Point Confidence Interval

Lesson Goal

Understanding outliers as they relate to confidence intervals.

Instructions

Uncheck the boxes denoting "line" confidence interval Make sure that the 95% confidence level and "Two-tail" boxes are checked.

Create a dataset with two properties

1) You can not place the red-dotted line with the 95% confidence region

2) You have one outlier that exists outside the 95% point confidence region.
ANSWER: An "outlier" far from the regression line decreases the precision with which points are predicted by the regression line. This is reflected in a larger point confidence interval.

R-Squared

Lesson Goal

Understand how different data sets produce different R-squared values

Background

R-squared is meant to measure how much of the variance in the dependent variable is explained by the independent variable.

Instructions

Create 4 separate data sets with the respective R-squared values. Use at least 10 observations for each category.

1) .00 < R² < 0.25

2) .25 < R² < 0.50

1) .50 < R² < 0.75

1) .75 < R² < 1.00

What major differences did you see between categories 1 and 4?

ANSWER: There is a lot less variability among data points for Category 4.

Value and Standard Error

Lesson Goal

To understand how the value of the slope and the standard error relate to statistical significance.

Instructions

1) Create a data set of at least 7 observations where the value of the slope minus the standard error of the slope is negative. (value - standard error) < 0

What do you notice about the t-statistic of the slope?

ANSWER: The t-statistic has no asterix

2) Create a date set of at least 7 observations where the value of the slope minus the standard error of the slope is positive. (value - standard error) > 0

What do you notice about the t-statistic of the slope?

ANSWER: The t-statistic has at least one asterix

Trial and Standard Error: Learn Simple Regression

Simple
Regression

Differences
Between
Groups

Functional
Forms

Reverse Regression x=a+b*y

Confidence Interval

Line Point

Two Tail

One Tail

99%***

95%**

90%*

Show Mean Line

Residuals i Residuals are the differences between observed values and the values predicted by a regression model. They are used to assess the fit of the model, with smaller residuals indicating a better fit.

Show

Plot

	Intercept i The intercept is the value at which the regression line crosses the y-axis in a linear regression model. It represents the predicted value of the dependent variable when all independent variables are zero.	Slope i The slope is a measure in a linear regression that represents the expected change in the dependent variable for a one-unit change in the independent variable. It indicates the strength and direction of the relationship between variables.
Value:
Std. Error: i The standard error measures the precision with which a sample mean estimates the true population mean. It indicates how much the sample mean is expected to vary from sample to sample.
t-Statistic: i The t-statistic is a ratio that compares the difference between the sample mean and the population mean relative to the variability in the sample data. It is used in a t-test to determine whether to reject the null hypothesis.


Correlation:
R-Squared:
Observations:

© 2023 Luke M. Froeb, Keyuan Jiang & Gray Curtis