The simplest relationship between an independent variable and a dependent variable is a linear relationship.

Many data relationships can be adequately described as a linear relationship.

A linear relationship is a relationship that can be modeled as a straight line.

Many data relationships are not linear and are called nonlinear.

Sometimes nonlinear relationships can be approximated by linear relationships - especially if the range of deviation is small.

Issues arise much more with extrapolation rather than with interpolation.

A straight line has the following form.

• m is the slope
• b is the y-intercept
• x is the independent variable
• y is the dependent variable

For more information on lines, see Linear equations .

An independent variable is assumed not to depend on anything else of concern to the problem. A dependent variable is assumed to depend on the independent variable and, perhaps, other dependent variables.

For example purposes, the following data represent the picture size, in inches, measured diagonally, of TV screens for sale at a retail outlet and the price of that TV.

Is there a relationship between TV size and TV price?

If so, what is the nature of that relationship?

Can the relationship be modeled with a straight line?

In order to do predictions, and as a first approximation, the data can be analyzed using linear regression.

• The size is the independent data value (i.e., x_i).
• The price is the dependent data value (i.e., y_i).

The regression is done on the dependent variable y_i and independent variable x_i.

The next step is to plot the data.

What type of chart is appropriate?

There are (at least) two reasons why a line chart is not appropriate.

First, a line chart has equally spaced horizontal increments, and the independent variable does not have equally spaced increments.

Second, a line chart permits only one dependent value to be plotted for each independent value, and there are several independent values that are the same. List two reasons why a line chart is not appropriate for plotting regression data.

Why is a scatter plot more appropriate than a line graph for displaying dependent versus independent variables?

A scatter chart is appropriate and might appear as follows.

A scatter chart is a way to visually depict the relationship between two variables.

Below is a Lua program for a scatter plot for this data. Here is the Python code [#1]
import matplotlib.pyplot as plt import rsPlot import numpy as np from sklearn.linear_model import LinearRegression dataList1 = [ [13,169], [13,178], [13,139], [19,199], [19,188], [19,238], [20,278], [20,219], [25,347], [25,299], [25,299], [27,467], [27,498], [27,598], [31,697], [32,699], [32,1099], [50,1999], ] xData0 = [] yData0 = [] for Size1,Price1 in dataList1: xData0.append(Size1) yData0.append(Price1) xData1 = np.array(xData0).reshape(-1, 1) yData1 = np.array(yData0).reshape(-1, 1) reg1 = LinearRegression() reg1.fit(xData1,yData1) slope1 = float(reg1.coef_) coef1 = float(reg1.coef_) intercept1 = float(reg1.intercept_) equation1 = "Price = {0:1.1f} * Size + {1:0.2f}".format(slope1,intercept1) print("") print("Regression line:") print("") print("\t{0:s}".format(equation1)) print("") print("\tSlope: {0:0.2f}".format(slope1)) print("\tIntercept: {0:0.2f}".format(intercept1)) print("\tCorrelation coefficient: {0:0.2f}".format(coef1)) print("") print("Predictions:") print("") print("\t{0:>5} {1:>5} {2:>5}-actual".format("Size", "Price", "Price")) for Size1,Price1 in dataList1: y1 = slope1 * Size1 + intercept1 print("\t{0:5.1f} {1:5.1f} {2:5.1f}".format(Size1,y1,Price1)) print("") yPredict1 = reg1.predict(xData1) plot1 = rsPlot.plot("regr2-1",1,0,72,640,480) plt.title("TV Size vs. Price") plt.xlabel("Size") plt.ylabel("Price") plt.scatter(xData1,yData1,color="#990099") plt.plot(xData1,yPredict1,color="#009900",label=equation1) plt.legend(fontsize=10) plot1.save()
Here is the output of the Python code.
(module rsPlot imported) Regression line: Price = 47.4 * Size + -671.61 Slope: 47.37 Intercept: -671.61 Correlation coefficient: 47.37 Predictions: Size Price Price-actual 13.0 -55.8 169.0 13.0 -55.8 178.0 13.0 -55.8 139.0 19.0 228.3 199.0 19.0 228.3 188.0 19.0 228.3 238.0 20.0 275.7 278.0 20.0 275.7 219.0 25.0 512.5 347.0 25.0 512.5 299.0 25.0 512.5 299.0 27.0 607.3 467.0 27.0 607.3 498.0 27.0 607.3 598.0 31.0 796.7 697.0 32.0 844.1 699.0 32.0 844.1 1099.0 50.0 1696.7 1999.0 ./regr2-1-01.png : 601 x 462 , 26,885 bytes
Notice that the intercept is negative! This is a nonlinear feature of the model for this data.

Here is the regression chart.

Note that since there is only one dependent variable, and since a legend would repeat information on the vertical axis, a legend is not needed.

Fitting a straight line to a set of data is often called curve fitting, and the best curve is a straight line, since linear relationships are easy to work with quantitatively.

Unfortunately, most relationships in the real world are nonlinear in nature.

A regression equation is an equation that defines the relationship between independent and dependent variables.

Many regressions can be approximated, at least initially, with straight lines.

In determining a regression equation for a linear relationship, the least squares principle minimizes the sum of the squared error (i.e., the difference between the actual and predicted y values).

A qualitative least squares method would determine a best fit of a line to data points by plotting the points by hand and drawing a line that seems to best fit the data. That is, approximating a line through the data points that appears to minimize the error.

Quantitatively, the regression line should minimize the square of the error, where the error is the distance from the actual (measured) value to the predicted (model) value.

Why is the error squared?

The error is squared to make all errors positive and to weight larger errors more than smaller errors. What are the two primary reasons for squaring the error terms in a regression model?

The term regression comes from the fact that the data values tend towards, or regress to, the mean. If n data points (x_i, y_i) are collected, then a linear regression model will attempt to find slope m and intercept b such that the (linear) equation

minimizes the sum of the squares of the errors.