1

Business Statistics

Lecture 8: More Hypothesis

Testing

Goals for this Lecture

• Review of t-tests

• Additional hypothesis tests

• Two-sample tests

• Paired tests

2

3

The Basic Idea of Hypothesis Testing

• Start with a theory or hypothesis

• For example, m = 814.3

• Collect some data

• Ask: How unusual is it to see this data if the null hypothesis is true?

• If it’s unusual, reject the null hypothesis

• If not, fail to reject the null

• Remember, determine the hypothesis to be tested before looking before looking at the data

4

It All Ties Back to the Empirical Rule

• If we hypothesize that the data come from a N(0,1)

distribution, how unusual an observation must we see to

reject our hypothesis?

It depends on the alternative hypothesis…

-4 -3 -2 -1 0 1 2 3 4

Z

68%

95%

5

For Example, a Two-sided Test

-4 -3 -2 -1 0 1 2 3 4

Z

68%

95%

Null: The mean is equal to zero (H0: m = 0)

Alternative: The mean is not equal to zero (Ha: m ≠ 0)

If the rejection criterion is p-value < 0.05, we reject if our

observation is greater than 1.96 or less than -1.96:

6

In JMP

• JMP computes the probability of seeing

data as extreme or more extreme under

various alternate hypotheses

• You have to choose the appropriate p-value

• Then compare the JMP p-value to 0.05

• Smaller: reject the null

• Larger: fail to reject the null

• Output is in terms of rescaled “t-scores”

• Using t distribution comes from using s to

estimate s

Conducting the Test in JMP

• With one continuous variable, Analyze >

Distribution > red triangle > Test Mean

• Type in the mean to be tested (“Specify

Hypothesized Mean”)

• If population (“true”) standard deviation

known, enter it

• This will be a z-test

• If you leave it blank, JMP does a t-test

• It uses s to estimate s

7

8

Back to the Paint Case (primer.jmp)

• A More Complicated Question:

• Suppose we are less interested in the value of 1.2 and more interested in whether processes “a” and “b” have the same mean

• Null hypothesis

• Means are the same: ma- mb = 0

• Alternative hypothesis

• Means are different: ma- mb 0

9

Solution: Two-sample t-test

Process “a”Process “b”

X Y

Mean = mx

SD = sx

Mean = my

SD = sy

• Two sample t-test assumes Xs

and Ys are independent

X1, X2, …, XnY1, Y2, …, Ym

Random Samples

10

• What do you think the test statistic is?

• How should we rescale the test statistic?

• What does the p-value represent?

Results of Two Sample t-test

• Null Hypothesis: mx- my = 0

• Test Statistic:

• Fact: since and are independent:

• So

X Y

11

YX

)()()( YVarXVarYXVar

mn

yx

22 ss

22

( )yxSE X Y

n m

ss

Two-sample t-test

• Test statistic:

• Estimated standard error:

• Rescaled test statistic:

12

X Y

22yx

x y

ss

n n

22

0

yx

x y

X Yt

ss

n n

Rescaled Test Statistic

13

• For some test statistic T where m and s

are not known, compute

where

• m * is the hypothesized true value

• sT is the sample standard error of the

statistic T

Remember: Rescaling

*

T

Tt

s

m

14

• In a one-sample test of, choose m*

• Then T = , so the test statistic is

• In a two-sample test, you’re often

testing whether the means are equal

• T = , and the test statistic is

One-sample and Two-sample Tests

* *

. .( ) . .( )

T Xt

s d T s e X

m m

* ( ) 0 ( )

. .( ) . .( ) . .( )

T X Y X Yt

s d T s e X Y s e X Y

m

X

YX

• We must estimate sx and sy

• If sx = sy then we can get a better

estimate

• Remember: Sample variance for a

single sample:

15

n

j j xxn

s1

22 )(1

1

Sample mean

Deviations from sample meanAverage squared deviation

from the mean

Equal Variances?

16

• Remember, SD is

calculated using

differences from

the mean

• Each group can

have very different

mean but standard

deviations can be

similar

Different Means But Similar SD

-3

-2

-1

0

1

2

3

4

5

6

• Pooled estimate of sample variance:

17

2 2

1 12( ) ( )

( 1) ( 1)

n m

j jj j

p

x x y ys

n m

Sample mean for process a

Sample mean for process b

Used two degrees of freedom, n+m-2 left over

• Pooled estimate buys you more df

• Weighted average of and 2

xs 2

ys

Average squared deviation from different means

More Bang for the Buck

Conducting the Test in JMP

• Need two variables: one continuous and one

categorical (denoting group)

• Then: Analyze > Fit Y by X (continuous

variable is the Y and categorical the X) > red

triangle > Means/Anova/Pooled t

• See the “t Test” part of the output

18

Case: Taste Testing Teas

19

• Small taste test of teas (taste.jmp)

• 16 panelists in a focus group

• Each tasted two formulations of a

prepackaged iced tea

• Rated them on a scale of 1 (excellent) to 7

(really bad)

• Company wants to know if there is a

difference in ratings between the two

formulations

20

• Two-sample t-test on taste.jmp:

• Is there a

significant

difference?

An Initial Evaluation

21

Taste Case: Any Difference?

• Unless SD’s vastly different (factor of 2), the

equal variance assumption no big deal

22

Independence Assumption

Very Important

• Independence assumption for two

sample t-test is violated

• Good news: there is an alternate test

that can do even better

• Paired t-test assumes two observations

taken for each unit in the sample

• Observations on the same unit likely to be

more similar than obs’ns on different units

• Here same person tasted each formulation

23

Paired t-test Looks at Differences

x1-y1=d1

x2-y2=d2• .

•

•

xn-yn=dn

• Calculate differences for

each observation

• Calculate sample mean and

SD of differences

• Do a one sample t-test for

differences:

• H0: mean difference is zero

• Ha: mean difference is not 0

24

Paired t-test in JMP

• Use Analyze >

Matched Pairs

• Two variables,

paired by row:

25

Results: Paired t-test in JMP

Mean Difference is same as two sample test

SE is smaller –why??

26

• Heuristic:

• When xj and yj “vary together” then yj will

be big when xj is big

• Since xj & yj tend to be close together, xj-yj

is smaller than when X and Y independent

Why Pairing Helps

• Math:

• When and are not independent thenX Y

( ) ( ) ( ) 2 ( , )Var X Y Var X Var Y Cov X Y

• Cov or “covariance” measures linear

dependence between two variables

27

It Helps in this Case Because…

• People first have a like or dislike for tea

• Their ratings of the formulations are relative to

this overall opinion of tea

• Taking the difference removes the “person

effect”

0

1

2

3

4

5

6

7

Taste

1

0 1 2 3 4 5 6 7 8

Taste 2

Tend to

dislike tea

Tend to

like tea

28

-3

-2

-1

0

1

2

3

4

-3 -2 -1 0 1 2 3

X

-3

-2

-1

0

1

2

3

-3 -2 -1 0 1 2 3

X

• xj-yj is horizontal distance to the y=x line

• xj-yj is smaller (typically) in the right hand plot

Independence vs. Dependence

29

Case: Sales Force Comparison

• Newly merged pharmaceutical company

(PharmSal.jmp)

• Two sales forces (“BW” & “GL”), one from

each of the merged companies

• 20 sales districts are the same

• Sales reps divided into these districts

• Sell essentially the same drugs

• Management wants to know if one sales

force outperforms the other

30

Sales by Division

Sa

les

100

150

200

250

300

350

400

450

500

550

BW GL

Division

BW

GL

Level

112

119

Minimum

151.1

151.6

10%

215.25

197.75

25%

291

313.5

Median

385.5

409.75

75%

428.5

460.6

90%

525

547

Maximum

Quantiles

31

Two-Sample t-test ResultsS

ale

s

100

150

200

250

300

350

400

450

500

550

BW GL

Division

• Under the independence

assumption, we conclude

that there is no difference

in the means

• But are they

independent?

32

The Sales Forces Are Dependent

• Dependence occurs by sales district:

33

Paired t-test Comparison

• Which

sales force

is doing

better?

34

More Complicated Tests

• There are even more complicated tests

you can do

• E.G., test for equal variance

• You’re never going to remember all the

steps for each test anyway

• Let the computer do it for you

35

Terminology

• One-sided vs. two-sided

• Comes from the statement of the alternative hypothesis

• Are you calculating the p-value using one tail or two?

• One-sample vs. two-sample

• Comes from the type of data and the question you are answering

• Are you testing a mean or a difference between means?

36

Which Test?

• How many populations are sampled?• One: one-sample test

• Two: read on

• Are observations in first sample independent of observations in second sample?• Yes: two-sample t-test

• No: paired t-test

• Big Clue:• Paired t-test needs two observations from each

unit• Unequal sample sizes 2 sample test

• Equal sample sizes you have to decide

37

Hypothesis Tests in the Computer Age

• Know the null and alternative

hypotheses

• Have some idea of what test statistics

you would look at

• Let the computer figure out how to

rescale them

• Let the computer figure out the p-value

• p-values are always interpreted the

same way

38

What we have learned so far…

• Descriptive Statistics

• Probability

• Inference for a population mean

• Confidence intervals

• Hypothesis testing

• One-sample test of the mean

• Two-sample tests

• Paired tests