probability and statistics hypothesis testingvrt0020/statistics/chapter08... · 2019. 12. 2. ·...

Probability and StatisticsHypothesis Testing

Adéla Vrtková, Martina Litschmannová

Department of Applied Mathematics, V�B-TU Ostrava

Adéla Vrtková, Martina Litschmannová Hypothesis Testing 1 / 47

Course requirements

attendance - maximum 3 absences

4 tests during semester - required minimum 15 points, maximum 33points

�nal exam test - 5 exercises, maximum 67 points

total - minimum 51 points, maximum 100 points

Contact

adela.vrtkova[at]vsb.cz

CPIT building, 3rd �oor, o�ce no. A310

homel.vsb.cz/∼vrt0020

Terminology - revision

population - all units

sample - part of the population

statistical inference - process of generalizing from a sample to apopulation, includes con�dence intervals and hypothesis testing

population parameter - characteristic of population, e.g. populationmean µ, population proportion π

sample statistic - calculated from the data in the sample, e.g.sample mean µ̂, sample proportion π̂

Statistical hypothesis... is an educated guess about something in the world around you, anassumption about population, e.g. theory about the value of thepopulation mean. It re�ects researcher's scienti�c theory translated in"statistical language".

Hypothesis testing... refers to the formal procedures to reject or fail to reject the statisticalhypothesis.

Hypothesis Testing vs Con�dence Intervals

A con�dence interval enables to estimate a population parameterwith respect to the uncertainty (variability).

Hypothesis testing enables to verify (or disprove) whether collecteddata don't interfere with our theory about population.

So, if I have a theory about the population, consequently I want to �ndout, if my sample data support it, right? Therefore, I will use thehypothesis testing tools.If I don't have any theory and still want to �nd out something about thepopulation, I will use the tools of the estimation theory (e.g.con�dence intervals).

Some examples

The mean acidity of the local river A is below 7 pH.

The mean acidity of the local river A is greater than the mean acidityof the local river B.

The means of the acidity levels of three local rivers A, B and C di�er.

The acidity is normally distributed.

Parametric vs Nonparametric

Parametric hypotheses are hypotheses about parameter, e.g. aboutpopulation mean, about equality of variances, etc.

Nonparametric hypotheses relate to other properties, e.g. type ofdistribution, independence of variables, etc.

Parametric test makes assumptions about the distribution, e.g.normality. Performing parametric test without checking theassumptions can lead to misleading results.

Nonparametric test makes no such assumptions.

One-sample vs Two-sample vs Multiple-sample tests (forparametric hypotheses)

One-sample tests compare a population parameter with a knownvalue.

Two-sample tests compare two population parameters with eachother.

Multiple-sample tests check the equality of more than twopopulation parameters.

Some examples

One-, two- or multiple-sample problem?

The mean acidity of the local river A is below 7 pH.

The mean acidity of the local river A is greater than the mean acidityof the local river B.

The means of the acidity levels of three local rivers A, B and C di�er.

The acidity is normally distributed.

Null hypothesis and Alternative hypothesis

null hypothesis (H0) - general statement, default position, expressthe absence of any relationship or di�erence, populationparameter is equal to speci�ed value, two population parameter areequal to each other, data are normally distributed, etc...

alternative hypothesis (HA) - opposite of the null hypothesis (but notnecessarily logical negation), has to be exclusive of null hypothesis,expressing the presence of some relationship or di�erence, it canbe two-sided or one-sided (population parameter is greater thanspeci�ed value, one pop. parameter is di�erent from the otherpopulation parameter, data are not normally distributed, etc...)

Verifying the null hypothesis - example

Researchers think that the mean level of blood cholesterol in thepopulation of the Czech Republic is 4,7 mmol/l.

H0 : µ = 4, 7

HA : µ 6= 4, 7

So what?

H0 : µ = 4, 7

HA : µ 6= 4, 7

So what?

H0 : µ = 4, 7

HA : µ 6= 4, 7

So what?

Researchers think that the average level of blood cholesterol in thepopulation of the Czech Republic is 4,7 mmol/l.

H0 : µ = 4, 7

HA : µ 6= 4, 7

We collect data from 100 randomly chosen Czechs to see if theysupport or disprove the hypothesis about the population.

We compute the sample mean with the result µ̂ = 5, 4 mmol/l.

Well, if the true population mean is really 4,7 mmol/l (which we donot know), then the sample mean should be somewhere around it,right? We de�nitely would not expect a value of the sample meanequal to let's say 20 mmol/l...

H0 : µ = 4, 7

HA : µ 6= 4, 7

We collect data from 100 randomly chosen Czechs to see if theysupport or disprove the hypothesis about the population.

We compute the sample mean with the result µ̂ = 5, 4 mmol/l.

Well, if the true population mean is really 4,7 mmol/l (which we donot know), then the sample mean should be somewhere around it,right? We de�nitely would not expect a value of the sample meanequal to let's say 20 mmol/l...

H0 : µ = 4, 7

HA : µ 6= 4, 7

So, if obtained result goes "convincingly" against the null hypothesis,then it implies that the null hypothesis can't be true.

If the data are not that persuasive against the null hypothesis, thenthe null hypothesis can not be rejected. It does not mean that it istrue, it could be, but it is also possible that we just do not haveenough evidence in the data.

In other words, we (the math background) assume that the nullhypothesis is true, and if our data give something which is veryunlikely, therefore, we obtained evidence against the null hypothesis.

Verifying the null hypothesis?

Null hypothesis can't be accepted or proved! We can only reject it infavour of the alternative or not reject it.

Verifying the null hypothesis?

Hypothesis Testing

Let's summarize our thoughts.

The null hypothesis and the alternative hypothesis result from ourtheories, our assumptions, our subject of analysis.

The null hypothesis can be rejected in favour of the alternative or notrejected.

The null hypothesis determines a speci�c test to be used.

Parametric tests have some assumptions about the type of thedistribution which have to be met. Otherwise, nonparametric testsshould be used.

Can we make a mistake?

Type I error is denoted as α. It is more important to us, therefore α is �xedin advance (usually 0,05 or 0,01).

Type II error is denoted as β.

Can we in�uence mistakes?

We want low α and β. Unfortunately, if one decreases, the otherincreases. Hence, compromise is necessary.

Both mistakes decrease with larger sample size.

The power is in�uenced by the choice of the particular test and bysample size.

So how do we decide about the null hypothesis?

After performing the right test in JASP/R, the output will give usp-value.

If p-value < α, then the null hypothesis is rejected. If p-value ≥ α,then the null hypothesis can't be rejected. The value α is usually 0,05.

P-value is low, null hypothesis must go!

Hypothesis testing - from beginning to end

1 State the null hypothesis and the alternative hypothesis.

2 Think about possible tests and their assumptions.

3 Test the assumptions and choose the right test.

4 Perform it correctly in the software.

5 Look at the p-value.

6 Make �nal decision.

I p − value < α...The null hypothesis is rejected in favour of thealternative.

I p − value ≥ α...The null hypothesis can't be rejected.

7 Interpret the results.

Hypothesis testing - normality

Now, when we know the process of hypothesis testing, why don't we use itto test the normality-assumption?

with tools like QQ-plot, skewness and kurtosis we only guessedwhether the normality is violated

with Shapiro-Wilk test we can make certain decision

after performing the Shapiro-Wilk test, if p − value < α, we rejectthe normality assumption

One-Sample tests - comparing one population parameterwith one speci�c value

Null Hypothesis Assumptions Test JASP/R

H0 : µ = µ0 normality One-sample Student t-test T-Tests/t.test(. . . )

H0 : x0,5 = x00,5 symmetry One-sample Wilcoxon test T-Tests/wilcox.test(...)

H0 : π = π0 n > 9

π̂(1−π̂)One-sample Binomial test Frequencies/binom.test(...)

data are normally distributed Shapiro-Wilk test Descriptives/shapiro.test(. . . )

Possible alternatives (e.g. for H0 : µ = µ0)

HA : µ 6= µ0 .... two-sided

HA : µ < µ0 .... one-sided

HA : µ > µ0 .... one-sided

The choice of the alternative depends on the research question, for other parameters the alternative looksanalogously. The parameter with 0 index (i.e. µ0, x

0,5, π0) represents some speci�c value.

One sample tests - example 1Manufacturer claims that the mean capacity of all of his batteries is greater than 1750mAh. For veri�cation of the claim, 95 batteries were randomly chosen as a sample. Isthere enough evidence that the mean capacity of the batteries is truly greater than 1750mAh?

1. State the null hypothesis and the alternative hypothesis.

H0 : µ = 1750

HA : µ > 1750

µ stands for the (population) mean capacity of all batteries in the production

2. Think about possible tests and their assumptions.

I Student's t-test (normality; it tests the mean, our Plan A)I Wilcoxon signed-rank test (nonparametric alternative to Student's t-test, it

tests the median, use it in case of normality violation, our Plan B)

3. Test the assumptions and choose the right test.

I outliers XI skewness, kurtosis XI histogram XI QQ-plot XI Shapiro-Wilk test X

H0 : µ = 1750

HA : µ > 1750

H0 : µ = 1750

HA : µ > 1750

H0 : µ = 1750

HA : µ > 1750

One sample tests - example 1

3. Test the assumptions and choose the right test - let's say, that the normalityassumption is not violated → we choose the Student's t-test.

4. Perform it correctly in the software - T-Tests - One Sample T-Test - Student'sTest + setting the Test value and alternative.

5. Look at the p-value (p).

6. Make the �nal decision.

If p-value is below α = 0.05, the null hypothesis is rejected in favour of thealternative. It would mean that manufacturer's claim was justi�ed. The meancapacity of all of his batteries is truly greater that 1750 mAh.

If p-value is above or equal to α = 0.05, the null hypothesis can not be rejected. Itwould mean that the manufacturer's claim was not justi�ed and that we can notsay that the mean capacity is greater than 1750 mAh.

Let's use the same example as in the previous slides. However, this time, let's assumethat the decision about the normality will be di�erent...

Manufacturer claims that the mean capacity of all of his batteries is greater than 1750mAh. For veri�cation of the claim, 95 batteries were randomly chosen as a sample. Isthere enough evidence that the mean capacity of the batteries is truly greater than 1750mAh?

H0 : µ = 1750

HA : µ > 1750

H0 : µ = 1750

HA : µ > 1750

H0 : µ = 1750

HA : µ > 1750

H0 : µ = 1750

HA : µ > 1750

3. Test the assumptions and choose the right test - let's say, that the normalityassumption is violated → Wilcoxon test, it tests the median, our de�nedhypotheses change because we can not work with the mean (going for Plan B).

H0 : x0.5 = 1750

HA : x0.5 > 1750

4. Perform it correctly in the software - T-Tests - One Sample T-Test - Wilcoxon +setting the Test value and alternative.

If p-value is below α = 0.05, the null hypothesis is rejected in favour of thealternative. It would mean that manufacturer's claim was justi�ed. The mediancapacity of all of his batteries is truly greater that 1750 mAh.

If p-value is above or equal to α = 0.05, the null hypothesis can not be rejected. Itwould mean that the manufacturer's claim was not justi�ed and that we can notsay that the median capacity is greater than 1750 mAh.

One sample tests - example 3Company's management aims to verify their theory that less than 60% of theiremployees prefer the home-o�ce system. The company has many employees so theyasked only a random sample of 250 employees. Out of them, 137 con�rmed that theyprefer the home-o�ce system. Do the collected data support the management's theory?

H0 : π = 0.6

HA : π < 0.6

π stands for the (population) proportion of all employees who prefer thehome-o�ce system

I One-sample Binomial test (n > 30, n > 9/π̂(1− π̂))3. Test the assumptions - n = 250, π̂ = 137/250

4. Perform it correctly in the software - Frequencies - Binomial test + setting theTest value and alternative, or binom.test(...)

H0 : π = 0.6

HA : π < 0.6

H0 : π = 0.6

HA : π < 0.6

I One-sample Binomial test (n > 30, n > 9/π̂(1− π̂))

3. Test the assumptions - n = 250, π̂ = 137/250

H0 : π = 0.6

HA : π < 0.6

H0 : π = 0.6

HA : π < 0.6

Company's management aims to verify their theory that less than 60% of theiremployees prefer the home-o�ce system. The company has many employees so theyasked only a random sample of 250 employees. Out of them, 137 con�rmed that theyprefer the home-o�ce system. Do the collected data support the management's theory?

If the p-value is above or equal to α = 0.05, it means that the null hypothesis cannot be rejected. It would mean that the company's survey did not provide enoughevidence to support their theory.

If the p-value is below α = 0.05, it means that the null hypothesis is rejected infavour to the alternative. It would mean that the company's survey did provideenough evidence to support their theory.

Two-Sample tests - comparing two population parameters

H0 : µ1 = µ2

independent samples

normality

EQUAL variances

Two-sample t-test (Student) T-Tests/t.test(. . . , var.equal=T)

H0 : µ1 = µ2

independent samples

normality

UNEQUAL variances

Welch test T-Tests/t.test(. . . , var.equal=F)

H0 : x10,5 = x2

0,5 symmetry Mann-Whitney test T-Tests/wilcox.test(...)

Possible alternatives (e.g. for H0 : µ1 = µ2)

HA : µ1 6= µ2 .... two-sided

HA : µ1 < µ2 .... one-sided

HA : µ1 > µ2 .... one-sided

The choice of the alternative depends on the research question, for other parameters the alternative looksanalogously.

Independent samples = whether the measurements were made on two di�erent sets (groups) of items (objects,

units). The opposite is paired measurements which refers to a situation when the measurements are performed

repeatedly on the same set (group) of items.

Two-sample tests - example 1Manufacturer implemented new technology for the production of batteries. He randomlychose 100 batteries from the old production and 100 batteries from the new productionand after 1000 charging cycles he measured their capacity. Do the data provide enoughevidence to support the manufacturer's theory that the implementation of the newtechnology caused increase in batteries capacity after 1000 charging cycles? In otherwords, do the data provide enough evidence to say that the (population) mean capacityfor the new batteries is greater than the (population) mean capacity for the old ones?

H0 : µold = µnew

HA : µold < µnew

I Two-sample t-test (independent samples, normality, equal variances)I Welch test (independent samples, normality, unequal variances)I Mann-Whitney test (independent samples, symmetry, used if normality

violated, tests equality of medians)

H0 : µold = µnew

HA : µold < µnew

H0 : µold = µnew

HA : µold < µnew

H0 : µold = µnew

HA : µold < µnew

Two-sample tests - example 1

3. Test the assumptions and choose the right test - T-tests - Independent SamplesT-tests - Assumption Checks. If the normality is not violated, we decide btwTwo-sample t-test (Student) and Welch test (based on the result of the test forthe eq. of variances) - both test the equality of means against speci�edalternative, so the following process of the �nal decision works in the same way forthese two tests.

4. Perform it correctly in the software - T-tests - Independent Samples T-tests +setting the Test and alternative

If p is below α = 0.05, it suggest that the null hypothesis about the equality of means isrejected in favour of the alternative. It would mean that the (population) mean capacityfor the old batteries is signi�cantly lower than the (population) mean capacity for thenew batteries (after 1000 charging cycles). That would imply that the data truly providethe evidence that the new technology increased the capacity of batteries after 1000charging cycles.

If p is above (or equal to) α = 0.05, it suggest that the null hypothesis about the

equality of means is not rejected. That would imply that the data did not provide the

evidence that the new technology increased the capacity of batteries after 1000 charging

cycles.

Remember - the null hypothesis is always about the absence of anyrelationship/di�erence (parameter is equal to some value, parameters areequal to each other, etc.)!

H0 : µold = µnew

HA : µold < µnew

H0 : µold = µnew

HA : µold < µnew

H0 : µold = µnew

HA : µold < µnew

H0 : µold = µnew

HA : µold < µnew

3. Test the assumptions and choose the right test - T-tests - Independent SamplesT-tests - Assumption Checks. This time, let's assume, that the normality is

violated, Mann-Whitney test must be used. However, it tests the equality ofmedians, so formally, the hypotheses should be changed.

H0 : xold0.5 = xnew

HA : xold0.5 < xnew

4. Perform it correctly in the software - T-tests - Independent Samples T-tests +setting the Test and alternative

If p is below α = 0.05, it suggest that the null hypothesis about the equality of mediansis rejected in favour of the alternative. It would mean that the (population) mediancapacity for the old batteries is signi�cantly lower than the (population) median capacityfor the new batteries (after 1000 charging cycles). That would imply that the data trulyprovide the evidence that the new technology increased the capacity of batteries after1000 charging cycles.

If p is above (or equal to) α = 0.05, it suggest that the null hypothesis about the

equality of medians is not rejected. That would imply that the data did not provide the

evidence that the new technology increased the capacity of batteries after 1000 charging

cycles.

Multiple-Sample tests - comparing more populationparameters

H0 : µ1 = µ2 = ... = µk

independent samples

normality

EQUAL variances

ANOVA ANOVA

H0 : x10,5 = x2

0,5 = ... = xk0,5 symmetry Kruskal-Wallis (non-parametric ANOVA) ANOVA

The alternative hypothesis states, that the equality of means or medians is violated. However, it does not provide

any more speci�c information. So, in case of rejection of the null hypothesis in multiple-sample tests, it is necessary

to perform the post-hoc analysis to �nd out which means or medians are signi�cantly di�erent.

Exercises - only One- and Two-sample problems

Manufacturer claims that the mean capacity of all of his batteries is greaterthan 1750 mAh. For veri�cation of the claim, 95 batteries were randomlychosen as a sample. Is there enough evidence that the mean capacity ofthe batteries is truly greater than 1750 mAh?

Use the datasets batteries1.csv and batteries2.csv (separately, �rstly, solvethe exercise with the �rst one and do the same with the second). Becareful about the analysis of assumptions.

Use slides 23-26 as inspiration.

Company's management aims to verify their theory that less than 60% oftheir employees prefer the home-o�ce system. The company has manyemployees so they asked only a random sample of 250 employees. Out ofthem, 137 con�rmed that they prefer the home-o�ce system. Do thecollected data support the management's theory?

Manufacturer implemented new technology for the production of batteries.He randomly chose 100 batteries from the old production and 100 batteriesfrom the new production and after 1000 charging cycles he measured theircapacity. Do the data provide enough evidence to support themanufacturer's theory that the implementation of the new technologycaused increase in batteries capacity after 1000 charging cycles? In otherwords, do the data provide enough evidence to say that the (population)mean capacity for the new batteries is greater than the (population) meancapacity for the old ones?

Use the datasets batteriesA.csv, batteriesB.csv and batteriesC.csv(separately, �rstly, solve the exercise with the �rst one and do the samewith the others). Be careful about the analysis of assumptions.

The solution must include

the hypotheses (null and alternative)

analysis of the assumptions

the name of the chosen test

the p-value and the �nal decision about the null hypothesis

interpretation regarding the original question

probability and statistics hypothesis testingvrt0020/statistics/chapter08... · 2019. 12. 2. ·...

Documents