lot quality assurance sampling (lqas). what is lqas a sampling method that: is simple,...

LOT QUALITY ASSURANCE SAMPLING (LQAS)

What is LQAS

A sampling method that: Is simple, in-expensive, and probabilistic .

Combines two standard statistical techniques: stratified random sampling for data collection, and one-sided hypothesis testing for data analysis.

Is increasingly used in monitoring and Evaluation of Health services HIV/AIDS Neonatal Tetanus Elimination Child Survival Immunization coverage Quality of Health centers Contraception

LQAS

Can be used locally, at the level of a “supervision area,” to identify priority areas (e.g., county, sub-county), or indicators that are not reaching average coverage or an

established benchmark

Can provide an accurate measure of coverage or health system quality at a more aggregate level (e.g., program catchment area or district or province)

Can be used for quality assurance using a ‘minimal sample’, ‘maximal security’ principle

Sample size ‘n’ is chosen so that the manager has a high probability of:

accepting lots that meet the quality standards, and

rejecting lots that fail to meet those standards

The decision rule is set based on the desired coverage standards and

statistically determined sample size

Basic LQAS terminologies

• Supervision area: The catchment area that a supervisor wants to assess

• Coverage benchmark: for a predetermined level of coverage that a project aims to reach at a specified time period.

• Average coverage: For the indicator is the no of people in the sample who respondent correctly to a question divided by the total no of people responding to the question.

• Coverage: the proportion of respondents in the SA that show desired effect-the actual outcome

• Decision rule (DR): tells us weather a individual supervision area reaches the average coverage/benchmark or is below the average coverage

Sample size & Decision values• Depend on the risks that investigators are willing to take• Two types of risks

– Risk of accepting a “bad” lot-Type I error– Risk of not accepting a “good” lot-Type II error

• Four steps to decide “n” and decision values:– Determine coverage that would be deemed

unacceptable– Determine coverage that is desired– Determine amount of type I error– Determine amount of type II error

LQAS & Sample size (19? )

• A sample size of 19 provides an acceptable level of precision for making management decisions;

• At least 92% of the time, it identifies whether:– a coverage benchmark has been reached, or

– whether an SA is substantially below the average coverage of a program area

• Little is added to the precision of the measure by using a sample larger than 19, despite the level of coverage being

assessed.

WHY 19?

20 19 18 17 16 15 14 13 12 11 10 9 8

Pr

ecis

ion

100

80

60

40

20

0

Sample sizes less than 19, however, have low precision

Rapid deterioration in the precision of the measure, for n<19

LQAS: By BG BULI/October 2007/UP

8

Why 19? Contd…..

• Good for deciding which are higher performing supervision areas to learn from

• Good for deciding what are the lower performing supervision areas

• Good for identifying knowledge/practices that have high coverage from those of low coverage

• Good for setting priorities among supervision areas with large differences in coverage

• Good for setting priorities among knowledge/practices within an SA

Optimal LQAS Decision Rules Alpha and Beta Errors <10% for All Decision Rules

Coverage→

Sample↓ 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95%

12 1 1 2 2 3 4 5 5 6 7 7 8 8 9 10 11

13 1 1 2 3 3 4 5 6 6 7 8 8 9 10 11 11

14 1 1 2 3 4 4 5 6 7 8 8 9 10 11 11 12

15 1 2 2 3 4 5 6 6 7 8 9 10 10 11 12 13

16 1 2 2 3 4 5 6 7 8 9 9 10 11 12 13 14

17 1 2 2 3 4 5 6 7 8 9 10 11 12 13 14 15

18 1 2 2 3 5 6 7 8 9 10 11 11 12 13 14 16

19 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

20 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16 17

21 1 2 3 4 5 6 8 9 10 11 12 13 14 16 17 18

Source: Valadez JJ, et al (2001)LQAS: By BG BULI/October 2007/UP

10

Steps of LQAS

1.Select the intervention or service to assess

2. Define supervision areas

3. Define communities within supervision areas

4. Define benchmarks

5. Define the level of acceptable error

Percent of women (15-49) who know 2 or more ways to prevent HIV transmission in 5 Supervision Areas?

Step 1. Defining Catchment Area and Supervision Areas

Selection of study subjects

Step 3: Data collection Suppose following is data was collected.

Step 4. Analysis

Add Number Correct in all SAs: 12 + 9 + 16 + 11 +14 = 62 Add all Samples Sizes: 19 + 19 + 19+ 19 + 19 = 95 Knowledge Coverage Estimate = Average Coverage = 62/95 = 65.3% = 70%

Step 5. Use table to find Decision Rule

11

Step 6. Deciding “defect”

Benefits of LQAS

• Low sample size needed (n=19 in most cases)

• Simple to apply yet has very specific conclusions

• Provides high quality information at low & affordable cost

• Fast – ‘supervision areas’ are able to conduct self-evaluation and obtain results immediately after the survey

• Results are locally relevant and can be utilized @ district level planning and decision-making

LQAS: By BG BULI/October 2007/UP

17

Drawbacks of LQAS

LQAS is not cost effective or practical for use in remote rural areas where populations are widely scattered.

LQAS is unable to detect “poor” quality unless it is defined to be much inferior to “good” quality.

LQAS: By BG BULI/October 2007/UP

18

References

1. MEASURE Evaluation. Report of a Technical Meeting on the Use of Lot Quality Assurance Sampling (LQAS) in Polio Eradication Programs. July 1998

2. World Health Organization. Description and comparison of the methods of cluster sampling and lot quality assurance sampling to assess immunization coverage. Department of Vaccines and Biologicals, WHO, Geneva; 2001

3. Valadez JJ, Weiss W and Davis R. A Trainers Guide for Baseline Surveys and Regular Monitoring: Using LQAS for Assessing Field Programs in Community Health in Developing Countries. NGO Networks for Health, Washington DC; 2001

4. Valadez JJ and Devkota BR. Decentralized Supervision of Community Health Programs: Using LQAS in Two Districts of Southern Nepal. In: Management Sciences for Health, Inc. Community-Based Health Care: Lessons from Bangladesh to Boston; 2002.

5. MkkNelly B, Valadez J, Treiber J and Davis R. Considering Applicability of Lot Quality Assurance Sampling (LQAS) to Credit with Education Progress Tracking. California 2001.

LQAS: By BG BULI/October 2007/UP

19