دولة الكويتوزارة التربية

Summative AssessmentEnglish Lang. Workshop

April 2018

Purpose of Workshop

• Develop reliable, valid and discriminating objective assessment items;• Develop test items that align with the subject curriculum

competences and standards;• Undertake pilot testing of the test items using a sample of schools and

students;• Use the training materials provided in the workshop to train Subject

Supervisors and Heads of Departments in each district whenever required;

Review of Item Development Process

True or False Items

True-False Test Items

• Presentation of a statement• Student indicates ”true” or “false”

Applicability

• Basic knowledge level content• Student understanding of popular misconceptions• Concepts with two logical responses

Advantages

• Can test large amounts of content• Students can answer 3-4 questions per minute• Can be written quickly• Easy to score

Disadvantages

• Easy• Difficult to discriminate between students that know content or not• 50-50 chance of getting the right answer by guessing• Need large number of items for high reliability

Notes to Remember

• Simple and clear language• Complicated statements lead to testing reading skill, not content• Use Tick or x Cross• Present correct relationships first, vary second parts• Equal length of responses• Randomize true and false statements• Use popular misconceptions as false statements• Write items with a plausible false response

Example 1

Read the story and then look at the sentences below and Tick (√) all the statements that are True and Cross (x) all the statements that are false.

    True (√) False (x)

a) The boy had blue-green eyes.    b) The boy was dark haired.    c) The boy was fat compared with his brothers.    Xd) The boy was 10 years old.    Xe) The boy was fair skinned.    Xf) The boy was the youngest in his family.    

Group Discussion

• In Kuwait do you currently use this type of item either in the classroom or on the End of Semester Exam? Why or why not?

• What competency does this example measure?• Which competencies could this type of item measure?

Activity 2: Item Development

• In pairs develop 4 True False items • Share with group and select from the items

developed 10 good items• Share the groups 10 best items with another

group• Collect 10 best items from each group

Activity 3: Data

• What data would we be able to collect from a spot test of True/False Items?

F F T T F T T F T T

10 9 8 7 6 5 4 3 2 1

10 F F T T F T T F T T 101

9 F F T T F T T T T T 102

8 T T T T F T T F T T 103

104

Matching Test Items

Matching Test Items

Good for testing:• Knowledge• Comprehension

Types of items:

• Terms with definitions• Phrases with other phrases• Causes with effects• Parts with larger units• Problems with solutions

Advantages

• Maximum coverage at knowledge level in minimum amount of time and space

• Valuable in content areas that contain many facts

Disadvantages

• Time consuming for students• May not be appropriate for higher-level critical thinking skills• Faulty instructions or matrix layout causes confusion

Matching Test Items

• Two or more columns (A-B-C)• Stem and responses• Student finds the correct answer from Column B• Student prints the letter of correct answer beside Column A

Notes to Remember

• Make sure a matching component is appropriate• 5-8 items maximum• Use short phrases or single words in response list• Arrange list of responses in a systematic order• Ensure only one correct response for one stem• Avoid breaking set of items over two pages

Example 1

Bird Description Answer

1. Gannet A. It sings a very sweet song 52. Herring Gull B. It builds large nests 13. Cormorant C. A small bird found in hedges

4

4. Sparrow D. Dives for fish 35. Blackbird E. Eats rubbish 2

From the reading passage match each bird with the correct information by writing the number of the bird in the answer column.

Group Discussion

• In Kuwait do you currently use this type of item either in the classroom or on the End of Semester Exam? Why or why not?

• What competency does this example measure?• Which competencies could this type of item measure?

Activity 4: Item Development

• In pairs develop 1 Matching Items• Share with group and select from the items

developed 2 good items• Share the groups 2 best items with another

group• Collect 2 best items from each group

Activity 5: Data

• What data would we be able to collect from a spot test of Matching Items?

4 3 2 1

16 4 4 4 4 101

15 4 4 3 4 102

14 4 4 3 3 103

104

Fill In the Blanks

Completion/ Fill-In-the-Blank Test Items

• The student answers a question or finishes an incomplete statement

• Correct word or phrase

Applicability

• Basic knowledge level content• Recall and memorization of facts

Advantages

• Who, what, where, when content• Minimizes guessing• Student must know answer no just recognize• Provides an objective measure of student achievement• Can cover wide sample of content

Disadvantages

• Difficult to assess higher level critical thinking skills• Difficult to construct for a clear response• Overemphasizes memorization of facts• Questions may have more than one answer• Scoring is time consuming

Notes to Remember

• Provide clear directions as to what amount of variation will be accepted in the answers

• FIB items worth more marking points that TF or MCQs• Avoid using a long quote with multiple blanks to complete• For numbers, indicate the degree of precision or units expected• Students can write responses on lines arranged in a column to the

left of items

• Marking key should contain correct answer and acceptable alternatives

Example 1

Based on the reading passage above choose one word from the box to complete the sentences below. Use each word only once. Some words will not be used.

a. When the fisherman first fell into the sea, he felt ___________________.b. Once he was under the water, the fisherman was __________________.

frightened angry sad relieved surprised nervous curious ashamed forgiving

Activity 6: Item Development

• In pairs develop 2 Fill in the Blank Items • Share with group and select from the items

developed 10 good items• Share the groups 10 best items with

another group• Collect 10 best items from each group

Activity 7: Data

• What data would we be able to collect from a spot test of Fill in the Blank Items?

Sequencing

Sequencing

• The student puts a series of statements in a correct order• The student writes the number between 1 and xx beside the

statements

Applicability

• Basic knowledge level content• Complex levels of reasoning• Understanding logical order• Following directions

Advantages

• Used for putting events, ideas, and objects in a logical order• Can have pictures, story, or experiment sequencing• Can be fiction and non-fiction• Items can be written quickly and easily• Students can answer fairly quickly

Disadvantages

• If partial marks are awarded then the item may be more difficult to score

• Partial marks can result in a greater weight of importance over other types of items

Notes to Remember

• Make sure that a sequencing component is appropriate• Keep sequencing items brief, limit to 5-6• Language as simple and clear as possible• Reduce the amount of reading time by including only short phrases or single

words in the response list• Avoid breaking a set of items (stems and responses) over two pages• Require students to enter the number corresponding to the item, thus avoiding

having to interpret confusing handwriting.• Randomize the sequence• Write items so that the incorrect response is more plausible or attractive to those

without the specialized knowledge being tested

Example 1

A. The sea bird would not eat anything 5 B. Farah tried to clean the bird’s feathers 4C. A sick bird was found on the rocks 2D. The sea bird was covered with oil. 3E. There was an oil slick spreading across the water 1

The five sentences below are in the wrong order. Put them in order. Do this by writing ‘1’ beside the first, ‘2’ beside the second and so on.

Three ways to score the item

1. Total only - 1 mark if the sequence is totally correct- • Easiest to score • May be appropriate weight on a test that has different

item types that are measuring higher order competencies.

• Penalizes if a student got only one part of the order incorrect

2. Partial credit modelsa. Location only - If all five answers = 5 marks 1 mark for each one in correct spot• Rewards guessing b. Couple responses If all five in correct spot = 5 marksif not then

• If C follows E = 1 mark• If D follows C = 1 mark• If B follows D = 1 mark• If A follows B = 1 mark

A B C D E5 4 2 3 1

3/ 5 marks

1/ 5 marks

1/ 1 marks

A B C D E5 3 2 4 1

Activity 11 Item Development

• In pairs develop 2 Sequencing Items • Share with group and select from

the items developed 5 good items• Share the groups 10 best items with

another group• Collect 10 best items from each

group

Inference

An inference is “reading between the lines” to understand things

not directly stated by the author.

Inferences are based on information stated in text as well as

what is known from personal experience which relates to the

passage being read.

Inference question starters: • From the title/heading, you can

predict that the text will be about …..? • Why do you think …..? • Why did …..? • How do you know that …..? • What might happen if …..? • What can you infer from the picture

on the page? • Why do they …..? How do you know?

• Do you think …..? Why do you think that?

• Why does the author use the word..? • What is the author implying? /What

message is the author trying to give? • Can you infer the meaning of …..? • What probably caused …..? • How might ………….. feel ………….?

Inference : Deduction

Activity 12: Reflection

• Look at each competency in grade 7 English Language Arts

• Which objective item type (MCQ, True/False, Matching, Fill in the Blank, or Sequencing) is best for measuring the competency?

• Give your reason for your choice.