language assessment 4 listening comprehension testing language assessment lecture 4 listening...

18
Language Assessment Language Assessment Lecture 4 4 Listening Comprehensio Listening Comprehensio n Testing n Testing Instructor Instructor Tung-hsien He, Ph.D. Tung-hsien He, Ph.D. 何何何何何 何何何何何 [email protected] [email protected]

Upload: jasper-walton

Post on 26-Dec-2015

227 views

Category:

Documents


0 download

TRANSCRIPT

Language Assessment Language Assessment

Lecture 44Listening Comprehension TListening Comprehension T

estingestingInstructor Instructor

Tung-hsien He, Ph.D.Tung-hsien He, Ph.D.何東憲老師何東憲老師

[email protected]@tea.ntptc.edu.tw

Auditory Skill vs. Oral SkillAuditory Skill vs. Oral Skill

a. Closely Linked in Normal Speech Situationsa. Closely Linked in Normal Speech Situations

b. Listening Abilities May Be Developed b. Listening Abilities May Be Developed Beyond Range of Speaking Or WritingBeyond Range of Speaking Or Writing

Therefore, listening comprehension testing is Therefore, listening comprehension testing is needed.needed.

Purposes of Testing Listening ComprehensionPurposes of Testing Listening Comprehension

a. To test the ability to distinguish phonemesa. To test the ability to distinguish phonemes

b. To test listening comprehension of b. To test listening comprehension of utterances (it could be a single sentence, a utterances (it could be a single sentence, a paragraph, or even a lecture).paragraph, or even a lecture).

Threats to Validity & Reliability of Listening Threats to Validity & Reliability of Listening Comprehension Testing:Comprehension Testing:a. Abilities to discriminate phonemes do not a. Abilities to discriminate phonemes do not imply abilities to understand utterances;imply abilities to understand utterances;b. Testing materials may not simulate real-b. Testing materials may not simulate real-world speeches in terms of lack of world speeches in terms of lack of contextual clues and/or spoken (redundant) contextual clues and/or spoken (redundant) features likefeatures like

hesitations, time filler (like Well, N…, hesitations, time filler (like Well, N…, Kind of), false starts ;Kind of), false starts ;

c. Administrations of testing may not c. Administrations of testing may not include communication strategies used include communication strategies used such as asking again or body language;such as asking again or body language;

d. Certain types of items may test d. Certain types of items may test intelligence or memory in addition to intelligence or memory in addition to listening comprehension proficiency listening comprehension proficiency (Think about “lecture” type of testing).(Think about “lecture” type of testing).

e. If the distractors or any other part of testie. If the distractors or any other part of testing items are written, these items also test ong items are written, these items also test other skills (reading comprehension, vocabulther skills (reading comprehension, vocabulary, lexicon, . . . ) in addition to listening coary, lexicon, . . . ) in addition to listening comprehensionmprehension

Types of Items:Types of Items:1. Phoneme (Sound) Discrimination Tests1. Phoneme (Sound) Discrimination TestsType 1: Type 1: a. Word sets in isolation (p. 33):a. Word sets in isolation (p. 33):(1) (1) Minimal PairsMinimal Pairs

(2) Minimal pairs with an extra word: Two (2) Minimal pairs with an extra word: Two Identical Sentences & One Different Identical Sentences & One Different Sentence (with only one different word) Sentence (with only one different word) (p. 66)(p. 66)

Type 2: Words in context (p. 34)Type 2: Words in context (p. 34)

(1) A heard sentence with two pictures(1) A heard sentence with two pictures

(2) A heard sentence with a written (2) A heard sentence with a written sentence with an empty blank (target sentence with an empty blank (target word) word)

Rules of Thumb for Item Writing:Rules of Thumb for Item Writing:a. Use Recordings (e.g., movies, tapes. . . )a. Use Recordings (e.g., movies, tapes. . . )b. Avoid extremely difficult lexical itemsb. Avoid extremely difficult lexical itemsb. Use simple structures and lexiconb. Use simple structures and lexiconc. Use the “normal” & consistent stress & c. Use the “normal” & consistent stress & pitch patternspitch patternsd. Avoid switching or changing stress & pitch d. Avoid switching or changing stress & pitch patternspatternse. Use “Standard American English e. Use “Standard American English Pronunciation”Pronunciation”f. Simulate spontaneous speeches as possible f. Simulate spontaneous speeches as possible

Criticisms:Criticisms:a. Appropriate for diagnostic purposes a. Appropriate for diagnostic purposes (1) contrast of individual phonemes of L2 (1) contrast of individual phonemes of L2 (2) contrast of individual phonemes between (2) contrast of individual phonemes between L1 & L2L1 & L2(3) identification of difficulties(3) identification of difficulties b. For Type 2-1 (A heard sentence with two b. For Type 2-1 (A heard sentence with two pictures), what else has been tested as well? pictures), what else has been tested as well? c. For Type 2-2 (A heard sentence with a c. For Type 2-2 (A heard sentence with a written sentence with an empty blank (target written sentence with an empty blank (target word), what else has been tested as well?word), what else has been tested as well? (Vocabulary & Syntax)(Vocabulary & Syntax)

2. Sentences & Dialogues:2. Sentences & Dialogues:Type 1: Sentences (p. 37)Type 1: Sentences (p. 37)(a) Questions: A heard question with four di(a) Questions: A heard question with four distractors used as answers to the questionstractors used as answers to the question(Hint: Testees have to understand the entir(Hint: Testees have to understand the entire questions)e questions)(b) Statements: A heard statement with fou(b) Statements: A heard statement with four complete sentences as distractorsr complete sentences as distractors (Hint: Testees have to understand the targe(Hint: Testees have to understand the target word in the statement)t word in the statement)

(c) A heard sentence with four pictures(c) A heard sentence with four pictures(Hint: Testees have to understand the entir(Hint: Testees have to understand the entire sentence and the implications of the four e sentence and the implications of the four pictures without reading any passages)pictures without reading any passages)Type 2: Dialogues: A heard short speech & Type 2: Dialogues: A heard short speech & Distractors (p.40)Distractors (p.40)

a. Short samples of speeches & a. Short samples of speeches & dialogues dialogues

b. Appropriate for administrations in b. Appropriate for administrations in language labslanguage labs

c. Resemble natural discourses c. Resemble natural discourses without redundant featureswithout redundant features

d. Evaluate “hearing” not d. Evaluate “hearing” not “interpreting”“interpreting”

e. Avoid testing memorye. Avoid testing memory

Rules of Thumb: Rules of Thumb: Keep the problems in Keep the problems in the stems rather than in the alternatives. the stems rather than in the alternatives. Criticisms:Criticisms:a. For Type 1, what else has been tested? a. For Type 1, what else has been tested? (Grammatical & Lexicon) So, what will (Grammatical & Lexicon) So, what will happen?happen?b. For Type 2, what else has been tested? b. For Type 2, what else has been tested? (Ability to select an appropriate reply to a (Ability to select an appropriate reply to a stimulus & all abilities tested in type 1)stimulus & all abilities tested in type 1)3. Testing Listening Comprehension 3. Testing Listening Comprehension Through Visual MaterialsThrough Visual Materials::

Features:Features:a. Use pictures or photos to replace written a. Use pictures or photos to replace written passagespassagesb. May be appropriate for young testeesb. May be appropriate for young testeesc. Pictures or photos must be clear enough sc. Pictures or photos must be clear enough so that problems can be easily spotted from to that problems can be easily spotted from themhemd. May also test cognition/intelligenced. May also test cognition/intelligenceTypes of Items:Types of Items:a. Type 1: One Photo First then Questionsa. Type 1: One Photo First then Questions

b. Type 2: A Few Photos First then Questionsb. Type 2: A Few Photos First then Questionsc. Type 3: A Set of Photos First then c. Type 3: A Set of Photos First then Questions Questions d. Type 4: A Few Diagrams First then d. Type 4: A Few Diagrams First then QuestionsQuestionse. Type 5: Following Instructions e. Type 5: Following Instructions f. Type 6: Drawingf. Type 6: DrawingRules of Thumb: Rules of Thumb: a. Always give clear instructions & examplesa. Always give clear instructions & examplesb. Define the acceptable drawingsb. Define the acceptable drawingsc. Avoid testing intelligencec. Avoid testing intelligence

Criticisms:Criticisms:a. Be cautious about the testees’ ability to draa. Be cautious about the testees’ ability to draw w b. Hard to score drawingsb. Hard to score drawingsc. May test intelligencec. May test intelligence4. Lectures & Talks: 4. Lectures & Talks: Lecture then QuestionsLecture then Questions Rules of Thumb:Rules of Thumb:a. Avoid testing memorya. Avoid testing memoryb. Provide clear instructions & Examplesb. Provide clear instructions & Examples

c. Provide real speeches with normal c. Provide real speeches with normal delivery rate, pauses, and breath delivery rate, pauses, and breath segments.segments.

5. TPR5. TPR

a. See the reading booklet for a. See the reading booklet for detailed information:detailed information:

Rules of Writing Distractors:Rules of Writing Distractors:1. Target words may belong to more than one pa1. Target words may belong to more than one part of speech; so it may have different meanings if rt of speech; so it may have different meanings if used as different part of speech (p. 43, E.g. A)used as different part of speech (p. 43, E.g. A)2. For a question used as the stem, think about w2. For a question used as the stem, think about what can be misunderstood by testees and these hat can be misunderstood by testees and these misunderstandings can serve as good distractors:misunderstandings can serve as good distractors: See E.g. B See E.g. B3. For a question with a subordinate clasue used 3. For a question with a subordinate clasue used as the stem, think about what can be misunderstas the stem, think about what can be misunderstood: See E.g. C & Dood: See E.g. C & D