antología para la unidad académica material design ... · harsono, y. (2007). developing ... for...

Antología para la unidad académica Material Design.

Licenciatura en Inglés, modalidad a distancia.

Primer bloque del tercer semestre.

Mtra. Martha Lorena Obermeier Pérez

Agosto, 2013

2

Índice

Pg.

Introducción 3

Programa del curso 4

Unidad 1 Types of Materials

1.1 Authentic and non Authentic material 8

1.2 Materials for specific purposes 10

1.3 Self access material 12

Unidad 2 Materials for specific purposes

2.1 Grammar, Vocabulary and functions 15

2.2 Receptive skills 20

2.3 Productive skills 22

2.4 TIC´S in the classroom 24

Unidad 3 Stage of the design

3.1 Situational diagnosis 26

3.2 The design and production of materials 30

3.3 Material development 37

Unidad 4 Evaluation of materials

4.1 Process of the evaluation of the authentic material 41

4.2 Design, application and analysis of the tools to evaluate non-authentic

materials 47

3

Bibliografía 52

Introducción

This anthology has been prepared to offer you materials for the academic unit of material

design. The course has already some materials with the information necessary to perform the

tasks. The objective of this anthology is to offer you more information from different

perspectives to deepen your knowledge about the topics.

This academic unit is very important, class activities and dynamics are determined by the

material used. Most teachers learn to design materials because they do not have a textbook. Most

of them do not know the criteria to design and evaluate materials. That is the reason why this

academic unit is so important in your training as an English teacher.

4

Programa del curso

Presentación: Esta unidad académica forma parte del eje Formación Disciplinaria del Plan de Estudio del PE de Licenciatura en Inglés. Esta unidad académica ofrece la oportunidad para el estudiante de aprender a diseñar, adaptar, seleccionar y evaluar materiales didácticos y auténticos y aplicarlos en contextos reales.

Propósito General:

Al finalizar el curso, el estudiante será capaz de seleccionar, diseñar y adaptar el material didáctico válido y confiable para las diferentes habilidades y sistemas de una lengua para el desarrollo de su competencia comunicativa, además de evaluarlos. Será también capaz de organizar su aplicación dentro del salón de clase.

Competencias Genéricas:

La habilidad que debe poseer el estudiante es la de aprender a diseñar material didáctico y su aplicación dentro del salón de clase.

Competencias Específicas:

Diseña, adapta, selecciona y evalúa material didáctico según su contexto.

Contenido:

Tipos de materiales

Auténticos vs. no auténticos

Para propósitos específicos

Para el aprendizaje autodirigido

Etapas en el diseño de materiales

Selección

Determinación del contenido y formato

Tipos de ítems

Diseño

Piloteo

Aplicación

Diseño de materiales de acuerdo con los objetivos y contenidos

Gramática, vocabulario y funciones

Habilidades receptivas

5

Habilidades productivas

Evaluación de materiales

Resultados de aprendizaje:

Selecciona, diseña y adapta material didáctico válido y confiable

Aplica sus habilidades dentro del salón de clases

Evalúa material didáctico según su contexto

Estrategias didácticas: se utilizará una metodología en la cual los alumnos se involucran activamente a través del trabajo en equipo realizando actividades como:

Proyectos individuales

Presentaciones en power point

Documentos en Word

Escuchar textos provenientes de los medios de comunicación: radio, televisión, cine

Videograbaciones de conversaciones en pares y grupos

Seguir instrucciones

Participar en conversaciones de tipo formal e informal

Completar formularios y cuestionarios

Producir carteles

Escribir finales de cuentos o sucesos

Escribir cartas personales, notas, mensajes breves, correos electrónicos y un artículo para la gaceta escolar

Explorar temas, cuadros sinópticos y mapas conceptuales

Organizar ideas: idea principal, oraciones de apoyo, de ejemplificación, etc.

Reconocer la estructura textual de un discurso oral o escrito

Identificar palabras clave que indiquen la naturaleza del discurso oral o escrito

Identificar el tema central de un diálogo, exposición oral o texto

Inferir connotaciones, actitudes e intenciones

Parafrasear, usar redes semánticas o sinónimos

Recursos didácticos:

Computadora, accesos a internet, libros, artículos digitales, chat, rubricas, portafolio

digital, DVD CD.

6

Técnicas de Enseñanza-aprendizaje

Discusiones grupales

Trabajo en equipo

Presentaciones

Lectura guiada

Análisis de materiales

Lluvia de ideas

Microenseñanza

Actividades a desarrollar

Selección de materiales adecuados

Diseño de materiales de acuerdo con objetivos a desarrollar

Adaptación de materiales

Prácticas de microenseñanza

Evaluación de materiales diseñados por parte de los estudiantes

Criterios de Evaluación

Nota: Acreditación. El estudiante se considera competente cuando haya cumplido con el 60% de los criterios de evaluación.

Elaboración de material para la enseñanza de gramática, vocabulario y funciones

15%

Elaboración de material para la enseñanza de habilidades receptivas 15%Elaboración de material para la enseñanza de habilidades productivas 15%2 prácticas de microenseñanza 40%Reporte de evaluación de materiales 15% _________

100%

7

Martinez, K. (2008). Authentic & Non-Authentic Material.. [ONLINE] Available at: http://karitol1304.blogspot.mx/2008/02/authentic-non-authentic-material.html. [Last Accessed 06 August 2013].

8

Authentic and non authentic material

Authentic Materials are language materials that were originally intended for native

speakers, not second-language learners. We offer an Authentic Materials Guide that provides in-

depth information about the nature of Authentic Materials and resources for their use. On this site

we also offer Authentic Materials.

For example: Menus, Letters, some publication and many others.

In teaching language for learning purposes, the educators’ main goal is to provide learners

with the linguistic knowledge about the language. This knowledge includes grammatical rules,

vocabulary lists and other linguistic codes. The underlying premise of this approach is that by

informing learners about the language, it will help them decode a text in the target language.

order to decode the text. Under these circumstances, the language educators tend to create

teaching materials that are specifically designed to teach the linguistic concepts of the language.

Usually the non-authentic material does not have the characteristics of real-life language.

It typically consists of the linguistic items that aim to teach or advance learners’ knowledge about

the language. An example of a non-authentic text is passages from textbooks that were created by

the authors to illustrate or practice new linguistic items. The semi-authentic material tries to

include the characteristics of real-life language, but at the same time still focus mainly on

teaching and reinforcing the language goals. The latter includes newspapers in easy Hebrew or

edited stories.

9

Harsono, Y. (2007). DEVELOPING LEARNING MATERIALS FOR SPECIFIC PURPOSES. [ONLINE] Available at: http://journal.teflin.org/index.php/teflin/article/viewFile/191/109. [Last Accessed 06 August 2013].

10

Material for specific purposes

Materials should provide the learners with opportunities to use the target language to

achieve communicative purposes. After learning the materials, learners should be given

opportunities to practice the language they have learned for communication in real life situation

not just practicing it in the classroom controlled by the teacher. Materials should take into

account that the positive effects of instruction are usually delayed. As it can be observed in our

daily teaching, learners learning a language is not an instantaneous process but a gradual one. In

order to facilitate the gradual process of learning or acquiring a language, it is important to give

frequent and ample exposure to the instructed language features in communicative use. Materials

should take into account that learners differ in learning styles. Not all learners have the same

learning styles. Language learning styles include visual, auditory, kinaesthetic (e.g. the learner

prefers to do something physical, such as following instructions), studial (e.g. the learner likes to

pay conscious attention to the linguistic features of the language and wants to be correct),

experiential, analytic, global, dependent and independent.

11

Tomlinson, B. (2010). Principles and Procedures for Self-Access Materials. [ONLINE] Available at: http://sisaljournal.org/archives/sep10/tomlinson/. [Last Accessed 09 08 2013].

12

Self-access material

A pre-requisite for language acquisition is that the learners are exposed to a rich,

meaningful, and comprehensible input of language in use (Krashen, 1999; Long, 1985).

In order to acquire the ability to use the language effectively the learners need a lot of experience

of the language being used in a variety of different ways for a variety of purposes. They need to

be able to understand enough of this input to gain positive access to it and it needs to be

meaningful to them.

1. Provide extensive reading, extensive listening, and extensive viewing materials which

provide experience of language being used in a variety of text types and genres in relation to

topics, themes, events, locations, and so on, likely to be meaningful to the target learners. One

way of doing this is to timetable teachers to provide live ‘performances’ of stories, jokes, extracts

from novels and plays, anecdotes, newspaper articles etc in a closed off area of a self-access

centre. Copies of the texts could be made available for interested learners to take away and file in

their Anthology of Interesting English.

2. Encourage the learners to experience the extensive materials holistically and enjoyably, but

also provide opportunities to revisit the materials to discover more about how the language is

used. For ideas for creative follow up activities for extensive reading see Fenton-Smith

(forthcoming 2011) and for ideas for noticing activities after video clip viewing see Stillwell,

McMillan, Gillies, and Waller (forthcoming 2011).

3. Make sure that the language the learners are exposed to in all their self-access materials is

authentic in the sense that it represents how the language is typically used. If many of their texts

13

are inauthentic because they been written or reduced to exemplify a particular language feature

then the learners are unlikely to acquire the ability to use the language typically or effectively.

14

Nordquist, R (2013). What Is Grammar?. [ONLINE] Available at: http://grammar.about.com/od/basicsentencegrammar/a/grammarintro.htm. [Last Accessed 2013].

15

Grammar, vocabulary and functions

Grammar is the structural foundation of our ability to express ourselves. The more we are

aware of how it works, the more we can monitor the meaning and effectiveness of the way we

and others use language. It can help foster precision, detect ambiguity, and exploit the richness of

expression available in English. And it can help everyone--not only teachers of English, but

teachers of anything, for all teaching is ultimately a matter of getting to grips with meaning.

(David Crystal, "In Word and Deed," TES Teacher, April 30, 2004)

16

Vocabulary is vital to communicating with others and understanding what one is reading.

I know that information is known to everyone, but acknowledging it instead of taking it for

granted adds important goals to your child's list of basic skills to master.

If your child is struggling with reading comprehension, then add more concentration of

studying words in context. Encourage your children to look up words that they do not understand.

Each of my children had their own small dictionary and thesaurus.

Vocabulary is the study of:

The meanings of words

Many words have several different meanings each, study the meanings of the words and the

part of speech.

How the words are used

Study the words in context, apply what you learn by writing sentences with your words.

Root words, prefixes, suffixes

Studying these will aid in the study of vocabulary.

Analogies

This is comparing two pairs of words and choosing the pair that goes together.

17

Bilash, O. (2009). Functions of Language. [ONLINE] Available at: http://www.educ.ualberta.ca/staff/olenka.bilash/best%20of%20bilash/functionsof%20lang.html. [Last Accessed 07 August 2013].

18

A lot of what we say is for a specific purpose. Whether we are apologizing, expressing a

wish or asking permission, we use language in order to fulfill that purpose. Each purpose can be

known as a language function. Savignon describes a language function as “the use to which

language is put, the purpose of an utterance rather than the particular grammatical form an

utterance takes” (Savignon, 1983). By using this idea to structure teaching, the instructional focus

becomes less about form and more about the meaning of an utterance. In this way, students use

the language in order to fulfill a specific purpose, therefore making their speech more

meaningful.

19

British Council (2013). TeachingEnglish. [ONLINE] Available at: http://www.teachingenglish.org.uk/knowledge-database/receptive-skills. [Last Accessed 07 August 2013].

20

Receptive skills

The receptive skills are listening and reading, because learners do not need to produce

language to do these, they receive and understand it. These skills are sometimes known as

passive skills. They can be contrasted with the productive or active skills of speaking and writing.

Often in the process of learning new language, learners begin with receptive

understanding of the new items, then later move on to productive use.

In the classroom; the relationship between receptive and productive skills is a complex

one, with one set of skills naturally supporting another. For example, building reading skills can

contribute to the development of writing.

21

Muñoz, L. (2013). Productive skills. [ONLINE] Available at:

http://englishonline.jimdo.com/productive-skills-teaching/. [Last Accessed 07 August

2013].

22

Productive skills

Productive skills are those skills in which we have to produce language, those skills are

Writing and Speaking.

Reasons for teaching writing:

Reinforcement

Lg development

Learning style

Writing as a skill

Approaches for writing:

The product approach.

In this approach the teacher or guide will be focus on the final product (piece of writing)

not in the effort or processes the students go through.

The process approach

In this approach the most important point to grade is the process of writing and not the

final product.

23

Chris Cope, Peter Ward (2002). Integrating learning technology into classrooms: The

importance of teachers’ perceptions.. [ONLINE] Available at:

http://www.ifets.info/journals/5_1/cope.html. [Last Accessed 07 August 2013].

24

TIC´s in the classroom

The integration of learning technologies into high school classrooms is being promoted

and supported around the world. Underlying the promotion and support are claims that successful

integration will lead to enhanced learning outcomes (DoE, 1998). These claims are difficult to

justify, however. Research into the impact of learning technologies on the quality of students’

learning outcomes is limited and outdated according to Honey, Culp and Carrigg (2000). A

limiting factor has been the difficulty of defining and measuring enhanced learning outcomes

attributable specifically to the use of learning technologies (Mitchell & Bluer, 1997).

Recent research has avoided this difficulty and focussed on investigating the requirements

for successful integration of learning technologies into classrooms. The research is beginning to

show that success requires understanding the complex interactions in classrooms between

teachers, students and technology (Honey, Culp & Carrigg, 2000). This understanding is

currently incomplete. Parr (1999) studied students’ perceptions of learning contexts that

incorporated learning technologies. Students’ perceptions were found to influence the success of

integration, specifically the amount of technology use, the ways in which the technology was

used, and teachers’ and students’ expectations about learning. Teachers’ perceptions of learning

technologies and influence on students’ perceptions have not been studied recently. To further

understanding, our study investigated experienced teachers’ perceptions of learning technologies.

The research also considered the likely impact of teachers’ perceptions on students’ perceptions

and, consequently, on the integration of learning technologies into classrooms.

25

Jordan, R.(1997).English for academic purposes. Cambridge University Press: UK.

26

Situational Diagnosis

Beyond listing specific content features, is to draw up a list of factors that you would wish

to compare and evaluate. Sheldon (1988) adds a caveat: No one is really certain what criteria and

constraints are actually operative in ELT contexts worldwide, and textbook criteria are

emphatically local.

In an evaluation taken from Hamp-Lyons and Heasley (1984), there’s a list of seven

standards of “textuality”, in which one is called “situationality”, the definition of such scope is:

“The way in which a text is relevant to the situation in which it occurs”. So, it is vital to diagnose

and analyze our situation in order to know what type of materials we need to select or produce.

27

Dudley-Evans, T. Jo St John, M. (1998). Developments in English for specific purposes. Cambridge University Press: UK.

28

To make good choices we need to have good criteria on which to base our decision.

Numerous criteria, such as factors about the learners, the role of the materials, the topics, the

language, the presentation, have been put forward for the analysis of materials and each of them

has validity

29

Hutchinson, T. Waters, A. (1987). English for Specific Purposes. Cambridge University Press: UK.

30

The Design and Production of Materials

Materials’ writing is one of the most characteristic features of ESP in practice. In

marked contrast to General English teaching, a large amount of the ESP teacher's time

may well be taken up in writing materials. There are a number of reasons for this:

a) A teacher or institution may wish to provide teaching materials that will fit the

specific subject area of particular learners. Such materials may not be available

commercially. In addition to the profusion of subject specialisms, there is also a wide

range of course types. Whereas schools, for example, work to standard timetables with

a similar number of hours, ESP courses can vary from one week of intensive study to an

hour a week for three years or more. Publishers are naturally reluctant to produce

materials for very limited markets. The cost of producing and marketing a book is

much the same regardless of whether it sells one thousand copies or one hundred

thousand copies. It is likely, then, that a course tailored to the needs of a specific group

of ESP learners will not be available.

b) Even when suitable materials are available, it may not be possible to buy them

because of currency or import restrictions.

c) ESP materials may also be written for non-educational reasons: for example, in order

to enhance the reputation of an institution or an individual. Materials are a visible

product of activity, regardless of whether such activity is useful or even necessary.

For these and other reasons, there is already an established tradition of ESP teachers producing

in-house materials. These may then be distributed to other institutions or even published, but in

general they are written by the teachers of a particular institution for the students at that

institution. Such a pattern of work is often something of an abuse of teachers. Few have

31

had any training in the skills and techniques of materials writing. It also shows a rather

cavalier attitude to the activity of materials writing, implying, as it does, that if you can teach

you can write materials. How many actors are expected to write their own plays or singers their

own songs ? On the other hand, it can be argued that the process of materials writing may

help to make teachers more aware of what is involved in teaching and learning.

Rightly or wrongly, materials writing is a fact of life for a large number of ESP teachers, and so,

accepting this fact, let us look at some techniques for producing useful and creative ESP materials.

1. Defining objectives

We can start by asking ourselves the question: What are materials supposed to do?

In defining their purpose, we can identify some principles which will guide us in the actual

writing of the materials.

a) Materials provide a stimulus to learning. Good materials do not teach : they encourage

learners to learn. Good materials will, therefore, contain:

— interesting texts;

— enjoyable activities which engage the learners' thinking capacities;

— opportunities for learners to use their existing knowledge and skills;

— content which both learner and teacher can cope with.

b) Materials help to organise the teaching-learning process, by providing a path through the

complex mass of the language to be learnt. Good materials should, therefore, provide a clear

and coherent unit structure which will guide teacher and learner through various

activities in such a way as to maximise the chances of learning. This structure should help the

teacher in planning lessons and encourage in the learner a sense of progress and achievement.

On the other hand, materials should not be so tightly structured as to produce a

32

monotonous pattern of lessons the curse of so many materials. Avoid the assembly line

approach, which makes each unit look the same, with the same type of text, the same kind of

illustrations, the same type and number of exercises. If it doesn't send you to sleep writing them,

it will certainly send your learners to sleep using them. A materials model must be clear and

systematic, but flexible enough to allow for creativity and variety.

c) Materials embody a view of the nature of language and learning. In writing materials you,

as an author, are making all manner of statements about what you think language

learning consists of. Materials should, therefore, truly reflect what you think and feel

about the learning process. If you believe that people learn when their thinking capacities are

engaged, don't write exercises which require little or no active thought. If you think learning is

enhanced by intense

Application

Experiences with language, don't provide texts which have been stripped of any human

interest. If you think learning is helped by frequent reinforcement, make sure that items to be

learnt are processed several times.

d) Materials reflect the nature of the learning task. We have noted in previous chapters that

language learning is a complex process involving many different kinds and levels of knowledge.

In the heyday of structuralism, it was assumed that a knowledge of the structures was the same as

knowing a language and that repetition led to learning. Materials writing was a simple task of

isolating the structure, writing a text to exemplify it and pattern drills to practise it. We must now

take a more humble view and recognise that language learning is a very complex and little

33

understood process. Materials should try to create a balanced outlook which both reflects the

complexity of the task, yet makes it appear manageable.

e) Materials can have a very useful function in broadening the basis of teacher training, by

introducing teachers to new techniques.

f) Materials provide models of correct and appropriate language use. We have deliberately

placed this last on our list. This is a necessary function of materials, but it is all too often taken as

the only purpose, with the result that materials become simply a statement of language use rather

than a vehicle for language learning. Language teaching materials should not be the kind of

beginner's guide to Applied Linguistics, which is so prevalent in ESP. We, as linguists, may be

endlessly fascinated by the analysis of discourse : it is our chosen specialist field. For the doctor,

the secretary and the engineer language may have little such attraction.

2. A materials design model

Principles we have outlined, we can now present a model which we have used for writing

our own materials. The aim of this particular model is to provide a coherent framework for the

integration of the various aspects of learning, while at the same time allowing enough room for

creativity and variety to flourish. The model consists of four elements: input, content focus,

language focus, task.

a) Input: This may be a text, dialogue, video-recording, diagram or any piece of communication

data, depending on the needs you have defined in your analysis. The input provides a number of

things:

— stimulus material for activities;

34

— new language items;

— correct models of language use;

— a topic for communication;

— opportunities for learners to use their information processing skills;

— opportunities for learners to use their existing knowledge both of the language and the subject

matter.

b) Content focus: Language is not an end in itself, but a means of conveying information

and feelings about something. Non-linguistic content should be exploited to generate

meaningful communication in the classroom.

c) Language focus: Our aim is to enable learners to use language, but it is unfair to give

learners communicative tasks and activities for which they do not have enough of the necessary

language knowledge. Good materials should involve both opportunities for analysis and

synthesis. In language focus learners have the chance to take the language to pieces, study how it

works and practise putting it back together again.

d) Task: The ultimate purpose of language learning is language use. Materials should be

designed, therefore, to lead towards a communicative task in which learners use the

content and language knowledge they have built up through the unit.

35

These four elements combine in the model as follows:

CONTENT LANGUAGE

Figure 28: A materials design model

TASK

36

Tomlinson, B. (1998). Materials Development in Language Teaching. Cambridge University Press: UK.

37

Material Development

In terms of recognising a need, exploring the language required to meet the need and

finding a reasonable context for practice, this exercise may be said to pass muster; what has

clearly failed is the pedagogical realisation of the materials, i.e. if these materials were

intended to provide meaningful practice whereby students would make statements of greater

or less certainty, they clearly fail. Part of the materials-writers task must be to provide clear

exercises and activities that somehow meet the need for language-learning work initially

recognised. Some would say that this is the core of materials writing. Part of effective

pedagogical realisation of materials is efficient and effective writing of instructions, including the

proper use of metalanguage; poor instructions for use may waste a lot of valuable student time.

The physical appearance and production of materials is important both for motivation

and for classroom effectiveness. Teachers engaged in writing materials need to develop the

same care and attention to presentation that one would expect of good publishers, though the

first quote reveals that even very good publishers also fall down on the job.

However, we take an entirely different view, believing that materials writing as a

process is pointless without constant reference to the classroom. In short, a need arises,

materials are written, materials are used in the classroom to attempt to meet the need and

subsequently they are evaluated. The evaluation will show whether the materials have to be

rewritten, thrown away, or may be used again as they stand with a similar group. Writing the

materials is only a part of the activity of teaching.

38

Most materials writers move in this direction, and use some or all of these steps, if

not always precisely in this order: a movement from the identification of a need for

materials to their eventual use in the classroom. Some such simplified version of

the materials-writing process is also clearly how most publishers are constrained to

work• The one-directional simplicity of this model, however, may be what makes so

many materials, whether published or found in one's own a colleague's filing-cabinet,

lack that final touch of excellence that many teachers appear to wish for. In fact, the

excellence of materials lies less, in the products themselves than in the appropriate and

unique tuning for use that teachers might be better engaged in. The simple sequence fails

to illustrate the extent to which materials writing can be a dynamic and self-adjusting

process.

In the first place, by ending with use in the classroom, it equates materials production and

use of materials with effective meeting of need identified. What is lacking is a stage beyond use in

the classroom: evaluation of materials used. The act of evaluation at least in theory turns the

process into a dynamic one since it forces the teacher/writer to examine whether s/he has or

has not met objectives: furthermore, a failure to meet objectives may be related to any or all

of the intervening steps between initial identification of need and eventual use. (Failure may,

of course, be attributed to poor or inadequate use of perfectly adequate materials but that

39

becomes a matter of classroom management rather than materials evaluation except where

poor use is directly related to faulty production.)

Secondly, the human mind does not work in the linear fashion suggested

above when attempting to find solutions to problems. For example, a proposal about

what form a particular language exercise could take may very well generate

spontaneous second thoughts about the language being exercised; wondering about the

physical production of a piece of material may well spark off thoughts about

contextualisation and so on. Thus, in addition to evaluation as an essential

component of writing materials, we must also imagine a variety of optional

pathways and feedback loops which make the whole process both dynamic and self-

regulating. These then will allow us to deal in a concrete way with the reasons for the

failure of language materials and provide us with clues to their improvement, both during

the writing and after their use.

40


41

Process of the evaluation of authentic materials

Evaluation, which requires an in-depth look at two or more units in order to

investigate such aspects as the presentation of skills in the materials, the grading and

sequencing of the materials, the kinds of texts used and the relationship between exercises and

tests. Skierso (op. cit.) envisages a three-step procedure:

1. Identification of relevant contextual information relating to the students, the teacher, the

course syllabus and the institution,

2. analysis of the features of the textbook followed by an overall rating of the text, and

3 the actual judging of the acceptability of the textbook, involving both the rating and weighting

of specific evaluative criteria.

The goal of these textbook survey guides is to provide a systematic and principled

approach to materials evaluation. Although the guides may be of practical use to teachers,

they raise many questions relating to such issues as what aspects of the materials should be

considered and, crucially, how one aspect should be weighted in relation to another. For

example, many of the guides specifically ask whether the materials contain authentic texts,

but as Widdowson (1979) has pointed out, the whole question of authenticity in language

teaching is a complex one with authenticity being determined by what the learners do with

the text. Also, a positive rating on a criterion of authenticity (i.e. the materials contain

authentic texts) may be matched with a negative rating on the criterion of vocabulary load

(i.e. the number of new words introduced is excessive). How is the materials evaluator to

reconcile these conflicting ratings? Faced with such questions, which remain unresolved in the

I!

42

existing literature, it is hard not to agree with Sheldon (1988: 245) that coursebook evaluation is

fundamentally a subjective, rule-of-thumb activity, and that no neat formula, grid, or system

will ever provide a definite yardstick'. It is pertinent to ask whether greater objectivity is

achievable in a micro-evaluation focused on a particular language learning task.

The same guides that have been designed for before-programme materials

evaluation can be used for after-programme evaluation. There are, however, very few published

accounts of such evaluations. The ELT Documents issue devoted to 'ELT textbooks and

materials: problems in evaluation and development' (Sheldon 1987) disappointingly does

not contain a single after-programme evaluation.

Why have there been so few? One possible reason is that teachers see no need for a

systematic and principled post-programme evaluation. They feel they know whether the book

'works' or not as a result of their day-by-day experiences of using it in their teaching. Even

if they acknowledge a need for detailed evaluation, teachers may be dauntedby the size of the

task; carrying out an empirical evaluation of a textbook is an enormous undertaking.

The desirability of engaging teachers in a careful evaluation of language teaching

materials after they have been used is widely acknowledged, however (see, for

example, Rea-Dickens and Germaine 1992; 30). Such an evaluation provides an appraisal

of the value of specific teaching activities for particular groups of learners and, perhaps more

importantly, serves as encouragement to teachers to adopt a 'reflective approach' to their own

teaching (Richards and Lockhart 1994). One way of encouraging the systematic evaluation of

materials after use may be to engage in micro- evaluation by focusing on particular tasks.

Such an approach to the empirical validation of teaching materials may prove more manageable

and therefore less daunting to teachers.

43

What kind of evaluation?

Before we turn our attention to the evaluation of language learning tasks, it is

pertinent to ask what kind of evaluation will be required. I shall attempt to answer this by

examining a number of dimensions of evaluation: (I) approach, (2) purpose, (3) focus, (4)

scope, (5) the evaluators, (6) the timing and (7) types of information. These dimensions are

discussed in the existing literature in relation to macro- evaluation (i.e.

programme/project evaluation) but they are equally applicable to micro-evaluation (e.g. the

evaluation of a specific task).

The literature on educational evaluation distinguishes two broad approaches; the

'objectives model' and 'responsive evaluation' (see Norris 1990). The former belongs to

the psychological tradition of educational research and is nomothetic in approach (i.e. it

employs quantitative data to make statistical generalisations with a view to establishing

general laws'). It requires that curricula he expressed in terms of precise objectives, the

achievement of which can be determined by tests that measure learner behaviour and learning

outcomes. Responsive evaluation belongs more to the sociological tradition of educational

research and is idiographic in approach (i.e. it proceeds by means of intensive studies of

individuals or particular cases). Whereas the 'objectives model' approach is concerned with

determining whether the programme/project has achieved its goals, the 'responsive evaluation'

approach aims to illuminate the complex nature of the organisational, teaching and learning

processes at issue. The recent history of educational evaluation can be seen as a debate

between the rival claims of these two approaches. Increasingly, however, evaluators are

recognising the need for a broad-based approach to evaluation that incorporates both the

44

objectives model and responsive evaluation (see Weir and Roberts 1994, for example).

These two approaches to evaluation reflect two general purposes for carrying out an

evaluation: (I) accountability and ( z) development. Where accountability is at stake, the

purpose of the evaluation is to determine whether the stated goals of the programme have been

met. In the case of development, the purpose may be either (a) to improve the curriculum or (b)

to foster teacher-development or both. In general, evaluation for accountability will require

an 'objectives model' approach, while evaluation for development will require a 'responsive'

approach. However, it is not quite so simple as this, as the methods employed by either

approach can yield information that is of value to both accountability and to development. For

example, summative test scores can be used to determine whether the goals of the

programme have been met but they can also be used to help teachers understand what parts

of the curriculum need further attention.

In addition to overall purpose it is useful to consider the focus of the evaluation. Two

principal foci can be identified: effectiveness and efficiency. It is much easier to focus on

effectiveness as the question that needs to be answered is simply 'Is the programme effective in

meeting the needs of the learners?' It is much more difficult to address the efficiency of a

programme, where the key question is 'Does the programme meet the needs of the learners

more effectively than some alternative programme?' To address effectiveness the evaluator needs

to compare what the learners knew and were able to do before the programme started with what

they know and are able to do at the conclusion of the programme. If the learners show gains in

those aspects of language proficiency which were the stated goal of the programme, then the

programme can be considered 'effective'. To investigate efficiency, however, it is necessary to

compare the learning gains evidenced by one programme (the programme being

evaluated) with the gains evidenced by another programme that differed in some way from

45

the target programme. It is not always possible to carry out a comparison of this kind.

Evaluators also need to determine the scope of their evaluation. The important decision

to be made in this respect concerns whether the evaluation will examine the programme in

terms of its stated goals (i.e. ask 'Did the programme meet its goals?') or whether it will

also examine the goals themselves (i.e. ask 'Are the goals appropriate for the learners in

question?'). In other words, the scope of the evaluation can be 'internal' in the sense that it

uncritically accepts the goals of the programme or it can be 'external' in the sense that it

submits the goals of the programme to critical scrutiny. This distinction between 'internal' and

'external' evaluation is important in another way.

46


47

Design, application and analysis of the tools to evaluate non-authentic materials

Please answer these questions. There is no need to write your name.

1 How easily could you do this task?

VERY QUITE ONLY WITH

EASILY EASILY DIFFICULTY

2 How enjoyable did you find this task?

VERY QUITE NOT

ENJOYABLE ENJOYABLE ENJOYABLE

How much did this task help you to learn English?

VERY SOME NOT VERY

MUCH MUCH

Can you write one thing you liked about this task?

Can you write one thing you did not like about this task?

Figure 5 Learner questionnaire

Analsying the information

the analysis of the information will need (I) to determine whether the objectives of the

task have been met (an accountability evaluation) and 2) to indicate ways in which the task might

be improved (a development evaluation).

To deal with (I), a number of quantitative analyses will be carried out. First, the learners'

level of comprehension of the directions will he determined by deriving a comprehension score

for each learner from the matrix picture they completed. This will provide information to indicate

48

to what extent the learners were successful in comprehending the directions (objective ( t)).

Second, the number of times each learner requested clarification will be tabulated. This

information will be used to determine to what extent the objective (2) was met. This information,

together with that provided by the uptake chart and achievement test, will also enable the

evaluator to examine whether learners who did not actually request clarification themselves were

able to gain from the clarifications sought by other students. Finally, the differences between

each learner's score on the pre-test and the post-test will he calculated to provide an indication of

whether the learners have learnt the names of kitchen objects which they did not know prior to

the task. In addition, the learners' uptake charts will be inspected to identify which words the

learners thought they had learnt from the task. This provides a means of investigating

unpredictable learning. These analyses will be used to determine whether objective (3) has been

met. Of course, these analyses do not show that investigating unpredictable learning. These

analyses will be used to determine whether objective (3) has been met. Of course, these analyses

do not show that the learners retained the words in the long term. For that, further achievement

tests, administered weeks after lesson, would be needed.

To investigate ways in which the task might be improved, both quantitative and

qualitative analyses will be carry out. First, the number of learners who comprehended each

direction will be calculated. This will be provide an indication of which directions were very easy

and which ones were very difficult. Second, quantitative information relating to the learners ´own

evaluation of the task will be provided, based on their answers to the questionnaire. This can be

used to provide a general indication of the overall difficulty of the task, importantly, the level of

motivation generated by the task. Third, the two interviews with the teachers (the one before the

task and the one after) will be analysed qualitatively to determine to what extent the teacher felt

49

this particular task had "worked". Finally, the learner questionnaires and the teacher post-task

interview will be analysed qualitatively to identify their views about any problems with the task.

Clearly, a task evaluation such as this is costly in time and effort. It is intended not as an

example of the kind of evaluation that should always be undertaken but as an example of the kind

of comprehensive evaluation that is possible and that might be undertaken occasionally, Teachers

wishing to evaluate the task they use on a regular basis will need to undertake narrower

evaluations, focusing on either accountability or development, and thereby, limiting the kinds of

information that need to be collected and analysed.

Conclusion

This chapter has examined the case for carrying out evaluations of communicative tasks.

It has been argued that evaluation in ELT has been primarily concerned with macro-evaluation

(i.e. the evaluation of complete programmes or projects through the evaluations of materials,

teachers and learners) and that attention also needs to be given to micro-evaluations. A task

evaluation constitutes an example of micro-evaluation. Such evaluations, it has been suggested,

accord more with teachers ´own ideas of what evaluation entails and can contribute to teacher

development by promoting reflection on teaching.

As yet there are few published evaluations of language learning tasks, one of the few

being Murphy 1993, which reports an evaluation of task used by 20 Malaysian teacher of

English. There is, however, a substantial body of published research that has investigated the

relationship between the design features of task and the language use that results from them (e.g.

Long 1981; Gass and Varonis 1985; Duff 1986; Berwick 1990). It is pertinent to conclude this

50

paper, then, by considering in what ways this published research differs from the kind of task

evaluation I have proposed.

It is not possible to draw a clear distinction between research and evaluation. As Rea-

Dickens states:

Distinct boundaries do not operate between educational evaluation on the one hand

educational evaluation on the one hand and educational research on the other.

(1994:71)

It is not surprising, therefore, to find that task-directed research and task evaluation have

much in common. For example, they share a common set of procedures for collecting data. All

the procedures listed in Figure 4 can be used in task-directed research. There are, however,

differences in the goals, design and outcomes of research and evaluation.

Task-directed research has been typically theory-driven. In particular, researches have

been concerned to discover what effect varying task features has on the negotiation of meaning

that Long (1983) has Hypothesised to facilitate second language acquisition. In contrast, task

evaluation, like all educational evaluation, is motivated by the desire to contribute to the

effectiveness of pedagogy. It is, As Norris (1990: 98) puts it, designed " to elucidate a problem

in action". Of course, task-directed research may also be motivated in part by a desire to increase

the effectiveness of pedagogy, but, its primary purpose has been theory development. It does not

directly address pedagogic problems. Evaluation, as Rea-Dickens (1994) points out, is concerned

with " immediate practical use rather than ultimate use".

This difference in goal is reflected in design differences. Task-directed research has

sought to control extraneous variables in order to examine the effects of particular design features

51

on output. In contrast, task arises when tasks are used in real classrooms. It is concerned with the

outcomes of task use in natural, not experimental or pseudo-experimental, environments.

Finally, there are differences in the outcomes of task-directed research and task

evaluation. The former provides information that is used to frame general laws to account for

how task variables affect language use. Subsequently, of course, these general laws can be

applied to pedagogy but problems of application may arise because the information on which

they are based was not derived from an actual teaching context. Task evaluation, in contrast, is

directly concerned with the effectiveness and efficiency of particular task in particular contexts.

As Glass and Worthen (1971; cited in Norris 1990) point out, the outcomes of research are

judged in terms of reliability and validity while the outcomes of an evaluation are judged in terms

of utility and credibility.

The distinction between task-directed research and task evaluation need to be as sharp as I

have described, however. Much depends on the kind of research. As Norris (1990) points out

there are obvious similarities between much contemporary educational research and evaluation.

This similarity is, perhaps, more evident In research that is qualitative and exploratory in style

than in research that is quantities and experimental. Task evaluation, then, can be seen as a way

of developing our understanding of the ways in which tasks work and, in so doing, of

contributing to both acquisition theory and pedagogic practice. It can also be seen as one way or

carrying out action research.

52

Bibliografía

British Council (2013). TeachingEnglish. [ONLINE] Available at: http://www.teachingenglish.org.uk/knowledge-database/receptive-skills. [Last Accessed 07 August 2013].

Chris Cope, Peter Ward (2002). Integrating learning technology into classrooms: The importance of teachers’ perceptions.. [ONLINE] Available at: http://www.ifets.info/journals/5_1/cope.html. [Last Accessed 07 August 2013].

Harsono, Y. (2007). DEVELOPING LEARNING MATERIALS FOR SPECIFIC PURPOSES. [ONLINE] Available at: http://journal.teflin.org/index.php/teflin/article/viewFile/191/109. [Last Accessed 06 August 2013].

Hutchinson, T. Waters, A. (1987). English for Specific Purposes. Cambridge University Press: UK.

Martinez, K. (2008). Authentic & Non-Authentic Material.. [ONLINE] Available at: http://karitol1304.blogspot.mx/2008/02/authentic-non-authentic-material.html. [Last Accessed 06 August 2013].

Muñoz, L. (2013). Productive skills. [ONLINE] Available at: http://englishonline.jimdo.com/productive-skills-teaching/. [Last Accessed 07 August 2013].

Nordquist, R (2013). What Is Grammar?. [ONLINE] Available at: http://grammar.about.com/od/basicsentencegrammar/a/grammarintro.htm. [Last Accessed 2013].

Situational Diagnosis (p. 130 & 133 English for academic purposes. R. R. Jordan)



Tomlinson, B. (2010). Principles and Procedures for Self-Access Materials. [ONLINE] Available at: http://sisaljournal.org/archives/sep10/tomlinson/. [Last Accessed 09 08 2013].