is one additional english language class enough? · the toeic should be used for and what sorts of...

1人文論集　第 50 巻

Is One Additional English Language Class Enough?

Brian D. Bresnihan

AbstractThis paper considers whether requiring Japanese college or university students

to attend one additional English language class, in addition to the two, or perhaps three,

which are normally required for first-year students, is enough to ensure that a good

majority of them will demonstrably improve their English abilities, as measured by

the TOEIC. The paper begins with background information concerning the creation

of the TOEIC and the increase in its usage, followed by explanations of some of the

problems of using the TOEIC in schools. Then, TOEIC scores achieved by students

at one Japanese university are presented and analyzed to try to answer the research

question in this specific case, finding that one extra English language class is not

suffi cient to ensure measurable progress. The paper ends with a discussion of why this

one additional class is not enough and suggestions that could be implemented to better

ensure that a majority of the students in a program improved their English abilities by a

measurable amount.

1．BackgroundAt the request made by a number of institutions in Japan to the Educational

Testing Service (ETS) in 1977, 1 the TOEIC was created “to measure the English

language skills used in international corporations around the globe,”2 “the everyday

English skills of people working in an international environment . . . (and) how well

people can communicate with others in the global workplace,”3 It “is designed for use

by organizations working in an international market where English is the primary

language of communication.”4 “The fi rst form of TOEIC was administered in Japan on

December 2, 1979.”5 　　　　　　　　　　　　　　　　　　　　　　　　　　　　1 For details about the creation of the TOEIC, see Chapman, 2004; McCrostie, 2009, 2010; IIBC, 2009a; ETS,

2013b.2 This quote is from Wilson, 1993, p. 1.3 This quote is from CGI, 1998, p. II-1.4 This quote is from CGI, 2000, p. 2.5 This quote is from Woodford, 1982, p. 5.

After the fi rst few years, it appeared that the TOEIC might have to be abandoned

as a fi nancial loss,6 but in 1982, two changes were made that eventually reversed this

situation.7 One was the introduction of a second type of administration procedures, in

which organizations could decide on the date, time, and location for the testing to take

place and would administer it themselves, though ETS would not guarantee the rigor of

these testing administrations. This is the TOEIC Institutional Program (IP) Test, with

the original being the TOEIC Secure Program (SP) Test. Both types of administrations

have continued to the present. The other change was to begin administering the

TOEIC in a second country, South Korea. Within a few years, the numbers of TOEIC

test takers each year was increasing greatly, in both Japan and South Korea.

Usage of the TOEIC has now spread to many other countries around the world,

though the great majority of examinees are still in Japan or South Korea. In 2013, over

5,280,000 TOEIC tests were sat for worldwide,8 and over 2,360,000 TOEIC tests were

administered in Japan.9 Of those in Japan, about 44 percent were taken by primary

to graduate school aged students.10 Worldwide in 2013, 52 percent of the tests were

administered to full-time students, and 49 percent of all examinees had taken the test on

at least three other occasions.11

2．Problems concerning usage of TOEIC in schoolsThe very fi rst paragraph gives the stated audience of the TOEIC, people working

using English in international settings. Clearly, Japanese school-aged and university

students are not a part of this group. Most Japanese students do not have the kinds

of experiences or knowledge that the test makers expect TOEIC examinees to have.

This raises doubts as to the validity of using TOEIC scores to measure their English

abilities. It is expected that there will be elements and details of the test items, which

are unrelated to English language abilities, that the great majority of Japanese students

do not possess. Therefore, the test results will at least partially be a measure of their

knowledge of things that are not part of their English profi ciency.

If school administrators and teachers still feel that TOEIC should be used, despite 　　　　　　　　　　　　　　　　　　　　　　　　　　　　6 See McCrostie, 2009, 2010, for more about this.7 For general information about these and other changes and developments in the TOEIC, its administrations,

and test takers, see IIBC 2009a.8 For information on numbers of TOEICs administered worldwide, see ETS, 2014a, p. 30.9 For information on numbers of TOEICs administered in Japan, see IIBC, 2014, p. 3.10 For this fi gure, see IIBC, 2014, p. 3.11 For these fi gures, see ETS, 2014a, p. 3.

2


人文論集　第 50 巻

the obvious validity problem just mentioned, there are yet other factors that need

to be taken into account in order to use it correctly. According to the organization

responsible for the TOEIC in Japan, “At schools, the test is utilized to evaluate student

proficiency levels and measure the effectiveness of instruction, and also serves as an

entrance examination or a standard for accreditation or units.”12 It is also used in

various ways to assist in producing grades in courses.

This list of uses of the TOEIC suggest that it is both a norm-referenced test,

“based on knowledge, skills, or abilities that are common to a number of institutions,

programs, or courses” and “used for (evaluating) aptitude (and) proficiency and (for

making) placement decisions,” and a criterion-referenced test, “based on the very

specifi c objectives of a course or program” and used “for making diagnostic, progress,

and achievement decisions.”13 However, the TOEIC is really only a norm-referenced

test, which may be able to evaluate the English abilities of adult workers well. Earlier

questions have been raised as to the extent to which the TOEIC can accurately measure

Japanese students’ English profi ciencies. Being created for millions of people to take

all over the world, the TOEIC clearly cannot necessarily contain test items that are

directly connected with what has been taught in any specifi c course. Therefore, it is

diffi cult to believe that it can measure how well or how much students have learned in a

course, that it can be a criterion-referenced test, and seems an odd source of assistance

in producing students’ grades. It is also worth pondering what sorts of accreditation

the TOEIC should be used for and what sorts of courses a certain TOEIC score should

be considered as equivalent to. For example, should a certain TOEIC score be required

for certification of or degrees awarded to English teachers? Should achievement of

a certain TOEIC scores allow students to be given the credits for an English course

emphasizing communicative skills, seeing as its test items only measure listening and

reading? Such uses are diffi cult to justify.

There is also one other mistaken usage of the TOEIC that needs to be mentioned

here. Many schools offer courses that focus primarily on TOEIC preparation; i.e.,

that are taught mainly by drilling students on TOEIC-like questions. While having

knowledge of and some experience with taking the test is helpful, once test takers

understand the format and timing of the TOEIC, continuing to study English by 　　　　　　　　　　　　　　　　　　　　　　　　　　　　12 This quote is from IIBC, 2014, p. 3.13 These quotes are from Brown, 1995, pp. 13-14.



answering such questions is inefficient and ineffective. As one professional language

test writer with detailed knowledge of ETS’s tests has stated, “TOEIC is a test, not a

language, so teaching TOEIC is not really an option. The best thing to do is to teach

English focusing on profi ciency . . . to stress ability over knowledge.” “First the student

needs to be motivated to learn English and NOT simply to pass the test.”14 Yet, many

students, administrators, and teachers believe that studying TOEIC-like questions and

materials is the best way for students to improve their English abilities and TOEIC

scores, and there are a great many college and university textbooks published and

courses taught each year solely for this purpose. However, as the TOEIC is a test

of English language proficiency, there should be no specific materials that will help

students to truly improve their TOEIC scores. Therefore, it would be much better

to aim at improving the various English language abilities overall than at answering

TOEIC-like questions correctly.

3．Subjects, materials, methods, and research questionWhat follows is a detailed examination of student TOEIC scores, focusing

primarily on measurable progress made in terms of increases in scores. The TOEIC

IP Test scores examined in this study were achieved by fi rst-year students, who were

not English majors, in one department at a Japanese university. Though the scores of

students from 2004 to 2013 are presented, the emphasis is on the scores attained by two

groups within this department in 2011, 2012, and 2013, when a special program had been

instituted, which required those students to take one extra English language class during

their first year. Students in this program were both recommended and required to

take some content classes that included more usage of English than the regular classes

off ered. The two groups are those who joined this program and those who did not. The

students in both groups, and those for all ten years, were required to take the TOEIC IP

Test on campus twice during the school year, around the eleventh week of each fi fteen-

week semester.15　　　　　　　　　　　　　　　　　　　　　　　　　　　　14 These quotes are from Wood, 2010, p. 44.15 From the 2004 to the 2011 school year, all fi rst-year students in this department were required to take

and pass three English language classes each semester. In the 2012 and 2013 school years, they were only required to take and pass two. So, the students in the new program took four classes in 2011 and three in 2012 and 2013. All of the approximately 200 students who entered the department each year were eligible to apply for this new program just before they began their studies at the university. From this self-selected group, 40 students were chosen based on their entrance examination scores and high school grades each year. Before the beginning of the second semester, students were again asked to apply for the new program

4



The main research question is: Did requiring students to take and pass one

additional English language course cause them to increase their TOEIC score more than

students who were not required to do so?

4 . Results

Table 1Basic Statistics for Total Scores of All Students in This Study

Year-Test N Max Min Var Mean Med SD2004-1 213 890 235 655 462 450 86.1 2004-2 204 695 265 430 472 470 85.1 2005-1 207 725 210 515 446 445 84.7 2005-2 200 705 230 475 435 435 85.3 2006-1 207 675 160 515 469 480 83.6 2006-2 205 770 230 540 479 480 94.6 2007-1 224 820 230 590 461 455 82.0 2007-2 218 835 280 555 476 473 88.1 2008-1 210 710 275 435 466 470 73.1 2008-2 206 735 275 460 484 483 86.2 2009-1 199 685 280 405 479 480 79.5 2009-2 194 730 255 475 491 490 88.3 2010-1 200 710 285 425 500 495 80.9 2010-2 186 760 165 595 519 510 86.1 2011-1 215 740 265 475 500 500 87.7 2011-2 207 800 280 520 520 520 90.9 2012-1 208 740 285 455 505 510 93.3 2012-2 207 820 305 515 498 495 94.3 2013-1 194 760 260 500 498 497.5 92.8 2013-2 197 840 225 615 519 515 104.5

　　　　　　　　　　　　　　　　　　　　　　　　　　　　

if they wished, as the total number would be increased to 50. Those who joined later needed to take the fi rst semester’s additional English class some time before graduation, hopefully in the fi rst semester of their second year. No attempt was made to balance the female to male ratio of students in the program. The chart below provides details.

Number Number accepted Number dropped Number Number accepted Total numberapplied in in April out in July applied in in September in September

Year April Female Male Female Male September Female Male Female Male2011 78 22 18 1 1 12 5 7 26 242012 92 22 18 2 1 23 4 9 24 262013 95 20 20 0 0 10 2 8 22 28



Table 2Basic Statistics for Listening Scores of All Students in This Study

Year-Test N Max Min Var Mean Med SD2004-1 213 465 150 315 249 250 47.9 2004-2 204 375 140 235 247 245 45.5 2005-1 207 395 145 250 252 255 46.4 2005-2 200 425 115 310 242 245 46.8 2006-1 207 380 100 280 264 265 48.4 2006-2 205 415 125 290 269 270 53.1 2007-1 224 435 125 310 255 255 46.8 2007-2 218 465 115 350 257 255 49.5 2008-1 210 390 110 280 244 240 47.3 2008-2 206 390 120 270 269 270 44.6 2009-1 199 385 125 260 243 240 41.4 2009-2 194 410 145 265 263 260 45.8 2010-1 200 385 155 230 268 265 43.8 2010-2 186 405 90 315 275 275 47.4 2011-1 215 400 135 265 260 255 49.3 2011-2 207 410 120 290 269 275 50.1 2012-1 208 385 140 245 266 265 47.3 2012-2 207 430 145 285 266 260 50.4 2013-1 194 385 105 280 264 255 49.4 2013-2 197 440 95 345 276 275 58.4

Table 3Basic Statistics for Reading Scores of All Students in This Study

Year-Test N Max Min Var Mean Med SD2004-1 213 425 60 365 212 210 52.8 2004-2 204 335 85 250 225 225 53.5 2005-1 207 330 55 275 194 190 51.4 2005-2 200 310 75 235 194 192.5 51.3 2006-1 207 335 60 275 205 210 49.4 2006-2 205 360 80 280 210 210 55.2 2007-1 224 385 90 295 206 200 48.9 2007-2 218 370 95 275 219 215 51.2 2008-1 210 385 115 270 222 220 47.3 2008-2 206 385 85 300 216 215 55.1 2009-1 199 360 95 265 236 235 51.3 2009-2 194 410 100 310 228 230 55.3 2010-1 200 340 95 245 232 235 48.3 2010-2 186 380 75 305 244 245 50.1 2011-1 215 380 115 265 240 240 51.4 2011-2 207 405 110 295 250 255 55.0 2012-1 208 395 105 290 239 235 58.4 2012-2 207 395 110 285 232 230 57.9 2013-1 194 375 110 265 233 230 55.7 2013-2 197 400 90 310 243 245 55.3

6



Tables 1, 2, and 3 present the basic statistics for the TOEIC scores, Total,

Listening, and Reading, respectively, on Test 1 and Test 2 achieved by all of the students

in this department during a ten-year period, from 2004 to 2013. Almost all of the paired

means and medians are quite similar to each other, and none are very diff erent. There

are very few scores that are more than three standard deviations greater than or less

than the associated means, though this information is not included in the tables. These

facts indicate that the scores for each test administration exhibit a normal distribution.

Also, all of the variations in the achieved scores are fairly wide considering the possible

variations in scores are 980 (10 to 990) for the Total scores and 490 (5 to 495) for the

Listening scores and for the Reading scores. This would be expected in a normal

distribution of scores.

In most cases, the maximum scores, minimum scores, means, and medians

are a little higher on the second test administration than on the fi rst, but not always.

Concerning the Total scores for the ten pairs of test administrations, eight of the

maximum scores, six of the minimum scores, eight of the means, and seven of the

medians are higher on Test 2 than on Test 1. For the Listening scores, eight of the

maximum scores, four of the minimum scores, seven of the means, and six of the

medians are higher on Test 2 than on Test 1. For the Reading scores, five of the

maximum scores, six of the minimum scores, six of the means, and six of the medians

are higher on Test 2 than on Test 1. All of these diff erences between Test 2 and Test

1 results, though, are small. Simply due to prior experience taking the test, generally

higher scores would be expected on the second administration than the first, though

such results are not always the case. One possible cause for this would be that, being a

standardized, norm-referenced test of English profi ciency and not a criterion-referenced

test made specifically for these students, the TOEIC did not measure the students’

English abilities very precisely. To account for this lack of precision, standard errors

need to be used to determine if scores are truly diff erent or only appear to be diff erent

due to factors inherent in the testing instrument. Details concerning this will be given

later. However, use of these standard errors indicates that none of these paired Test 2

and Test 1 means or medians are truly diff erent; i.e., no measurable group progress or

deterioration in English abilities was determined.

As a general trend, the means and medians for all three scores appear to increase

from 2004 to 2013, but they do not do so continuously or consistently. However, even



within the whole range of the ten years of scores, only the Total scores’ means and

medians for both tests in 2005 are truly lower than any other means and medians,

respectively, and none of the Listening scores’ means or medians are lower than any

of the others. All of the Reading scores’ means and medians in the later years are

statistically higher than some of the earlier years’ means and medians, respectively. So,

this department’s students as groups statistically showed possible improvement over the

ten years only in their Reading scores, but not in their Listening scores or their Total

scores, if we assume that TOEIC scores have remained objectively equivalent over this

time period.

Also, almost every Total scores’ mean and median from the second test in 2008

to the second test in 2013 is higher than almost every earlier Total scores mean and

median, respectively. All of the Reading scores’ means and medians from the first

test in 2009 to the second test in 2013 are greater than all of the previous means and

medians, respectively. The Listening scores’ means and medians show more variability

throughout the ten years, but generally, from the second test in 2009 until the second

test in 2013, the means and medians are larger than they are from the fi rst test in 2004

to the first test in 2009, respectively. Changes were made to the TOEIC that first

appeared on these tests in the first administration in 2007. This may have resulted

in the generally higher scores that appear later. It is also possible that the students

enrolled in the later years were generally a little better at English.

Table 4TOEIC IP Test Scores of All Post High School Students in Japanwith Majors Similar to Those of the Students in this Study*

Number ofStudents

AverageTotal

AverageListening

AverageReadingYear

2004 52,891 418 235 1832005 56,240 433 246 1872006 63,138 425 242 1832007 60,046 408 231 1772008 60,953 412 230 1822009 61,779 424 233 1912010 63,399 433 242 1912011 67,784 437 241 1962012 58,264 422 237 1862013 62,145 430 239 191

* These students’ majors in 2004 to 2006 were Business, Economics, Law, or Sociology, in 2007 to 2011 were Commerce or Finance, and in 2012 and 2013 were Commerce, Economics, or Finance. These figures come from IIBC, (corresponding year), p. 7 (2004 to 2006) and p. 8 (2007 to 2013).

8



Table 4 displays the average TOEIC IP Test scores, Total, Listening, and

Reading, for all post high school students in Japan who were majoring in fi elds similar

to those of the students in the department whose scores are being examined in this

study. Comparing these average scores with those in tables 1, 2, and 3, we fi nd that all

of the average scores of the fi rst-year students in this department are higher than those

achieved overall nationally by students studying similar subjects. The implication would

be that these students’ English abilities are above the average for students in Japan with

similar academic and vocational interests.

Table 5TOEIC IP Test Total Scores of All First-Year University Students in Japan with Majors Similar to Those of the Students in This Study and of the Students in This Study

All 1st-YearUniversityStudents

with SimilarMajor*

All Studentsin This Study(1st-Year atUniversity)on Test 1


All 1st-YearUniversityStudents*Year

2004 387 N=91,853 NA 462 4722005 401 N=108,636 NA 446 4352006 401 N=132,470 NA 469 4792007 406 N=146,901 378 N=26,890 461 4762008 405 N=148,772 383 N=26,752 466 4842009 412 N=152,937 400 N=27,287 479 4912010 419 N=161,976 411 N=29,174 500 5192011 421 N=183,609 408 N=31,743 500 5202012 412 N=186,793 399 N=31,674 505 4982013 423 N=211,169 409 N=35,167 498 519

* These students’ majors in 2004 to 2006 were Business, Economics, Law, or Sociology, in 2007 to 2011 were Commerce or Finance, and in 2012 and 2013 were Commerce, Economics, or Finance. These fi gures come from IIBC, (corresponding year), p. 8 (2004 and 2005), p. 9 (2006), pp. 9 and 11 (2007 to 2011, and pp. 7 and 8 (2012 and 2013).

Tables 5, 6, and 7 present comparisons similar to what was mentioned in the

previous paragraph for average Total scores, Listening scores, and Reading scores,

respectively. Here, the average TOEIC IP Test scores of fi rst-year students throughout

Japan are presented in two specific groups: all first-year university students and all

fi rst-year university students with similar majors. Again, the average scores achieved

by the students in this department seem very favorable compared with those achieved

by students across Japan who are in similar circumstances. They are, again, always

higher.



Table 6TOEIC IP Test Listening Scores of All First-Year University Students in Japan with Majors Similar to Those of the Students in This Study and of the Students in This Study


with SimilarMajor*






Table 7TOEIC IP Test Reading Scores of All First-Year University Students in Japan with Majors Similar to Those of the Students in This Study and of the Students in This Study


with SimilarMajor*






10



Table 8Basic Statistics for Extra English Course Students' Total Scores

Year-Test N* Max Min Var Mean Med SD2011-1 49 740 395 345 556 550 69.7 2012-1 49 735 345 390 568 595 86.1 2013-1 42 760 365 395 591 585 70.1 2011-2 49 795 370 425 572 580 76.9 2012-2 49 820 375 445 545 515 94.0 2013-2 42 840 400 440 628 627.5 88.5

* All students who were in the Extra English Course in the second semester and took the TOEIC on campus both times.

Table 9Basic Statistics for Extra English Course Students' Listening Scores

Year-Test N* Max Min Var Mean Med SD2011-1 49 400 180 220 298 300 43.2 2012-1 49 385 160 225 296 305 46.2 2013-1 42 385 225 160 308 305 44.4 2011-2 49 410 205 205 298 285 47.8 2012-2 49 430 200 230 296 295 48.6 2013-2 42 440 210 230 334 345 55.3


Table 10Basic Statistics for Extra English Course Students' Reading Scores

Year-Test N* Max Min Var Mean Med SD2011-1 49 355 190 165 258 250 42.4 2012-1 49 395 130 265 272 270 55.2 2013-1 42 375 135 240 283 277.5 41.2 2011-2 49 385 145 240 274 280 48.9 2012-2 49 395 140 255 249 245 62.8 2013-2 42 400 190 210 294 295 43.7


Tables 8, 9, and 10 give the basic statistics for the TOEIC Total, Listening, and

Reading scores, respectively, of the students in the special program that includes one

extra compulsory English class, the Extra English Course, from 2011 to 2013. Although

very few scores are more than three standard deviations greater or less than the

associated means, information not included in the table, the means and medians are

sometimes not quite as similar as one would expect in a data set that is normally

distributed. However, this is not surprising since the populations are so small. In most



cases, the maximum and minimum scores are higher on the second test administration

than the first, the only differences being for the minimum Total score in 2011, the

minimum Listening score in 2013, the maximum Reading score in 2012, and the

minimum Reading score in 2011.

Table 11Basic Statistics for Not Extra English Course Students' Total Scores


* All students who were not in the Extra English Course in the second semester and took the TOEIC on campus both times.

Table 12Basic Statistics for Not Extra English Course Students' Listening Scores

Year-Test N* Max Min Var Mean Med SD2011-1 155 355 135 220 247 245 44.5 2012-1 152 360 140 220 258 257.5 42.0 2013-1 144 355 105 250 252 250 42.8 2011-2 155 395 120 275 261 260 47.7 2012-2 152 395 145 250 256 245 47.4 2013-2 144 370 95 275 260 260 46.3


Table 13Basic Statistics for Not Extra English Course Students' Reading Scores



Tables 11, 12, and 13 display the basic statistics for the TOEIC scores of the

students who were not in the special program of study from 2011 to 2013. Very few of

12



the scores are greater than or less than three standard deviations from the associated

means, but this data is not included in the tables. Only one mean and median are

not very similar, for the Listening scores on Test 2 in 2012, but they are also not too

diff erent. So, all of these sets of scores are likely normally distributed. More of the

maximum and minimum scores are greater on Test 2 than on Test 1, but there are also

many that are not.Table 14

Average TOEIC IP Test Total Scores in JapanAll 1st-YearUniversityStudents

with SimilarMajor*

All EECStudents

in This Studyon Test 1

All Not EECStudents


All EECStudents


All Not EECStudents



2011 421 408 562 (N=40) 485 (N=175) 572 (N=49) 504 (N=158)2012 412 399 555 (N=40) 493 (N=169) 545 (N=49) 483 (N=158)2013 423 409 603 (N=36) 474 (N=158) 619 (N=44) 490 (N=153)* These students' majors in 2011 were Commerce or Finance and in 2012 and 2013 were Commerce, Economics, or Finance. These fi gures come from IIBC, (corresponding year), pp. 9 and 11 (2011) and pp. 7 and 8 (2012 and 2013).

Table 15Average TOEIC IP Test Listening Scores in Japan


with SimilarMajor*

All EECStudents


All Not EECStudents


All EECStudents


All Not EECStudents




Table 16Average TOEIC IP Test Reading Scores in Japan


with SimilarMajor*

All EECStudents


All Not EECStudents


All EECStudents


All Not EECStudents






Tables 14, 15, and 16 give the average TOEIC Total, Listening, and Reading

scores, respectively, for all fi rst-year university students in Japan, all fi rst-year university

students in Japan who were majoring in fields similar to that of the students whose

scores are being used in this study, all of the students in the Extra English Course, and

all of the students who were not members of the Extra English Course. The average

scores of the students in both the EEC groups and the Not ECC groups on both test

administrations are clearly higher than the average scores of all first-year university

students in Japan and of all first-year university students in Japan studying subjects

similar to that of the students who achieved the scores that will be further examined.

Table 17Diff erences in Extra English Course Students' and Not Extra English Course Students'

Total Scores' Means and MediansEECMean

Not EECMean

Diff erencein Means

EECMedian

Not EECMedian

Diff erencein MediansYear-Test

2011-1 556 480 76 550 480 702012-2 572 504 68 580 500 802013-1 568 486 82 595 490 1052011-2 545 482 63 515 475 402012-1 591 470 121 585 470 1152013-2 628 489 139 627.5 485 142.5

* Only students who took the TOEIC both times on campus are included. Students were assigned to EEC or Not EEC based on which they were in during the second semester.

* EEC: 2011 N=49, 2012 N=49, 2013 N=42* Not EEC: 2011 N=155, 2012 N=152, 2013 N=144


Listening Scores' Means and MediansEECMean

Not EECMean

Diff erencein Means

EECMedian

Not EECMedian


2011-1 298 247 51 300 245 552012-1 298 261 37 285 260 252013-1 296 258 38 305 257.5 47.52011-2 296 256 40 295 245 502012-2 308 252 56 305 250 552013-2 334 260 74 345 260 85

* Only students who took the TOEIC both times on campus are included. Students were assigned to EEC or Not EEC based on which they were in during the second semester.


14




Reading Scores' Means and MediansEECMean

Not EECMean

Diff erencein Means

EECMedian

Not EECMedian


2011-1 258 233 25 250 235 152012-2 274 244 30 280 240 402013-1 272 228 44 270 230 402011-2 249 225 24 245 225 202012-1 283 218 65 277.5 220 57.52013-2 294 229 65 295 230 65

* Only students who took the TOEIC both times on campus are included.Students were assigned to EEC or Not EEC based on which they were in during the second semester.


Tables 17, 18, and 19 present the diff erences in the means and medians for the

Total, Listening, and Reading scores, respectively, between the Extra English Course

students and those students who were not in the Extra English Course. These show

which group of students performed better on each TOEIC administration. However,

to determine if two TOEIC scores are truly diff erent statistically, the standard error

of diff erence (SEdiff ) must be used. ETS states that these measures are about +/-35

points for both the Listening scores and the Reading scores. Since these two scores are

simply added to calculate the Total score, I will assume that the SEdiff for the Total

scores is +/- 70 points, which is not provided by ETS. Using one SEdiff provides about

68% certainty concerning the similarity or diff erence of two scores, and by implication in

ability. To be 95% certain, one should use two SEdiff , or +/-69 points for Listening and

Reading scores and +/-138 points for Total scores. 16

Looking at Table 17, we can see that four of the EEC Total scores’ means and

four of the medians are more than one SEdiff, 70 points, greater than the Not EEC

Total scores’ means and medians, indicating a statistically true difference with 68%

certainty. Only one of each, for the second test in 2013, show a true diff erence with

95% certainty, as the diff erences are more than 138 points. In Table 18, we fi nd that all

of the EEC students’ Listening scores’ means and all but one of the medians are more

than one SEdiff , 35 points, greater than the Not EEC students’ means and medians, and

one of each, those for Test 2 in 2013, are more than two SEdiff , 69 points, greater. So,

　　　　　　　　　　　　　　　　　　　　　　　　　　　　16 For these details, see CGI, 1998, pp. I-3 and IV-6.



we can be about 68% certain that most of these EEC students’ Listening scores’ means

and medians are truly higher than most of the Not EEC students’ means and medians,

and we can be 95% certain that one pair of them is truly higher. As for the Reading

scores, Table 19 indicates that we can be about 68% certain that three of the EEC

students’ Reading scores’ means and four of their medians are truly greater than those

of the Not EEC students, because of being more than 35 points higher. These analyses

suggest that the EEC students performed consistently better as a group than the Not

EEC students as a group on the Listening section of the TOEIC while they did so only

half of the time on the Reading section. On both test administrations in 2013 and on the

fi rst test administration in 2012, all three scores’ means and medians imply that the EEC

students as a group had stronger English abilities than the Not EEC students as a group.

Table 20Diff erences in Extra English Course Students' Total Scores' Means and Medians

T1Mean

T2Mean

Diff erencein Means

T1Med

T2Med

Diff erencein MediansYear N*

2011 49 556 572 16 550 580 302012 49 568 545 -23 595 515 -802013 42 591 628 37 585 627.5 42.5


Table 21Diff erences in Not Extra English Course Students' Total Scores' Means and Medians

T1Mean

T2Mean

Diff erencein Means

T1Med

T2Med

Diff erencein MediansYear-Test N*

2011 155 480 504 24 480 500 202012 152 486 482 -4 490 475 -152013 144 470 489 19 470 485 15


Table 22Diff erences in Extra English Course Students' Listening Scores' Means and Medians

T1Mean

T2Mean

Diff erencein Means

T1Med

T2Med


2011 49 298 298 0 300 285 -152012 49 296 296 0 305 295 -102013 42 308 334 26 305 345 40


16



Table 23Diff erences in Not Extra English Course Students' Listening Scores' Means and Medians

T1Mean

T2Mean

Diff erencein Means

T1Med

T2Med


2011 155 247 261 14 245 260 152012 152 258 256 -2 257.5 245 -12.52013 144 252 260 8 250 260 10


Table 24Diff erences in Extra English Course Students' Reading Scores' Means and Medians

T1Mean

T2Mean

Diff erencein Means

T1Med

T2Med


2011 49 258 274 16 250 280 302012 49 272 249 -23 270 245 -252013 42 283 294 11 277.5 295 17.5


Table 25Diff erences in Extra English Course Students' Listening Scores' Means and Medians

T1Mean

T2Mean

Diff erencein Means

T1Med

T2Med


2011 49 298 298 0 300 285 -152012 49 296 296 0 305 295 -102013 42 308 334 26 305 345 40


Tables 20 through 25 display the diff erences in the Total scores’, Listening scores’,

and Reading scores’ means and medians for the two TOEIC administrations each year

achieved by the EEC students and the Not EEC students. These show how much

the TOEIC scores of the students as groups improved or did not improve each year.

Comparing these differences reveals inconsistent results between the two groups in

their actual achieved gains in means and medians. For the changes in the Total scores’

means and medians, the amounts are fairly similar for 2011; for 2012 it appears that the

Not EEC group outperformed the EEC group; and the opposite seems to be the case for

2013. As for the changes in the Listening scores’ means and medians, it seems that in

2011 the Not EEC students improved more than the EEC students; in 2012 both groups

of students appear to have performed similarly; while the EEC students’ improvement

seems to be greater than the Not EEC students’ in 2013. Concerning the changes in



the means and medians for the Reading scores, the EEC students’ change in median in

2011 is greater than that of the Not EEC students, but the changes in their means are

very similar; in 2012 the Not EEC students’ changes appear to be greater than those

of the EEC students; and in 2013 the changes for both groups seem to be very similar.

However, if we use the SEdiff discussed earlier, we find that none of these achieved

differences between changes in means or medians indicate true differences in these

changes. Therefore, both groups improved or did not improve by the same amounts in

all cases.

Table 26Numbers of Students Whose TOEIC Total Scores Changed by More Than 70

and 138 PointsNo. >70 No. >138 No. <70 No. <138

EEC 2011 N=49 8　16% 0 0% 1　　2% 1 2%Not ECC 2011 N=155 38　25% 5 3% 9　　6% 1 1%EEC 2012 N=49 2　　4% 0 0% 10　20% 0 0%Not EEC 2012 N=152 16　11% 4 3% 24　16% 3 2%EEC 2013 N=42 11　21% 3 7% 1　　2% 0 0%Not EEC 2013 N=144 30　21% 3 2% 10　　7% 0 0%

Table 27Numbers of Students Whose TOEIC Listening Scores Changed by More

Than 35 and 69 PointsNo. >35 No. >69 No. <35 No. <69

EEC 2011 N=49 8 16% 4　　8% 8　16% 2 4%Not ECC 2011 N=155 45 29% 11　　7% 17　11% 4 3%EEC 2012 N=49 10 20% 4　　8% 12　24% 3 6%Not EEC 2012 N=152 27 18% 7　　5% 31　20% 9 6%EEC 2013 N=42 16 38% 6　14% 1　　2% 0 0%Not EEC 2013 N=144 37 26% 9　　6% 13　　9% 2 1%

Table 28Numbers of Students Whose TOEIC Reading Scores Changed by More Than

35 and 69 PointsNo. >35 No. >69 No. <35 No. <69

EEC 2011 N=49 15　31% 6　12% 5 10% 1　　2%Not ECC 2011 N=155 37　24% 12　　8% 15 10% 6　　4%EEC 2012 N=49 2　　4% 1　　2% 17 35% 7　14%Not EEC 2012 N=152 27　18% 8　　5% 32 21% 12　　8%EEC 2013 N=42 11　21% 2　　5% 4 10% 1　　2%Not EEC 2013 N=144 38　26% 9　　6% 15 10% 4　　3%

18



Tables 26, 27, and 28 show the numbers and percentages of students in each of

the two groups whose Total, Listening, and Reading scores, respectively, on the TOEIC

IP Test were more than the SEdiff diff erent on the second test administration than the

fi rst. The fi rst two columns display increases in scores by one and two SEdiff , and the

last two columns display decreases in scores by one and two SEdiff .

Concerning Total scores, we fi nd that in 2011 and 2012 greater percentages of

the Not EEC students’ scores were calculated, with about 68% certainty, to have truly

increased than of the EEC students’ scores, while in 2013 the same percentages of

students’ scores were determined to have increased with about 68% certainty. In 2012,

a slightly greater percentage of the EEC students’ scores than the Not EEC students’

scores indicated, with about 68% certainty, that they had truly decreased. All of the

other percentages of changes in scores in Table 26 are quite small.

As for the changes in Listening scores, greater percentages of the students’

scores in the Not EEC group appear, with about 68% certainty, to have increased than

in the EEC group in 2011, and the opposite seems to have been true in 2012 and 2013,

though only very slightly in 2012. In 2013, we also see that 14% of the EEC students’

scores indicated a true increase with about 95% certainty. In 2011 and 2012, greater

percentages of the EEC students’ scores than the Not EEC students’ scores were

calculated to have decreased, with 68% certainty. The rest of the percentages of

increases and decreases in Table 27 are rather small.

The changes in Reading scores show that, with 68% certainty, higher percentages

of the EEC students’ scores increased than of the Not EEC students’ scores in 2011,

while greater percentages of the Not EEC students’ scores than of the EEC students’

scores increased in 2012 and 2013. We also see that a slightly higher percentage of

the EEC students’ scores than the Not EEC students’ scores increased with about 95%

certainty in 2011, 12% versus 8%, respectively. In 2011 and 2013, the same percentages

of students’ scores in both groups seem to have decreased with about 68% certainty,

while in 2012 a higher percentage of the EEC students’ scores than the Not EEC

students’ scores appear to have decreased with about 68% and 95% certainty, 14% and

8%, respectively. The other percentages of changes in Table 28 are rather small.

So, we see very mixed results in the comparisons of the changes in students’

TOEIC scores between these two groups. With about 68% confidence, we see some

diff erences year by year. However, with 95% confi dence, there are few students’ scores



that changed very much, and the percentages of students are usually quite similar in the

two groups.

The highest percentages of increases, with about 68% confidence, were in the

Total scores for the students’ scores in the Not ECC group in 2011, 25%, and for both

groups’ students’ scores in 2013, 21% each. As for the Listening scores, with about

68% confi dence, 29% and 26% of the Not EEC students’ scores increased in 2011 and

2013, respectively, and 20% and 38% of the EEC students’ scores increased in 2012 and

2013, respectively. Concerning the Reading scores, in 2011, we can have about 68%

confidence that 31% of the EEC students’ scores and 24% of the Not EEC students’

scores increased. With the same 68% confi dence, it can be claimed that 21% of the EEC

students’ scores and 26% of the Not EEC students’ scores increased in 2013.

There were also percentages of decreases in scores with 68% confidence that

were similarly high in amount to the greatest increases in scores in some cases; for

the EEC students’ Total scores in 2012, 20%, for the EEC and the Not EEC students’

Listening scores in 2012, 24% and 20%, respectively, and for the EEC and the Not EEC

students’ Reading scores in 2012, 35% and 21%, respectively. These percentages of

decreases are difficult to explain, given that the company that produces the TOEIC

will issue proof of a score for up to two years after it has been achieved, indicating

that it claims the test scores remain valid during that time period.17 Perhaps the best

explanation for these decreases would be that “jumping around is in the nature of

TOEIC scores,”18 which could also explain the increases.

TOEIC scores by individuals often vary greatly from one administration to

another, at least partially because the SEdiff is rather large. This variety in achieved

scores may be the reason for decreases in scores, and it also may be the reason for

increases in scores. Teachers and administrators “can expect that by chance alone

(students) will (sometimes) achieve a score that is higher that their true score,”19 and

they will sometimes achieve a score that is lower than their true score. Obviously,

caution must be the rule when trying to make use of TOEIC scores for understanding

the English language abilities of individuals.

　　　　　　　　　　　　　　　　　　　　　　　　　　　　17 ETS (2013a, p. 22; 2013b, p. 11; 2014a) suggests that a TOEIC score is acceptable for up to two years.18 This quote is from Childs, 1995, p. 73.19 This quote is from Childs, 1995, p. 74.

20



Table 29 Numbers of Students Whose TOEIC Total and Listening Scores Increased by

More Than 70/138 and 35/69 PointsNo, >70/35 No. >70/69 No. >138/35 No. >138/69

EEC 2011 N=49 6 12% 3 6% 0 0% 0 0%Not ECC 2011 N=155 27 17% 9 6% 2 1% 2 1%EEC 2012 N=49 1 2% 1 2% 0 0% 0 0%Not EEC 2012 N=152 10 7% 5 3% 2 1% 2 1%EEC 2013 N=42 9 21% 5 12% 3 7% 3 7%Not EEC 2013 N=144 22 15% 8 6% 3 2% 3 2%

Table 30Numbers of Students Whose TOEIC Total and Listening Scores Decreased by

More Than 70/138 and 35/69 PointsNo, <70/35 No. <70/69 No. <138/35 No. <138/69


As a balanced development of language abilities is most favorable, we will

now examine the data from this point of view. Table 29 displays the numbers and

percentages of students in each group whose Total and Listening scores both increased

by more than one and/or two SEdiff . The highest amounts are in 2011, with 12% of

the EEC students and 17% of the Not EEC students, and in 2013, with 21% of the EEC

students and 15% of the Not EEC students, achieving both a Total score and a Listening

score that are more than one SEdiff higher on the second test administration than the

fi rst. Also, in 2013, 12% of the EEC students and 6% of the Not EEC students achieved

a Total score that was one SEdiff higher and a Listening score that was two SEdiff

higher on the second administration than the fi rst. In 2012, Table 30, which presents

decreases in both Total and Listening scores by one and/or two SEdiff , shows that 10%

of the EEC students and 13% of the Not EEC students achieved both Total scores and

Listening scores that were more than one SEdiff lower on the second sitting than on the

fi rst.



Table 31Numbers of Students Whose TOEIC Total and Reading Scores Increased by

More Than 70/138 and 35/69 PointsNo, >70/35 No. >70/69 No. >138/35 No. >138/69


Table 32 Numbers of Students Whose TOEIC Total and Reading Scores Decreased by

More Than 70/138 and 35/69 PointsNo, <70/35 No. <70/69 No. <138/35 No. <138/69


Tables 31 and 32 present the same sort of statistics for Total and Reading scores

as the two tables immediately above present for Total and Listening scores, and the

trends of increases and decreases are similar. In 2011 and 2013, increases in students’

Total and Reading scores by more than one SEdiff on the second administration from

the fi rst are indicated for 8% of the EEC students and 15% of the Not EEC students and

for 17% of the EEC students and 15% of the Not EEC students, respectively. In 2012,

there are decreases of more than one SEdiff in both the Total scores and Reading scores

for 16% of the EEC students and 9% of the Not EEC students, and there are decreases

of more than one SEdiff in Total score and two SEdiff in Reading score for 10% of the

EEC students and 4% of the Not EEC students.

Table 33Numbers of Students Whose TOEIC Listening and Reading Scores Increased

by More Than 35/69 and 35/69 PointsNo, >35/35 No. >35/69 No. >69/35 No. >69/69


22



Table 34 Numbers of Students Whose TOEIC Listening/Reading Scores Decreased by

More Than 35/35 and 69/69 PointNo, <35/35 No. <35/69 No. <69/35 No. <69/69


Table 33 displays the numbers and percentages of students in both groups whose

Listening and Reading scores both increased by one and/or two SEdiff . Only in 2013

are such increases in 10% or more of the students’ scores found, with 12% of the EEC

students and 10% of the Not EEC students achieving scores in both sections more than

one SEdiff higher on the second test administration than the fi rst. Table 34 shows that

none of the students achieved both Listening scores and Reading scores on the second

sitting that were more than one SEdiff lower than on the fi rst.

Table 35Numbers of Students Whose TOEIC Total, Listening, and Reading Scores

Increased by More Than 70, 35/69, and 35/69 PointsNo, >70/35/35 No. >70/69/35 No. >70/35/69 No. >70/69/69



Increased by More Than 138, 35/69, and 35/69 PointsNo, >138/35/35 No. >138/69/35 No. >138/35/69 No. >138/69/69




Tables 35 and 36 display the numbers and percentages of students in each group

whose Total scores, Listening scores, and Reading scores all increased by more than

one and/or two SEdiff . Only in 2013 can we fi nd such increases in 10% or more of the

students’ scores, with 12% of the EEC students and 10% of the Not EEC students having

achieved scores one SEdiff higher on the second administration than the first for all

three measures.


Decreased by More Than 70, 35/69, and 35/69 PointsNo, <70/35/35 No. <70/69/35 No. <70/35/69 No. <70/69/69



Decreased by More Than 138, 35/69, and 35/69 PointsNo, <138/35/35 No. <138/69/35 No. <138/35/69 No. <138/69/69


Tables 37 and 38 display the numbers and percentages of students in each group

whose Total scores, Listening scores, and Reading scores all decreased by more than

one and/or two SEdiff . They indicate that there are no cases of 10% or more of the

students having achieved scores that were more than one SEdiff lower on the second

sitting than the fi rst in all three measures.

5 . Discussion, conclusions, and recommendationsAs the EEC students studied in more English classes and likely had more interest

in improving their English, since they chose this course of study that requires more

English classes and usage, why did they not noticeably improve their TOEIC scores

24



more than the students who did not belong to the EEC program? The reason for this

is that even though they studied English more than the other students, they still did not

study enough to ensure that most of them would improve their TOEIC scores. Only

one study, by Saegusa using “thousands of TOEIC scores,”20 has attempted to equate

amount of classroom study time with TOEIC score gains and to make predictions of

the amounts of classroom English language study time needed for learners to improve

TOEIC scores by certain amounts. The main fi ndings are cited below.

“less than 80 hours of (English language) instruction is not very

eff ective. In such classes, a majority will make little or no progress.

If eff ectiveness is given top priority, at least more than 100 hours of

instruction, and ideally 200 hours of instruction, as a unit should be

recommended.”21

“It usually takes more time to improve English proficiency than is

generally believed. Our studies show that it will take an average of

400 hours of instruction to raise the profi ciency of TOEIC 450 . . . to

that of TOEIC 600 . . . . The general defi nition of (TOEIC 450) is the

elementary profi ciency of (survival English); and that of (TOEIC 600) is

the minimum working profi ciency. This distinction is very important,

because (TOEIC 600) can be a criterion upon which to distinguish

between working and non-working profi ciency. To successfully carry

out business in English, however, a higher level . . . roughly equivalent

to TOEIC 730 . . . will be required, and to reach that level it is

estimated that another 400 hours of instruction will be needed.”22

The implications from this study are often stated when the question of how much

study time is needed to improve TOEIC scores comes up, though the source is not often

provided.23 Yet, according to ETS’s explanations of standard errors, these estimates of 　　　　　　　　　　　　　　　　　　　　　　　　　　　　20 This quote is from Saegusa, 1985, p. 165.21 This quote is from Saegusa, 1985, p. 174.22 This quote is from Saegusa, 185, p. 181. TOEIC scores in parentheses have been substituted for another

test’s scores based on equivalencies provided by Saegusa in his article.23 See Wood, 2010, p. 42 for an example of this. However, see Prolingua for an example of both mentioning the

source and expanding on Saegusa’s results.



time needed for certain TOEIC score gains by a majority of the learners are 29% too

short, because Saegusa used standard errors of measurement where he should have

used standard errors of diff erence.24

It also should be noted that the typical college or university class may not be as

conducive to improvements in English as the classes in which the learners whose scores

were studied by Saegusa were in. At college and university in Japan, students usually

take two completely unrelated English classes in each of the two 15-week semesters

during their fi rst year, each class meeting for 90 minutes once a week. So, they will

study English in class for 45 hours per semester, with about two months of summer

vacation between the two semesters. Class size usually varies from 20 to 50 students

per class. Many college and university students will have a native English-speaking

teacher in one of these classes and a Japanese teacher in the other. The required

attendance rate for passing classes is usually 67%.

Saegusa’s subjects were company workers, mainly in their 20s and 30s, holding

college degrees. The usual class size was about ten workers. The classes met one,

two, or three times a week, usually for 120 minutes, for three to six months, with the

total hours of instruction being between 50 and 200. The teachers were native English

speakers. Attendance was approximately 80%.25

We can see from these descriptions that Japan’s college and university programs

usually do not provide enough class hours of English language study to ensure that most

students have the opportunities necessary for them to improve their TOEIC scores.

Even the EEC program only provides 67.5 class hours of English instruction, if a student

attends 100% of the classes.26 So, it is not surprising that so few EEC students, just

like the Not EEC students, achieved truly higher TOEIC scores on the second TOEIC

administration than the fi rst. They had only studied English in class for about 67.5 hours

at the most between tests, less than Saegusa’s minimum requirement of 100 hours and

far less than what he recommended, 200 hours, and what he would have recommended

if he had used the correct standard error, which would then be about 260 hours. Also,

the classes were not integrated, and there had been a two-month period of no classes

　　　　　　　　　　　　　　　　　　　　　　　　　　　　24 This mistake was fi rst pointed out in Bresnihan, 2010, p. 213-214, based on explanations provided by ETS

(CGI, 1998, pp. IV-4-IV-7).25 This information is from Saegusa, 1985, p. 167.26 In 2011, there were 90 total hours of classroom English language instruction for the EEC students. See

footnote 15 for why.

26



in the midst of these classes. Both of these factors make the curriculum unlike the sort

of continuous course of study that Saegusa recommended. This is surely an insuffi cient

amount of classroom English language study time and an inadequate arrangement of

classes for improvements in TOEIC scores to be expected.

Through individual experiences, motivation, and talent, a small number of

students will have the opportunities to make sufficient effort and be able to improve

their English abilities and their TOEIC scores. However, if the goal is to have a majority

of students improve their English abilities, colleges and universities need to provide a

great deal more time for students to study in English language classes aimed at “focusing

on profi ciency rather than rules” and “stress(ing) ability over knowledge”27 and need

to make available a great many more circumstances for students to use English for

communicative purposes outside of their classes. This does not mean that accuracy

should be ignored. Fluency requires accuracy. However, learning and practicing correct

English should be embedded in tasks and activities that require understanding and

communication through English to take place. Providing details about and explanations

of correct English should not be the focus of English language classes; nor should

translating between Japanese and English be the main activity. Students also need to

have opportunities outside of their classes to use the English abilities they are acquiring

and to witness Japanese people using English for communicative purposes regularly.

Possibilities for the latter most likely need to be interactions involving their teachers of

English and other courses.

Below is a list of some of the ways colleges and universities can assist students in

improving their English language profi ciencies and their TOEIC scores.

１．At least six somewhat integrated English language classes should be required

during each semester in which improvement in English is desired. (If only

maintenance of current English abilities is the goal, then half as many may be

sufficient.) The classroom language for students and teachers should primarily be

English. Classes should emphasize both accurate understanding and production of

English in various types of activities, often using the same materials for a number of

similar and diff erent tasks, and responding to English language prompts (for example,

oral and written texts, communications, and questions) in English in a timely manner, 　　　　　　　　　　　　　　　　　　　　　　　　　　　　27 These quotes are from Wood, 2010, p. 44.



i.e., quickly; when appropriate. Students should be told what the purposes are for

the activities they are required to do and what they can do on their own to improve

their English language abilities. Classwork and/or homework should include pairwork

conversations and small group discussions (both based on topics and materials

provided by the teacher and those decided upon by the students), transcribing small

portions of recordings students have made of their own, their classmates’, and their

teachers’ speaking in class and/or outside of class, reading both graded readers and

short one-page articles and essays, listening to short speeches, writing summaries of

and responses to what has been read or listened to or experienced, and answering

questions about what has been read or listened to. Only a few hours of practice

answering TOEIC-type questions should be carried out, shortly before sitting for the

test. Class size should be no more than 20 students.

２．In addition to the native speakers of English, the Japanese teachers of English

need to use English almost exclusively in class, and quite regularly outside of class

with students and with each other. The modeling of these kinds of communicative

interactions and behaviors will have very positive eff ects on the students’ motivation,

aspirations, and self-confidence concerning their usage and studying of English and

their images of themselves as future English language users. The same will be true

for many of the teachers. Other teachers, those not teaching English, also need to

behave similarly in their language usage, and this will benefi t them selves in the same

ways.

３．Funding should be provided and class credits should be awarded for students’

overseas study and other experiences abroad, whether or not the school arranges for

the programs the students participate in. Although short-term programs can help

to increase students’ motivation to use and learn English, programs of six to twelve

months will also help to ensure that their English will actually improve, as well.

４．English language study and usage must be continued throughout all of the students’

years in school, as language abilities decrease over time if they are not used.

28



ReferencesBresnihan, B. (2010). Possible reliability problems affecting use of TOEIC IP Test scores. Kobe:

Institute for Policy Analysis and Social Innovation, University of Hyogo.Brown, J. D. (1995). Diff erences between norm-referenced and criterion-referenced tests. In J. D.

Brown & S. O. Yamashita (Eds.), Language testing in Japan (pp. 12-19). Tokyo: JALT.Brown, J. D., & Yamashita, S. O. (Eds.). (1995). Language testing in Japan. Tokyo: JALT.CGI (The Chauncey Group International). (1998). TOEIC technical manual. Princeton: Author.CGI (The Chauncey Group International). (2000). TOEIC can-do guide: Linking TOEIC scores to

activities performed using English. Princeton: Author. Retrieved March 22, 2014, from https://www.ets.org/Media/Research/pdf/TOEIC_CAN_DO.pdf

Chapman, M. (2004). Insights in language testing: An interview with Kazuhiko Saito. JALT Testing & Evaluation SIG Newsletter, Vol. 8, No. 2, Aug. (pp. 8-11). Tokyo: JALT. Retrieved March 22, 2014, from http://www.jalt.org/test/sai_cha.htm

Childs, M. (1995). Good and bad uses of TOEIC by Japanese companies. In J. D. Brown & S. O. Yamashita (Eds.), Language testing in Japan (pp. 66-75). Tokyo: JALT.

ETS (Educational Testing Service). (2013a). TOEIC examinee handbook: Listening & reading. Princeton: Author. Retrieved March 22, 2014, from http://www.ets.org/Media/Tests/TOEIC/pdf/TOEIC_LR_examinee_handbook.pdf

ETS (Educational Testing Service). 2013b. TOEIC user guide: Listening & reading. Princeton: Author. Retrieved March 22, 2014, from https://www.ets.org/Media/Tests/Test_of_English_for_International_Communication/TOEIC_User_Gd.pdf

ETS (Educational Testing Service). (2014a). 2013 report on test takers worldwide: The TOEIC listening and reading test. Princeton: Author. Retrieved September 19, 2014, from https://www.ets.org/s/toeic/pdf/ww_data_report_unlweb.pdf

ETS (Educational Testing Service). (2014b). Frequently asked questions about the TOEIC listening and reading test. Princeton: Author. Retrieved March 22, 2014, from http://www.ets.org/toeic/test_takers/listening_reading/faq

IIBC (The Institute for International Business Communication). (2005). TOEIC test data & analysis 2004. Tokyo: Author & ETS.




IIBC (The Institute for International Business Communication). (2009a). TOEIC newsletter, no. 105. Digest version. Special feature. 30 years of TOEIC. Tokyo: Author & ETS. Retrieved March 22, 2014, from

　　http://www.toeic.or.jp/library/toeic_data/toeic_en/pdf/newsletter/newsletterdigest105.pdfIIBC (The Institute for International Business Communication). (2009b). TOEIC test data &

analysis 2008. Tokyo: Author & ETS.






IIBC (The Institute for International Business Communication). (2013). TOEIC program data & analysis 2012. Tokyo: Author & ETS.

IIBC (The Institute for International Business Communication). (2014). TOEIC program data & analysis 2013. Tokyo: Author & ETS. Retrieved September 19, 2014. from http://www.toeic.or.jp/library/toeic_data/toeic_en/pdf/data/TOEIC_Program_DAA.pdf

McCrostie, J. (2009). TOEIC no turkey at 30. The Japan Times: Online. Aug. 11. Tokyo: The Japan Times. Retrieved March 22, 2014, from http://www.japantimes.co.jp/community/2009/08/11/issues/toeic-no-turkey-at-30/#.Uy0i-yjB_dk

McCrostie, J. (2010). The TOEIC in Japan: A scandal made in heaven. Shiken: JALT Testing & Evaluation SIG Newsletter, Vol. 14, No. 1, Feb. (pp. 2-11). Tokyo: JALT. Retrieved March 22, 2014, from http://jalt.org/test/mcc_1.htm

Prolingua. (2000). TOEIC info. Tokyo: Prolingua Executive Language Services. Retrieved on March 22, 2014, from http://www.prolingua.co.jp/toeic_e.html

Saegusa, Y. (1985). Prediction of English profi ciency progress. Musashino English and American Literature, 18, (pp. 165-185). Tokyo: Musashino Women’s University.

Wilson, K. (1993). TOEIC research summaries, no. 1: Relating TOEIC scores to oral profi ciency interview ratings. Princeton: ETS. Retrieved March 22, 2014, from http://www.ets.org/Media/Research/pdf/TOEIC-RS-01.pdf

Wood, J. (2010). TOEIC materials and preparation questions: Interview with an ETS representative. The Language Teacher, Vol. 34, No. 6, Nov./Dec. (pp. 41-45). Tokyo: JALT. Retrieved March 22, 2014, from http://jalt-publications.org/tlt/articles/109-readers-forum-toeic-materials-and-preparation-questions-interview-ets-representativ

Woodford, P. (1982). TOEIC research summaries: An introduction to TOEIC: The initial validity study. Princeton: ETS. Retrieved March 22, 2014, from http://www1.ets.org/Media/Research/pdf/TOEIC-RS-00.pdf

30



is one additional english language class enough? · the toeic should be used for and what sorts of...

Documents