ritsuko ohta keiko kawaguchi tae ito - iibc-global.org · a study of the relationship between toeic...

A Study of the Relationship between TOEIC Scores and Functional Job Performance:

Self-assessment of Foreign Language Proficiency

Tae Ito

Keiko Kawaguchi

Ritsuko Ohta

Table of Contents

1. Introduction ........................................................................................................................................ 1

2. Review of the literature ...................................................................................................................... 2

2.1 Validation studies on self-assessment ......................................................................................... 2

2.2 Influential factors on self-assessment ......................................................................................... 3

2.3 Related studies on Japanese psychological tendencies in self-evaluation .................................. 6

2.4 Studies on score interpretation of the Test of English for International Communication (TOEIC) .. 7

3. Method ............................................................................................................................................... 8

3.1 Participants ................................................................................................................................. 8

3.2 Materials ..................................................................................................................................... 8

3.3 Procedures ................................................................................................................................ 10

4. Results ............................................................................................................................................... 11

4.1 Means and SDs for TOEIC scores and self-assessment ratings ................................................ 11

4.2 Self-assessment results by skills and by situations ................................................................... 12

4.3 The correlation coefficients between self-assessment and TOEIC scores ................................ 13

5. Discussion ........................................................................................................................................ 15

6. Development of a checklist .............................................................................................................. 18

6.1 3-point scale ............................................................................................................................. 18

6.2 Implications .............................................................................................................................. 19

7. Limitations ....................................................................................................................................... 19

8. Conclusion ........................................................................................................................................ 21

References ................................................................................................................................................ 22

Appendix 1: Self-assessment questionnaire (English) .......................................................................... 24

Appendix 2: Self-assessment questionnaire (Japanese) ........................................................................ 30

Appendix 3: Questionnaire items .......................................................................................................... 35

Appendix 4: Means, SDs and correlations between TOEIC scores and

self-assessment scores ...................................................................................................... 36

Appendix 5: List of functional job activities and proficiency levels ..................................................... 39

1

1. Introduction

First introduced in 1979, the TOEIC® (the Test of English for International Communication) has gained

in popularity as a standard assessment of English language proficiency in Japan. In fact, in the fiscal year

2002 alone, over 1.3 million people took either the TOEIC through the Secure Program (SP) or the

Institutional Program (IP) through their own companies and schools (The Institute for International

Business Communication [IIBC], 2003). In particular, facing forces of globalization and realizing English

is one of the key factors to maintain competitiveness in global markets, an increasing number of

organizations have come to use the test. According to the research conducted by IIBC in 2001, some of

these organizations have used the test scores in evaluating the effectiveness of their language training

programs and assessing the English proficiency of their prospective employees, whereas others have

adopted the test to set score benchmarks as a criterion in making personnel decisions such as overseas

assignments and promotion. For instance, IBM Japan now requires a TOEIC score of 600 for a

managerial position or a short-term overseas business trip and 730 for a director or a mid-to long-term

overseas business trip. Moreover, Fujitsu has been pushing all new employees to achieve 600 on the test

by their second year (IIBC, 2000).

Despite such a surge of interest in the TOEIC, there has always been the issue of interpretability of its

scores. That is, what does the score mean? How should score recipients interpret their scores? Or are

companies able to predict what an employee with a score of 600 is capable of doing with English in their

working environment?

The TOEIC was developed by Educational Testing Service (ETS). It is a norm-referenced test that aims

to measure the English language proficiency of people in a workplace context. The test consists of 200

multiple-choice questions divided into two sections: 100 listening comprehension questions and 100

reading comprehension questions. The test not only assesses listening and reading skills but also provides

an indirect measure of speaking and writing skills. The examinees are supplied with a total score ranging

from 10 to 990 along with two scaled sub-scores for each section of the test ranging from 5 to 495 (The

Chauncey Group International, 1999, 2002).

2

An unavoidable problem with norm-referenced tests of language proficiency is that they provide very

little information about the examinee’s actual linguistic abilities in specific settings (Wilson, 1989).

Percentile ranks on norm-referenced tests just show us the relative standing of each examinee with respect

to the performance of others. In other words, if an individual’s score of 600 gives him/her a percentile

rank of 60% on the TOEIC, the results simply tell us that he/she did better than 60% of the examinees who

took the same test. Inevitably, in order to get a deeper understanding of scores, many organizations have

had to establish their own standards to evaluate the English language capabilities of employees, while

individuals have ascertained their English proficiency levels by meeting the target score set by their

companies or referring to the Proficiency Scale provided by IIBC.

However, neither of the standards gives concrete ideas of what tasks a person in a certain score band is

able to do with English in the workplace. Considering the uses to which scores are put, as mentioned

above, we have seen an urgent need for substantiating the correlation between TOEIC scores and

work-related English abilities, yet to our surprise, very few studies have been implemented investigating

TOEIC score interpretations. Accordingly, it was our primary goal to develop a checklist by which the

test-users can interpret the TOEIC scores in more practical ways. For this purpose, a self-assessment

questionnaire was used to explore the relationship between the self-estimated English-language proficiency

and TOEIC scores. Then, based on the results, the checklist was developed to indicate how well a person

with a certain TOEIC score is likely to perform in English and what tasks they can carry out within a

professional framework.

2. Review of the literature

2.1 Validation studies on self-assessment

In the field of second/foreign language learning and testing, the notion of the use of self-assessment first

emerged in a movement that put emphasis on learner autonomy in the late 70’s (Oscarson, 1989, 1997).

The researchers, then, started to discuss the rationale and methods for the use of self-assessment as an

instrument to assess one’s own second/foreign language proficiency. For the last two decades the research

3

focus has been mainly on the development of methods and materials and on determining the validity of the

approach. Since the current study relies on the individual’s ability to self-assess, validation studies on

self-assessment and relevant research findings will be briefly reviewed.

Validity of self-assessment has been investigated in terms of concurrent validity statistics obtained by

means of correlating self-estimated ability data with more objective measures of the same abilities.

Blanche and Merino (1989) surveyed 16 self-assessment studies in foreign language skills conducted in the

late 70’s and 80’s and reported that “the accuracy of most students’ self-estimates often varies depending on

the linguistic skills and materials involved in the evaluations, but these estimates are generally good or very

good’” (p.315). This Blanche and Merino’s conclusion was meta-analytically confirmed by Ross (1998).

One notable example of the use of self-assessment in second language education is the use of

self-assessment as a placement test at the University of Ottawa, which originated from a sequence of

experiments on validity of self-assessment measure by LeBlanc and Painchaud (1985). In the experiments

new students in undergraduate second language programs were asked to rate their receptive skills in their

target language using a self-assessment questionnaire prior to taking standardized proficiency tests. High

correlations (.80, .82) were obtained between the two kinds of measure after modifying the questionnaire

content so that it was relevant to the students’ experience.

Another validation study on self-assessment of communicative language ability was conducted on

non-native speakers of English by Bachman and Palmer (1989). Employing a multitrait-multimethod

design and confirmatory factor analysis, they investigated three language traits (grammatical, pragmatic,

and sociolinguistic competence). The results indicate that ‘difficulty’ questions inquiring about subjects’

perceived difficulty are more effective than ‘ability’ questions (so-called can-do questions). They

concluded that “self-ratings can be reliable and valid measures of communicative language abilities” (p.22).

2.2 Influential factors on self-assessment

In the studies investigating the validity of self-assessment (see Oscarson, 1997, for other validation

studies), various factors affecting the accuracy of self-assessment have been pointed out. Given the

4

evidence that self-assessment represents a promising approach, researchers have started to pay more

attention to the factors that may affect accuracy. Such factors may be instrumental or personal.

Heilenman (1990) investigated response effects, “tendencies to respond to factors other than question

content” (p.175). Using a self-assessment questionnaire consisting of 65 ‘can-do’ (positively-worded) and

‘difficulty’ (negatively-worded) questions, she found that students seemed to find it harder to disagree with

‘difficulty’ items than to agree with ‘can-do’ items. In addition, acquiescence or yea-saying and

overestimation were observed more in the less experienced learners. She provided insightful reasoning

for such response effects from a perspective of information processing: Completing a self-assessment

questionnaire is a complex cognitive process where the respondents are required not only to process the

linguistic information on the questionnaire but also to activate the appropriate schemata and search and

retrieve information relevant to produce appropriate responses. Hence, the whole process imposes

“significant demands on learners” (p.190), especially on inexperienced learners and may induce response

effects. Provided the evidence that response effects are more or less present in self-report measures, she

discussed ways to minimize the possibility of response error.

MacIntyre, Noels and Clement (1997) examined bias in self-ratings of second language proficiency

created by language anxiety. They found that this variable was negatively correlated with both perceived

and actual proficiency in speaking, comprehension and writing tasks except reading tasks. Anxious

students tended to underestimate their ability while more relaxed students overestimated. They suggest

that anxious students, afraid of failing in performing potentially ego-threatening tasks, underestimate their

proficiency to protect their self-esteem.

Ross (1998) examined how the subjects’ direct experience of some language tasks affected self-ratings of

their ability regarding the same tasks. Given the fact that there is considerable variation in the correlations

between self-assessment and criterion variables in the past studies, he assumed that, apart from the

instrumental factors such as ambiguous descriptions on the self-rating items, the source of variation is the

learners’ experience with the language they bring at the time of self-rating. The assumption was

supported by the fact that the correlations with reading skills were the highest among those with the four

5

skills in the past studies. In analysis of the result, Ross suggested that language learners can assess their

own reading ability more accurately since “in many foreign language contexts, exposure to the written

word predates extensive opportunities for listening and speaking practice” (p.6). In his validation study

for a language training program at a Japanese company, he examined how students self-rated two different

types of self-assessment items: one dealing with exactly the same content they learned in class and the

other asking about the same content but presented in a different format from the one experienced in class.

He obtained evidence that “learners will be more accurate in the self-assessment process if the criterion

variable is one that exemplifies achievement of functional (‘can do’) skills on the self-assessment battery”

(p.16).

The influencing factors presented above may be intricately related to each other; it is possible that

multiple factors simultaneously operate in the process of self-rating. For instance, language anxiety may

be related to language experience and to the nature of language tasks. If a learner has enough experience

with a certain language task, he/she may feel secured in self-evaluation even though the task is a type of

ego-threatening one, such as speaking in public. Up to date, however, there have been no empirical

studies examining combined factors operating in the process of self-assessment.

The following is a summary of findings.

1. Self-assessment of one’s own second/foreign language proficiency can be highly reliable under

some conditions. The use of self-assessment is, therefore, a promising approach depending on its

purpose.

2. The accuracy of self-assessment can be influenced by various factors, and such factors may induce

overestimation and/or underestimation of proficiency. Such factors may be instrumental or

personal.

3. When descriptions on self-assessment items are ambiguous, or when self-rating scales are not

clearly defined, it is likely that subjects interpret them differently and the results of their self-ratings

become unreliable.

4. Learners are more aware of their difficulties than their abilities. Thus, difficulty questions (or

6

negatively-worded questions) might lead to accurate self-assessment but at the same time such

questions may produce response error because of their complexity.

5. Language anxiety can create bias in the accuracy of individuals’ self-perceptions of competence in

second language.

6. Experiential factors may affect the accuracy of self-assessment.

1) Self-assessments are more accurate when questions are relevant to learners’ experience.

2) Less experienced learners’ self-assessments can be less accurate than those of more experienced

learners because the latter have more experience with the language on which they can base their

judgment of their own performance.

3) Decoding skills (reading, listening) tend to obtain higher correlations with criterion variables

than encoding skills (speaking, writing) probably because learners can more accurately assess

language skills for which they have enough experience in instruction and/or in their real-life

language contact.

2.3 Related studies on Japanese psychological tendencies in self-evaluation

Another issue that needs to be brought to attention in the research of self-assessment of second/foreign

language proficiency is different attitudes or psychological tendencies and varying response effects due to

cultural differences in self-concept (Oscarson, 1997). For instance, in Tannenbaum, Rosenfeld, and

Breyer’s (1997) study, they pointed out the possibility that Japanese people may evaluate themselves

modestly and therefore hesitate to choose the highest rating scale ‘extremely well.’ To our knowledge

there has been no study dealing with such response effects related to cultural differences in the

self-assessment studies in the area of second language learning and testing. However, in the field of social

psychology, cultural differences in self-evaluation or self-enhancement have been extensively studied in

recent years. It has been suggested that Japanese tend to be more self-critical when encountering negative

information about themselves while north Americans tend to maintain and enhance an overall evaluation of

the self (Heine, Kitayama, & Lehman, 2001; Kitayama, Markus, Matsumoto, & Norasakkunkit, 1997).

7

Yoshida, Kojo and Kaku, (cited in Kurman, 2001) empirically showed that modest self-presentation

observed in Japanese is internalized through primary education. As far as the present study is concerned,

there might be such culturally embedded psychological tendencies in our subjects. Such cultural factors,

however, will not affect the correlations between self-assessment and criterion variables in our study since

our subjects are all Japanese.

2.4 Studies on score interpretation of the Test of English for International Communication (TOEIC)

In order to interpret TOEIC scores in terms of real-life oral proficiency, Wilson (1989) conducted a study

investigating correlations between scores on the Language Proficiency Interview (LPI) and listening

comprehension scores (TOEIC-LC), reading comprehension scores (TOEIC-R) and total scores

(TOEIC-Total). The LPI is a direct oral language proficiency interview procedure for assessing functional

ability to use English in face-to-face conversation. LPI ratings are based on a scale developed by the

Foreign Service Institute (FSI) of the U.S. Department of State. 393 data-sets consisting of TOEIC scores

and LPI ratings were collected initially in Japan (285), then in France (56), Mexico (42) and Saudi Arabia

(10). TOEIC scores (LC, R, Total) were generally strongly correlated with LPI ratings with the highest

correlations obtained between LC and LPI (typically in the mid-.70’s) across nationally and linguistically

diverse samples. He suggests that “the information conveyed by TOEIC scores and LPI ratings further

needs to be linked (formally-statistically and/or clinically-intuitively) to defined criteria of the ability of

ESL users/learners to use English in the workplace” (p.62). In the sub-study, he considered the possibility

of using self-ratings of speaking proficiency as a research surrogate for the LPI scale and obtained

empirical evidence of the validity of self-ratings.

The study of Tannenbaum et al. on which TOEIC Can-Do Guide (1998) is based further interprets

TOEIC scores in terms of real-world communicative competence in daily life and basic work-related

situations. Employing a self-assessment questionnaire consisting of 75 can-do statements, they asked

8,601 Japanese company employees to rate their performance. The correlations between the self-ratings

(on four skills and on interactive communication) and the three TOEIC scores ranged from .58 to .69,

8

displaying relatively strong correlations between self-ratings of proficiency and TOEIC scores. They also

asked the subjects’ instructors to rate the subjects’ performance and obtained a moderately high to high

level of agreement between the self-ratings and the third-party ratings. Finally, a practical behavioral

checklist for TOEIC users was developed. With the list, TOEIC score holders can know what they can

probably do using English in daily or basic work-related situations. In fact, the purpose and the overall

study design are quite similar to those of the present study. The difference is that the present study has

attempted to link TOEIC scores to a range of job-related tasks (basic to specialized) within various work

situations, while Tannenbaum et al. matched TOEIC scores to descriptions of activities reflecting more

generic aspects of work.

3. Method

3.1 Participants

The subjects in this study were 8,386 Japanese company employees. The majority of the participants

were males (92.7%). Ninety-one percent of the subjects were aged between 22 and 49, with the mean age

34.8 (SD 8.67). Many of the subjects (68.7%) came from the electrical field, while 66.3% were affiliated

with a company with 10,000 or more employees. 74% of the respondents had more than five years of

work experience. About half of the participants (48.8%) had done work that entailed the use of English,

but the majority had never lived abroad for work (89.1%) or had never made an overseas business trip

(59.7%). 7,538 provided information on whether they had experience living abroad, and the majority

(91.4%) indicated no experience living in a foreign country. The data of their educational background

were provided by only 3,448 subjects; 53.7 % of them were college graduates, 26.3 % received a graduate

education, and 8.8 % were high school graduates.

3.2 Materials

A self-assessment questionnaire with 65 can-do statements was designed for this study. In the process

of developing the questionnaire, in order to make it an accurate measurement of people’s English-language

9

proficiency in work-related contexts, we followed the procedure described below. First, we

content-analyzed various reference books dealing with so called ‘business English.’ We also reviewed the

can-do checklist constructed by Tannenbaum et al. (1997) but focused on the statements describing “basic

job activities” rather than “daily life activities” to attain our goal of developing a more business-oriented

questionnaire. Then, we made a list of Japanese statements of basic or specialized job activities relevant

to the respondents of the questionnaire. Next, unlike the existing can-do checklist where 75 statements

were arranged in five skill domains, our listed tasks were classified into seven situational domains (A.

meeting a guest, B. doing a routine task, C. making/receiving a call, D. placing orders/making

payments/making complaints, E. business talks/presentations, F. attending a meeting/conference, and G.

visiting/going on a business trip). Furthermore, each of the statements was followed by a box in which

the respondents would put a check in case they have done or are doing the activity given. Concurrent with

the list, a 5-point scoring scale ranging from 1 (not at all ), 2 (with a great deal of difficulty), 3 (with some

difficulty), 4 (with little difficulty) to 5 (easily) was developed to find out up to what point the subjects

perceive they can accomplish the task described in each statement. Next, we asked several people

working in various fields to review both the list and the rating scale and make any modifications necessary.

Taking the provided feedback into consideration, we reworded vague statements, added any missing job

activities and deleted some irrelevant working settings. For instance, in situation E “In business

talks/business presentations,” the phrase “The number of participants is 1-3” was added. This was based

on a suggestion that the number of the participants present might become an influential factor in estimating

the difficulty of the task. We also added the expression “with no use or only infrequent use of a

dictionary” to some statements to make them specific. The revised questionnaire was pilot-tested with 52

company employees whose TOEIC scores were mainly around 600 (550-650). The score range was

chosen to make the checklist meaningful to the users, mainly companies, many of which use scores around

600 as a criterion in making significant employment decisions. Those 52 people were asked to complete

the questionnaire and later to give any suggestions on it. Based on the analysis of their answers and

comments, we came up with 65 can-do statements, each of which described job activities necessary for

10

occupational or social aspects of work as well as dealing with one of the following five skills (listening,

speaking, reading, writing, and interactive communication). Tasks requiring two or more skills, for

instance, listening to someone and giving a response, were categorized in interactive communication.

Although the final version of the self-assessment questionnaire was constructed both in English and in

Japanese (see Appendix 1&2), the Japanese version was administered to the subjects, considering

Oscarson’s suggestion (1997) that self-assessments using the subjects’ native language will probably lead to

more accurate results. Finally, the coefficient alpha reliability of the instrument was checked and found

high (alpha = .993).

3.3 Procedures

After the IP tests in 2001-2002, the questionnaire (the Japanese version), proctored by the IIBC, was

given to 8,985 employees from 25 different companies that had consented to provide their data. However,

in the end the data of 8,386 were statistically analyzed due to the following two reasons: 1) The data of

subjects who did not complete all the items were eliminated. 2) The data with extreme cases were also

excluded, for example, a person in the lowest score band who marked 5 for all the items or a subject with

higher than 900 who gave 1 or 2 for all tasks. The questionnaire was basically composed of two parts.

In the first section, the subjects were asked to read the directions which would assure them that the result of

the questionnaire would be used only for research purposes and that it would not affect their TOEIC scores

and their evaluation of English proficiency. We believed that this step would mitigate their anxiety and

maximize their commitment, which would lead us to obtain more reliable outcomes. The subjects were

also asked to provide some personal information such as gender, age, educational background, job

experiences, use of English, and so on (see Appendix 3). In the following section, the subjects were asked

to read each of the 65 can-do statements and estimate their English ability on a 5-point scale as well as

indicate whether they have ever experienced doing the task described, by putting a check in a given box.

11

4. Results

4.1 Means and SDs for TOEIC scores and self-assessment ratings

SPSS-X Ver.11. was used to analyze the data. Table 1 shows descriptive statistics for the TOEIC test

results. The score range which included the biggest number of subjects was 300-399 with 1,728

examinees or 21.2% of the total number (Figure 1). In fact, 80.4% of the examinees were in the range

from 10 (minimum score) to 600. Table 2 shows the descriptive statistics for the results of the

self-assessment questionnaire. The questionnaire has 65 questions with a scale from 1 to 5. Therefore,

the minimum score of the questionnaire is 65, while the maximum is 325. The mean score was 127.2, or

1.96 on the 1-5 scale. The number of people who marked below 130, 2 in 1-5 scale, was 60.6% while

87.1% of the subjects marked below 185, 3 on the 1-5 scale (Figure 2).

Table 1: Descriptive Statistics for TOEIC Test Scores� N Min. Max. Mean Median SD Variance

TOTAL 8386 10 965 440.2 415 178.1 31722.2 LISTENING 8386 0 495 246.5 230 87.6 7669.2 READING 8386 5 480 193.7 185 97.4 9477.4

Table 2: Descriptive Statistics for Self-Assessment� N Min. Max. Mean Median SD Variance

Self-Assessment 8386 65 325 127.2 113 56.9 3240.2

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

Figure1: Distribution of TOEIC Scores Figure2: Distribution of Self-Assessment Scores

12

4.2 Self-assessment results by skills and by situations

Table 3 indicates self-assessment results by skills and by situations. The number of items for each skill

and situation is given in the table. The means for decoding skills, Listening and Reading, are higher than

those for encoding skills, Speaking and Writing. Activities involving interactive communication received

the lowest means. These results were consistent with several earlier findings (Ross, 1998; Tannenbaum et

al., 1997). The means for four situations, “A. Meeting a guest,” “B. Doing a routine work,” “C.

Making/Receiving a call” and “G. Going on a business trip” were relatively higher compared with the

means for the other three situations, “D. Placing orders, making payments, making complaints,” “E. In

business talks/business presentations” and “F. Attending a meeting or conference related to my job/my field

of specialty.” “G. Going on a business trip” marked the highest mean, 2.12, while “D. Placing orders,

making payments, making complaints” marked the lowest, 1.31.

Table 3: Number of Items, Means, and Standard Deviations (Self-Assessment)Scales # of Items Means SD

Listening 10 2.15 .94 Speaking 18 1.91 .90 Reading 12 2.16 .92 Writing 12 1.79 .93 Interactive Communication 13 1.70 .89 A (Meeting a guest) 9 2.01 .91 B (Doing a routine task) 20 2.03 .90 C (Making/Receiving a call) 8 1.98 .98 D (Placing orders, making payments, making complaints) 4 1.31 .95 E (In business talks/presentations) 6 1.71 .90 F (Attending a meeting/ conference) 7 1.74 .90 G (Going on a business trip) 11 2.12 .96

The graphs in Figures 3 & 4 show the means for skills and situations based in 100 score bands. The

means for decoding skills are higher than those for encoding skills in all score bands. The means for

interactive skills are lowest throughout all of the score bands. The differences among decoding, encoding

and interactive skills are relatively clear in the middle score bands. The differences become smaller in

low (below 295) and high (above 800) score bands. On the other hand, in terms of the means of situation

13

groups, all situations except “D” are close. In general, this graph shows when a test score becomes higher,

the means of a perceived job performance become greater.

The means for each item are in Appendix 4. The highest means are 2.86 (A-2) and 2.53 (B-8), while

the lowest means are 1.50 (E-6) and 1.55 (C-6).

��

��

��

��

��

��

��

��

��

��

��

�

�

�

�

�

�

��

��

��

��

��

Figure 3: Means for Self-Assessment (Skills) Figure 4: Means for Self-Assessment (Situations)

4.3 The correlation coefficients between self-assessment and TOEIC scores

Correlations among the three generated TOEIC scores, five skills, seven situations are presented in Table

4. The self-assessment total scores(x) were transformed by logarithm to the base 10 (x’=log10x) for

statistical analysis in order to adjust its skewed distribution.

The correlations ranging from .62 to .71 indicate that the self-assessment ratings were fairly highly

correlated with the TOEIC scores. Self-assessment Reading was the highest among the correlations

between TOEIC total and self-assessment scores, while situation D and E were the lowest. The

correlations between TOEIC Listening and self-assessment Listening and Speaking were found quite

significant, and similarly TOEIC Reading was highly correlated with self-assessment Reading. All

correlations were significant at the .01 level. These results indicate the positive relationship between

TOEIC scores and self-assessment ratings and provide positive evidence of the validity of self-assessment

�

��

��

��

��

��

��

��

��

��

��

��

��

�

�

�

�

�

�

�

�

�

�

�

�

�

14

approach to measure one’s proficiency of English as a foreign language.

Table 4: Correlations between TOEIC Scores and Self-Assessment �TOEIC Test Total TOEIC L TOEIC R

Self-Assessment Total .71 .69 .68Skills Listening .69 .68 .65 Speaking .70 .68 .66 Reading .71 .67 .70 Writing .67 .64 .66� Interactive communication .68 .66 .64Situations A (Meeting a guest) .68 .67 .64 B (Doing a routine task) .70 .66 .69 C (Making/Receiving a call) .67 .66 .63 D (Placing orders, making payments, making complaints) .65 .62 .62 E (In business talks/presentations) .65 .63 .62 F (Attending a meeting/conference) .66 .64 .63� G (Going on a business trip) .66 .65 .63All results are significant at the .01 level.

Table 5: Correlation Matrix by Skills (Self-Assessment)

SAL SAS SAR SAW SAI

Self-Assessment

Listening (SAL) 1

Speaking (SAS) .93 1

Reading (SAR) .91 .89 1

Writing (SAW) .87 .91 .91 1

Interactive Communication (SAI) .90 .96 .86 .91 1

All results are significant at the .01 level.

Table 6: Correlation Matrix by Situations (Self-Assessment) � � A B C D E F G Self-Assessment A (Meeting a guest) 1 B (Doing a routine task) .87 1 C (Making/Receiving a call) .90 .87 1 D (Placing orders, making payments, making complaints) .83 .88 .87 1 E (In business talks/presentations) .84 .87 .85 .88 1 F (Attending a meeting/conference) .85 .88 .86 .87 .93 1 � G (Going on a business trip) .85 .85 .87 .81 .80 .83 1All results are significant at the .01 level.

15

As presented in Tables 5 & 6, the intercorrelations among the self-assessment (ranging from .80 to .96)

were greater than the correlations between the TOEIC scores and the self-assessment domains (ranging

from .62 to .71). Figure 5 illustrates the correlations between the TOEIC total scores and self-assessment

total scores. There is a fairly high correlation between the TOEIC total scores and the self-assessment

(r=.71, R2 =.51). Thus, about 50% of the variability in self-assessment is explained by the TOEIC total.

��

��

��

��

��

��

�

Figure 5: The Correlations between the TOEIC Total Scores and Self-Assessment Total Scores

5. Discussion

On the whole, the data presented above provide evidence for a fairly high correlation between the

TOEIC scores and the self-assessment scores. The total scores on the TOEIC were highly correlated with

those on the self-assessment scores (.71). Correlations were also significant between each sub-test of the

TOEIC and each skill/situation of the self-assessment questionnaire. Despite the fact that these findings

are consistent with those reported in the related study (Tannenbaum et al., 1997), two points should be

y^=.228x+26.912

Self-

asse

ssm

ent S

core

s

TOEIC Total Scores

16

noted. First, the mean TOEIC Total score in the present study was 440 (SD 178), while Tannenbaum et

al's was 462 (SD 175). In addition, the mean self-ratings for the five skill domains were not as high (1.70

to 2.16 in our study; 2.34 to 2.87 in Tannenbaum et al's study). One possible explanation for this lower

self-assessment scores is that our self-assessment questionnaire included more specialized tasks, which

even advanced level subjects might have felt it difficult to perform. Second, on average, higher

correlations were obtained between the TOEIC scores and the self-assessments of the five skill areas (.64

to .71 in our study; .58 to .69 in Tannenbaum et al's study). This seems to prove that our self-assessment

items covering a wider range of task difficulty were well tailored to the subjects of all proficiency levels

and so could measure their specific English-language abilities accurately. In other words, these results,

confirming earlier research findings on the reliability of learners’ self-estimates, suggest that this

self-assessment questionnaire can be considered a reliable tool to measure people’s second language

abilities in a business context.

Throughout the data, a consistent pattern was observed in the self-assessments. That is, reading skills

were rated highest (mean=2.16) and correlated most strongly with TOEIC total scores (.71). This was,

however, not overly surprising for the following two reasons.

First, the high ratings and high correlations for reading skills could probably be attributed to the fact that

the majority of our subjects learned English in the Japanese educational system where reading

comprehension has been most emphasized. In his summary of self-assessment studies, Ross (1998) stated

that “a plausible reason for this slight advantage for reading [high correlations between self-assessments

and reading skills] may relate to the extent of experience learners have with second language reading” (p.6).

Although his study did not deal with Japanese subjects only, the result seems to point to the possibility that

the familiarity our subjects had with reading might have led to the high ratings and high correlations for

reading skills in our study.

The second reason for this is reading skills may not threaten one’s identity or ego as much as the other

skills may. According to MacIntyre et al. (1997), speaking and writing involve communication with

others, and this publicity might cause anxiety. On the other hand, reading is “the most private” in that

17

without being noticed, one can make many attempts to comprehend a passage given. This “ability to save

face” might have raised the self-ratings of reading skills in our study (p.280).

Another emerging pattern is that the interactive communication skills got the lowest scores and lowest

correlations within skill domains. This again seems plausible since the activities subsumed in the

“interactive” domain require a combination of two skills; for instance, in communicating with another

person, while trying to interpret what is being said in English (listening), a person has to plan his/her

message relevant to the interlocutor’s needs and actually produce it in English under the pressure of time

(speaking). The complexity of such a process probably lowered the self-estimates for interactive skills.

In addition, that this interactive communication entails interpersonal relations might have made it the most

threatening among the skills.

Further analysis of the data revealed that among seven situations, situation G “Going on a business trip”

was rated highest, whereas situation D “Placing orders, making payments, making complaints” was

assessed lowest. The reason seems to stem from how familiar the tasks given were with the subjects.

For example, most of the activities in situation G, such as “to read a train timetable, a restaurant menu or to

understand questions at the immigration,” are things that one generally does when one goes and stays

abroad for any purpose. That means more subjects are likely to have performed those tasks before and so

they might have found them rather easy to do. In contrast, situation D encompasses the tasks involving

specialized knowledge or skills that only people in certain positions are required to have. (e.g. D-1 “to

telephone a company to ask if a certain item is in stock, to ask for an estimate to be sent, or to place an

order for a certain item.” D-2 “to telephone a company to make a complaint about a wrong product

delivered, a broken product, or nonpayment.”) This unfamiliarity with the activities might have lowered

the self-assessments given by the subjects. The difference in the self-ratings might also be explained by

the fact that all the tasks in situation G were identified as decoding or receptive skills and situation D

comprised the activities dealing with encoding or productive skills.

18

6. Development of a checklist

6.1 3-point scale

This study aimed at finding out what functions a person with a TOEIC score could accomplish in English

in business. We found a substantial relationship between TOEIC test scores and the scores of functional

job activities in the self-assessment questionnaire. Given the results, we constructed a checklist that tied

TOEIC total scores to specific activities performed in English; each of the 65 job-related tasks in the

questionnaire was graded in terms of difficulty based on the average ratings of the subjects in a certain

score range.

In order to make our checklist a practical guide for the users, we converted the 5-point scale of the

self-assessment into 3-point scale following Heilenman’s (1990) procedure. When rescaled, the original 1

and 2 became 1, 3 remained as 3, 4 and 5 became 5. Then, the average of each self-assessment item was

interpreted to be 1 = can’t do, 2 = can do with some difficulty, and 3 = can do. The score range from 1 to

2.33 was marked as 1, 2.34 to 3.66 was 2 and from 3.67 to 5 was 3. The checklist is on Appendix 5. We

believe that this simplified checklist would help organizations more easily differentiate the proficient

employees from those who lack the skills necessary for specific jobs.

Now, let us compare the results of two score bands, 500-595 and 600-695, in the 3-point scale checklist

(Appendix 5). It shows that people in the score band from 500 to 595 perceive they may be able to do 13

of the 65 functions. In contrast, people in 600 to 695 can be expected to do 35 functional activities in

various situations. This means people with the score between 600 and 695 could perform approximately

three times more activities than the people in the 500-595 score band. In fact, 35 items are 53.8 % of the

total tasks on the list. Therefore, based on the list, it can be said that a person with over 600 on the TOEIC

can be expected to do a reasonable amount of work in English; on the other hand, the list also shows that

people holding the scores from 500 to 595 may need to be trained in some productive skills described in

situations A, C, and D to carry out jobs in English. In this way, the checklist indicates how well a person

in a certain score range is likely to perform each of the 65 job-related activities in English.

19

6.2 Implications

Having explicated the design of the checklist, we will now present how it can enhance the interpretation

of TOEIC scores. We believe that the checklist linking concrete tasks to TOEIC score bands satisfies the

needs of organizations that use the TOEIC for various reasons. This activity list, first of all, allows

companies to make more accurate decisions regarding employment, promotion, and selection. For

instance, if an applicant for the position of receptionist with a foreign affiliated firm has a total TOEIC

score of 700, he/she might be considered for the position. By using the checklist, the company can expect

the candidate with score 700 to be able to perform, with little job-training, for instance, the tasks described

in situations A-1 & A-2 and C-1 through C-5, which any receptionist is required to do (e.g. A-1 “to

understand a person’s name given and the reason of his/her visit and give the message to a person in

charge.”) More informative interpretations using this list may also aid organizations in deciding where to

set score standards for certain positions. Furthermore, the checklist might give a clear sense of what

job-related activities should be targeted in their language training programs to bring the trainees up to the

desired levels of English language proficiency for specific jobs.

Although our primary concern was to meet the demands of organizations, we believe that educational

institutions can benefit from this checklist as well. The current trend for more companies to use the

TOEIC as an important management tool has pushed an increasing number of colleges to incorporate

TOEIC preparation courses into their curriculum. With the help of this checklist, they can confirm if their

TOEIC curriculum really prepares students to have English abilities required in real life work settings.

7. Limitations

The limitations of the study must be outlined here as they may bear on the results given above. First,

some personal variables could not be controlled. Among them were gender and occupation. As already

shown, an overwhelming percentage of the subjects were male subjects (92.7 %), while 68.7 % were in the

electrical field. Although the 65 statements were designed with an effort to reflect various aspects of

20

work, caution must be exercised in applying the checklist to different groups of test-users.

Secondly, in this study, a self-assessment questionnaire was employed to measure the subjects’ specific

English abilities, which naturally raises a question of its reliability. As presented before, the research

conducted by Heilenman (1990) revealed that lower level students tended to overestimate their abilities and

advanced students, on the other hand, underestimated themselves when asked to assess their own language

proficiency. Although some extreme cases were deleted from the analysis as described in “Procedure,” in

fact, we still found that a person with 925 on the TOEIC marked 2 for tasks expected to be accomplished

easily, while a person with the score of 350 marked 4 and 5 for all the items.

On the other hand, the overall results might be biased toward underestimation due to two Japanese traits:

self-criticism and modesty. In a culture that places great emphasis on social harmony, people are likely to

develop a self-critical bias, the tendency to identify their own problem areas to meet the standards of their

roles in a given context (Heine et al., 2001), and a modesty bias, the tendency to understate their

capabilities in public (Cialdini, Wosinka, Dabul, Whestone-Dion, & Heszen, cited in Kurman, 2001). In

fact, much research has shown evidence of Japanese self-criticism in questionnaire responses (Heine et al.,

2001; Kitayama et al., 1997), while Heine, Takata and Lehman (2000) suggested that modest

self-presentation in public might make it difficult for Japanese to “[express] their private thoughts, even

anonymously in questionnaires” (p.72). Thus, we cannot preclude the possibility that many of the

subjects in this study might be able to perform better than what they have claimed in the questionnaire.

Another limitation was that the validity of some items was questionable. To establish the content

validity of this self-assessment questionnaire, each of the 65 can-do statements should measure whatever it

is supposed to measure and nothing else. However, it might be argued that for instance, A-8 “to explain

the meanings of some Japanese expressions or describe some Japanese customs/culture” is not valid since it

may assess not only language proficiency to perform the task in speech but also one’s knowledge of

Japanese customs and culture. Like this case, the items that did not get scale 3 even in the highest score

band need to be re-examined.

To minimize, if not eliminate, the effects of these problems inherent in a self-assessment study,

21

Heilenman suggests that the purpose of the research be clearly explained to the participants and that the

statements in the instrument include descriptions of concrete linguistic situations familiar with the subjects.

Although the self-assessment questionnaire was developed with those suggestions in mind, an additional

objective measure of English-language proficiency and protocol data collected from the subjects while

actually answering the questionnaire would have helped increase its reliability and validity.

8. Conclusion

Despite the limitations described above, the present study should be deemed exploratory in that it has

facilitated the interpretation of TOEIC scores by specifying work activities performed in English by people

in a given score range. In this study, first, we sought to explore the relationship between TOEIC scores

and self-assessment ratings. The result indicated that the TOEIC total score was highly correlated with

the self-assessment total (.71) as well as with self-assessment scores for the five skills and seven situations.

It was also found that correlations between each sub-test of the TOEIC and each part of the self-ratings

were, for the most part, robust. Having established a correlation between the two methods of

measurement, the second phase of our study aimed to design a checklist that would permit the users to

interpret the TOEIC scores in more meaningful ways. We succeeded in developing a checklist of 65

functional tasks graded according to perceived self-difficulty at different score levels. It is hoped that the

activity list will be a clear indicator or predictor of what kinds of job-related activities a person with a

certain TOEIC score can be expected to accomplish in English.

The study reported on here is far from exhaustive on the relationship between the TOEIC and

self-assessment, and it could be extended in several directions. First, whether the score interpretations

established here are applicable to many or most TOEIC test-users remains to be investigated. A pertinent

line of inquiry might focus on a certain score band (e.g. around 600) and identify its characteristics, given

that many organizations now use score 600 as one criterion in making significant personnel decisions. It

would be interesting to take a look at the effects of experiential factors on self-assessment scores using the

data collected from the present study. A great deal of work is ahead of us in this unexploited area.

22

References

Bachman, L.F., & Palmer, A.S. (1989). The construct validation of self-ratings of communicative language

ability. Language Testing, 6, 14-29.

Blanche, P., & Merino, B.J. (1989). Self-assessment of foreign-language skills: Implications for teachers

and researchers. Language Learning, 39, 313-340.

Chauncey Group International. (1998). TOEIC can-do guide: Linking TOEIC scores to activities performed

using English. Princeton, NJ: Author.

Chauncey Group International. (1999). The TOEIC User Guide. Retrieved December 5, 2002, from

http://www.toeic.com/2-3-1 informational.html

Chauncey Group International. (2002). TOEIC Examinee Handbook. Retrieved December 5, 2002, from

http://www.toeic.com/2-3-1 informational.html

Heilenman, L.K. (1990). Self-assessment of second language ability: The role of response effects.

Language Testing, 7, 174-201.

Heine, S., Kitayama, S., & Lehman, D. (2001). Cultural differences in self-evaluation. Journal of

Cross-Cultural Psychology, 32(4), 434-443.

Heine, S., Takata, T., & Lehman, D. (2000). Beyond self-presentation: Evidence for self-criticism among

Japanese. Personality and Social Psychology Bulletin, 26(1), 71-78.

Institute for International Business Communication (IIBC). (2000).TOEIC Seminar Report (81st). Tokyo:

Author.

Institute for International Business Communication. (2001). Dai juuikkai TOEIC katsuyo jittai hokoku

[TOEIC Practical Use Report]. Tokyo: Author.

Institute for International Business Communication. (2003). About TOEIC. Retrieved September 20, 2003,

from http://www. toeic.or.jp/about/main.html

Kitayama, S., Markus, H., Matsumoto, H., & Norasakkunkit, V. (1997). Individual and collective processes

in the construction of the self: Self-enhancement in the United States and self-criticism in Japan.

Journal of Personality and Social Psychology, 72(6), 1245-1267.

23

Kurman, J. (2001). Self-enhancement: Is it restricted to individualistic cultures? Personality and Social

Psychology Bulletin, 27(12), 1705-1716.

LeBlanc, R., & Painchaud, G. (1985). Self-assessment as a second language placement instrument. TESOL

Quarterly, 19, 673-687.

MacIntyre, P.D., Noels, K.A., & Clement, R. (1997). Biases in self-ratings of second language proficiency:

The role of language anxiety. Language Learning, 47, 265-287.

Oscarson, M. (1989). Self-assessment of language proficiency: Rational and applications. Language Testing,

6, 1-13.

Oscarson, M. (1997). Self-assessment of foreign and second language proficiency. Language Testing and

Assessment, 7, 175-187.

Ross, S. (1998). Self-assessment in second language testing: A meta-analysis of experiential factors.

Language Testing, 15, 1-20.

Tannenbaum R.J., Rosenfeld, M., & Breyer, F.J. (1997). Linking TOEIC scores to self-assessments of

English-language abilities: A study of score interpretation. Unpublished manuscript.

Wilson, K.M. (1989). Enhancing the interpretation of a norm-referenced second-language test through

criterion referencing: A research assessment of experience in the TOEIC testing context (TOEIC

Research Report No.1). Princeton, NJ: Educational Testing Service.

24

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

25

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

26

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

27

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

28

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

29

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

30

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

� � � � � �

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

� � � � � � � � � � � � �

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

31

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

� � � � � � � � � � � � � � � � � � � � �

32

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

� � � � � � �

��

��

� � � � � � � � � � �

33

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

� � � � � � � � ��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

� � � � � � � � � � � �

��

��

34

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

� � � � � � � � � � � � �

35

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

36

��

��

��

��

� ��

� ��

� ��

� ��

��

��

��

��

��

��

��

��

��

��

� ��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

� ��

��

��

� ��

��

��

��

��

��

��

� ��

��

��

��

��

��

37

��

��

��

��

��

��

��

��

��

��

��

��

��

��

� ��

� ��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

� ��

��

��

��

��

��

��

��

��

��

��

��

��

� ��

� ��

� ��

38

��

��

��

� ��

��

��

��

��

��

��

��

� ��

��

��

��

��

��

��

��

��

� ��

� ��

��

��

��

��

� ��

39

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

�

��

��

��

��

��

�

��

��

��

��

��

��

��

�

��

��

��

��

��

��

�

��

��

��

��

��

��

�

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

�

��

��

��

��

��

��

�

��

��

��

��

��

��

�

��

��

��

��

��

��

�

��

��

��

��

��

�

��

��

��

��

��

��

�

��

��

��

��

��

��

�

40

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

�

��

��

��

��

��

�

��

��

��

��

��

��

�

��

��

��

��

��

�

��

��

��

��

��

��

�

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

�

��

��

��

��

��

��

��

�

��

��

��

��

��

��

��

�

��

��

��

��

��

�

��

��

��

��

��

�

��

��

��

��

��

��

��

�

��

��

��

��

��

��

�

��

��

��

��

��

��

�

��

��

��

��

��

�

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

��

��

��

��

�

��

��

��

��

��

��.��

�

��

��

�

�

Date of Issue: September 2005

Publisher: The Institute for International Business Communication�IIBC�

2-14-2, Nagata-cho, Chiyoda-ku

Tokyo 100-0014, Japan

TEL 03-3581-5663 FAX 03-3581-9801

�

�

ritsuko ohta keiko kawaguchi tae ito - iibc-global.org · a study of the relationship between toeic...

Documents