research methods of applied linguistics and statistics
DESCRIPTION
Research Methods of Applied Linguistics and Statistics. By Qin Xiaoqing ( Huazhong University of Science and Technology ). Lecturer: 秦晓晴 Email: [email protected] Public email: [email protected] Password: 123456. An Introduction to this Course. - PowerPoint PPT PresentationTRANSCRIPT
Research Methods of Research Methods of Applied Linguistics and Applied Linguistics and
StatisticsStatistics
By Qin XiaoqingBy Qin Xiaoqing((Huazhong University of Science and Huazhong University of Science and
TechnologyTechnology))
22
Lecturer: Lecturer: 秦晓晴秦晓晴Email: Email: [email protected]
Public email: Public email: [email protected]
Password: 123456Password: 123456
33
An Introduction to this CourseAn Introduction to this Course
Definitions & classifications of research, Definitions & classifications of research, and research procedures and research procedures
The concept of variables The concept of variables
Quantitative research design Quantitative research design
Questionnaire surveyQuestionnaire survey
Item analysis,validity & reliabilityItem analysis,validity & reliability
Data exploringData exploring
44
Descriptive statisticsDescriptive statistics
Inferential statisticsInferential statistics
Writing a research proposalWriting a research proposal
Reporting a researchReporting a research
Research evaluationResearch evaluation
55
Nature of This CourseNature of This Course
To give lecturesTo give lectures
To hold discussions and To hold discussions and presentationspresentations
To do self-studyTo do self-study
To administer a test To administer a test
66
Course books and references (1)Course books and references (1)
Mackey, A., & Gass, S. M. (2005). Mackey, A., & Gass, S. M. (2005). Second Second Language Research : Methodology and Language Research : Methodology and DesignDesign. Mahwah, NJ: Lawrence Erlbaum.. Mahwah, NJ: Lawrence Erlbaum.
Brown, J. D. (2001). Brown, J. D. (2001). Using Surveys in Using Surveys in Language ProgramsLanguage Programs. Cambridge: . Cambridge: Cambridge University Press.Cambridge University Press.
Dörnyei, Z. (2003). Dörnyei, Z. (2003). Questionnaires in Questionnaires in Second Language Research: Construction, Second Language Research: Construction, Administration and ProcessingAdministration and Processing. Mahwah, . Mahwah, N.J.: Lawrence Erlbaum Associates.N.J.: Lawrence Erlbaum Associates.
77
Course books and references Course books and references (2)(2)
Hatch, E. M., & Lazaraton, A. (1991). Hatch, E. M., & Lazaraton, A. (1991). The The Research Manual: Design and Statistics for Research Manual: Design and Statistics for Applied LinguisticsApplied Linguistics. New York, NY: Newbury . New York, NY: Newbury House Publishers.House Publishers.Seliger, H. W. & E. Shohamy, 1999. Seliger, H. W. & E. Shohamy, 1999. Second Second Language Research Methods. Language Research Methods. Shanghai:Shanghai: 上海外上海外语教育出版社语教育出版社 ..刘润清,刘润清, 19991999 ,,《《外语教学中的科研方法外语教学中的科研方法》》。。北京:外语教学与研究出版社。北京:外语教学与研究出版社。秦晓晴,秦晓晴, 20032003 ,,《《外语教学研究中的定量数据外语教学研究中的定量数据分析分析》》。武汉:华中科技大学出版社。。武汉:华中科技大学出版社。文秋芳,文秋芳, 2004,2004, 《《应用语言学研究方法与论文写应用语言学研究方法与论文写作作》》。北京:外语教学与研究出版社。。北京:外语教学与研究出版社。
88
Definition of ResearchDefinition of Research
1.Dictionary definitions1.Dictionary definitions
(1) According to Longman Dictionary, (1) According to Longman Dictionary, research means advanced study of a research means advanced study of a subject, so as to learn new facts on subject, so as to learn new facts on scientific laws. scientific laws.
(2) Collin’s Dictionary says research is a (2) Collin’s Dictionary says research is a detailed study of a subject or an aspect of detailed study of a subject or an aspect of a subject.a subject.
99
2. Formal definition of research2. Formal definition of research
Research is a systematic approach to Research is a systematic approach to finding answers to questions.finding answers to questions.
question—appropriatequestion—appropriate
approach—systematicapproach—systematic
answer—validanswer—valid
1010
Purposes of researchPurposes of research
(a) increasing knowledge within the (a) increasing knowledge within the discipline of applied linguistics, and discipline of applied linguistics, and
(b) increasing knowledge within oneself to (b) increasing knowledge within oneself to understand new developments within the understand new developments within the discipline. discipline.
1111
How to increase knowledgeHow to increase knowledge
supports the supports the theoretical basis theoretical basis of the of the disciplinediscipline
providing evidence for the providing evidence for the efficacy efficacy of a of a curriculum, EFL teaching and learning, and curriculum, EFL teaching and learning, and so on.so on.
Developing Research Tools.Developing Research Tools.
For most students, the ability to understand For most students, the ability to understand research in applied linguistics may be more research in applied linguistics may be more important than making a research important than making a research contribution to the profession. contribution to the profession.
1212
Definition of theoryDefinition of theoryDictionary: A Dictionary: A theorytheory is a formal idea or set is a formal idea or set
of ideas that is intended to explain of ideas that is intended to explain something. something.
Kerlinger (1986: 9): A theory "is a set of Kerlinger (1986: 9): A theory "is a set of interrelated constructs (concepts), interrelated constructs (concepts), definitions, and propositions that present a definitions, and propositions that present a systematic view of phenomena by systematic view of phenomena by specifying relations among variables, with specifying relations among variables, with the purpose of explaining and predicting the purpose of explaining and predicting the phenomena" .the phenomena" .
1313
Types of research Types of research
Adapted from Brown Adapted from Brown 2001:12001:1
1414
Characteristics of statstical researchCharacteristics of statstical research
Systematic (systematically structured with Systematic (systematically structured with definite procedural rules)definite procedural rules)
Logical (based on a step-by-step logical Logical (based on a step-by-step logical pattern)pattern)
Tangible (based tangible, quantifiable Tangible (based tangible, quantifiable information, ie data)information, ie data)
Replicable (able to repeat the research)Replicable (able to repeat the research)
Reductive (help form patterns in the Reductive (help form patterns in the seeming confusion of facts)seeming confusion of facts)
(J. D. Brown 1988)(J. D. Brown 1988)
1515
Steps of quantitative research Steps of quantitative research
(1) Ask or form appropriate questions(1) Ask or form appropriate questions
(2) Review the literature(2) Review the literature
(3) State hypothesis(3) State hypothesis
null (Hnull (H00) alternative (H) alternative (H11): positive, ): positive,
negativenegative
(4) Select an approach and do the (4) Select an approach and do the researchresearch
(5) Analyze the data(5) Analyze the data
(6) Report results(6) Report results
1616
Sources of Research QuestionsSources of Research Questions
Step 1: Formulating a general research Step 1: Formulating a general research questionquestion
Interest and personal experienceInterest and personal experience
Journals of our fieldJournals of our field
Supervising facultySupervising faculty
Sources outside second languageSources outside second language
1717
Step 2: Narrowing down or Step 2: Narrowing down or focusing the questionfocusing the question
Importance (generate new information or Importance (generate new information or confirm old information in new ways) confirm old information in new ways)
FeasibilityFeasibility
List the possible factors comprising the List the possible factors comprising the general questiongeneral question
1818
Eg. How well do L2 learners Eg. How well do L2 learners perform speech act?perform speech act?
(Broad question)(Broad question)Which speech acts do you want to Which speech acts do you want to investigate?investigate?What types of L2 learners are you going to What types of L2 learners are you going to choose?choose?In what kinds of situations should the In what kinds of situations should the speech events be investigated?speech events be investigated?Is the research meant to support test Is the research meant to support test development? Material development? development? Material development? Theory developmentTheory development
1919
Narrowed QuestionNarrowed Question
What is the Chinese EFL learners’ ability to What is the Chinese EFL learners’ ability to recognize complaint behavior appropriate in recognize complaint behavior appropriate in an academic university setting?an academic university setting?
Total performance –recognition of one Total performance –recognition of one portion of the totalportion of the total
L2 learners – Chinese EFL learnersL2 learners – Chinese EFL learners
Speech acts – complaint (subcategory)Speech acts – complaint (subcategory)
The overall setting – university settingThe overall setting – university setting
2020
Activity 1Activity 1
Formulate a research question you are Formulate a research question you are interested ininterested in
2121
Activity 2Activity 2
Evaluate the following question:Evaluate the following question:
What are the characteristics of the good What are the characteristics of the good language learner?language learner?
2222
From theoretical definition to From theoretical definition to operational definitionoperational definition
Any terms must be operationally definedAny terms must be operationally defined
An operational definition is a clear An operational definition is a clear statement of how you judged or identified statement of how you judged or identified a term in your research a term in your research
Good operational definitions can often be Good operational definitions can often be drawn from the existing literaturedrawn from the existing literature
2323
Criterion for a good research Criterion for a good research questionquestion
The question shouldThe question should
Interest usInterest us
Promise new information or confirm old Promise new information or confirm old information in new ways (contribution)information in new ways (contribution)
Have reasonable scope (feasibility)Have reasonable scope (feasibility)
Have key terms that are clearly defined Have key terms that are clearly defined and operationalizedand operationalized
2424
Factors influencing feasibilityFactors influencing feasibility
The amount of time and cost neededThe amount of time and cost needed
Researcher’s prerequisite knowledgeResearcher’s prerequisite knowledge
Consistent definition of concepts and Consistent definition of concepts and termsterms
Access to dataAccess to data
2525
The purposes of research The purposes of research questionsquestions
DescriptiveDescriptive
ExploratoryExploratory
Explanatory Explanatory
2626
Step 3: Review the literature on the Step 3: Review the literature on the topic as completely as possibletopic as completely as possible
Which questions have been answered Which questions have been answered satisfactorily?satisfactorily?
Which seem trivial?Which seem trivial?
Which are too complex to tackle at the Which are too complex to tackle at the moment?moment?
Which procedures have been used in Which procedures have been used in answering the questions?answering the questions?
What results have been obtained?What results have been obtained?
2727
Step 4: Formulating the research Step 4: Formulating the research question or the hypothesisquestion or the hypothesis
A hypothesis is a statement of possible A hypothesis is a statement of possible outcome of research.outcome of research.
Null hypothesis and alternative Null hypothesis and alternative hypotheseshypotheses
2828
HH11 andand HH22
There is a positive relationship between There is a positive relationship between similarity of relative clause structures in the similarity of relative clause structures in the learners’ L1 and L2 and their performance on learners’ L1 and L2 and their performance on L2 relative clauses.L2 relative clauses.
There is a negative relationship between There is a negative relationship between similarity of relative clause structures in the similarity of relative clause structures in the learners’ L1 and L2 and their performance on learners’ L1 and L2 and their performance on L2 relative clauses.L2 relative clauses.
2929
HHoo
There is neither positive nor negative There is neither positive nor negative relationship between similarity of relative relationship between similarity of relative clause structures in the learners’ L1 and clause structures in the learners’ L1 and L2 and their performance on L2 relative L2 and their performance on L2 relative clauses.clauses.In most research reports, the null In most research reports, the null hypothesis, even if not formally stated, is hypothesis, even if not formally stated, is tested rather than a directional alternative tested rather than a directional alternative hypothesis. Why?hypothesis. Why?
3030
Activity 3Activity 31.1. Research topic: relationship of language Research topic: relationship of language
proficiency and spelling test scoresproficiency and spelling test scores
2.2. How to state and interpret the null and How to state and interpret the null and alternative directional hypothesesalternative directional hypotheses
Null hypothesis:Null hypothesis:
Alternative hypothesis:Alternative hypothesis:
Directional, positive hypothesis:Directional, positive hypothesis:
Directional, negative hypothesis:Directional, negative hypothesis:
3131
Null hypothesis:Null hypothesis: There is no relationship There is no relationship between l. proficiency and spelling between l. proficiency and spelling scores.scores.
The spelling scores will not vary in The spelling scores will not vary in relation to the l. proficiency of the Ss.relation to the l. proficiency of the Ss.
Interpretation of hypothesesInterpretation of hypotheses
3232
Alternative hypothesisAlternative hypothesis: There is : There is relationship …relationship …
The spelling scores will vary in The spelling scores will vary in relation to the l. proficiency of the Ss.relation to the l. proficiency of the Ss.
3333
Directional, positive hypothesisDirectional, positive hypothesis: There is a : There is a positive relationship between l. proficiency and positive relationship between l. proficiency and spelling scores.spelling scores.
Students at higher proficiency levels will obtain Students at higher proficiency levels will obtain higher spelling scores. Students at lower higher spelling scores. Students at lower proficiency levels will obtain lower spelling proficiency levels will obtain lower spelling scores. scores.
3434
Directional, negative hypothesisDirectional, negative hypothesis: There is : There is a negative relationship between l. a negative relationship between l. proficiency and spelling scores.proficiency and spelling scores.
Students at lower proficiency levels will Students at lower proficiency levels will obtain higher spelling scores while those obtain higher spelling scores while those with higher levels of l. proficiency will with higher levels of l. proficiency will score lower.score lower.
3535
Activity 4Activity 41.1. Research topic: gender differences in spelling Research topic: gender differences in spelling
test scorestest scores
2.2. state a separate null hypothesis for the effect of state a separate null hypothesis for the effect of gender on spelling scores (ignore proficiency)gender on spelling scores (ignore proficiency)
3.3. State all the interaction hypotheses (adding in State all the interaction hypotheses (adding in proficiency and L1 membership)proficiency and L1 membership)
HHo o for gender: for gender:
HHo o for gender and L1:for gender and L1:
HHo o for gender and language proficiency:for gender and language proficiency:
HHo o for gender, L1, and language proficiency:for gender, L1, and language proficiency:
3636
InterpretationInterpretation
HHo o for genderfor gender: There is no effect of gender on : There is no effect of gender on spelling scores.spelling scores.
HHo o for gender and L1for gender and L1: There is no interaction : There is no interaction of gender and L1 membership on spelling of gender and L1 membership on spelling scores. scores.
3737
HHo o for gender and language proficiencyfor gender and language proficiency: :
There is no interaction of gender and There is no interaction of gender and language proficiency on spelling scores. language proficiency on spelling scores.
HHo o for gender, L1, and language for gender, L1, and language
proficiencyproficiency: There is no interaction of : There is no interaction of gender, language background, and gender, language background, and language proficiency on spelling scores. language proficiency on spelling scores.
3838
HHo o for genderfor gender rejectedrejected: males and females : males and females
would show different spelling scores.would show different spelling scores.
HHo o for gender and L1 rejectedfor gender and L1 rejected: there might be a : there might be a
pattern where for females the scores wouldn’t pattern where for females the scores wouldn’t change across the L1 groups while for men change across the L1 groups while for men they would. they would.
3939
HHo o for gender and language proficiency for gender and language proficiency rejectedrejected: there might be a pattern where : there might be a pattern where for females the scores wouldn’t change for females the scores wouldn’t change across the proficiency levels while for men across the proficiency levels while for men they would.they would.
HHo o for gender, L1, and language for gender, L1, and language proficiency rejectedproficiency rejected: it might mean, eg, : it might mean, eg, that females of certain L1 groups who are that females of certain L1 groups who are at certain proficiency levels perform at certain proficiency levels perform differently than others. differently than others.
4040
Answering the question: Internal Answering the question: Internal and external validityand external validity
Internal validityInternal validity is the extent to which the is the extent to which the outcome is a function of the factor you outcome is a function of the factor you have selected rather than other factors have selected rather than other factors you haven’t controlled. you haven’t controlled.
External validityExternal validity is the extent that the is the extent that the outcome would apply to other similar outcome would apply to other similar situations in the real world. situations in the real world.
4141
Factors influencing the internal Factors influencing the internal validityvalidity
Subject selection (size variability, sample Subject selection (size variability, sample size)size)Maturation (developmental changes in Maturation (developmental changes in subjects) subjects) History (no control of input amount beyond History (no control of input amount beyond the experiment)the experiment)Task direction (no clear instructions)Task direction (no clear instructions)AttritionAttritionTest effect (the influence of pretest on Test effect (the influence of pretest on posttest)posttest)
4242
Subject selectionSubject selection
In order for research to be generalizable, In order for research to be generalizable, the sample used should be representative the sample used should be representative of the general population. Methods used to of the general population. Methods used to achieve this representativeness include achieve this representativeness include both both random samplingrandom sampling and the sample and the sample size large enough.size large enough.
4343
Example of subject selectionExample of subject selection
2 groups of learners have been randomly 2 groups of learners have been randomly assigned. Each group contains 10 subjects. assigned. Each group contains 10 subjects. After the instructional treatment, a test is given After the instructional treatment, a test is given to both groups. The average score for group A to both groups. The average score for group A is 9.3 while 11.3 for group B. The standard is 9.3 while 11.3 for group B. The standard deviations for both are 4.15 and 1.56 deviations for both are 4.15 and 1.56 respectively. Can we say the method used respectively. Can we say the method used with group B is superior to that with group A? with group B is superior to that with group A? Why or why not?Why or why not?
4444
With this difference between the groups With this difference between the groups for the SD, we would have to question for the SD, we would have to question whether test differences might be due to whether test differences might be due to some other, unidentified factor for the some other, unidentified factor for the score seem to indicate that the 2 groups score seem to indicate that the 2 groups are drawn from different populations.are drawn from different populations.
The problem for this research is that the The problem for this research is that the subjects are not from a homogeneous subjects are not from a homogeneous population.population.
4545
Maturation Maturation
Maturation refers to the general Maturation refers to the general developmental changes in subjects during developmental changes in subjects during the course of the research esp. in the course of the research esp. in longitudinal studies. The differences may longitudinal studies. The differences may be caused by the treatment , the be caused by the treatment , the maturation in cognitive development and maturation in cognitive development and short-term memory capability, or a short-term memory capability, or a combination of both. combination of both.
4646
HistoryHistory
History refers to the possible negative History refers to the possible negative effects of the passage of time on the effects of the passage of time on the study. For example, learners may have study. For example, learners may have more access to sources of input or use more access to sources of input or use English more outside of class. In this case, English more outside of class. In this case, we can’t say the difference in performance we can’t say the difference in performance is caused by the experimental treatment.is caused by the experimental treatment.
4747
Task directionsTask directions
When the instructions are clear, the results When the instructions are clear, the results may be valid. For example, when a may be valid. For example, when a researcher used an instrument of English, researcher used an instrument of English, the subjects may not understand the the subjects may not understand the instruments accurately. instruments accurately.
4848
AttritionAttrition
By attribution we mean that the By attribution we mean that the composition of the population studied may composition of the population studied may change the longer the study continues. change the longer the study continues. Subjects may lose interest when data Subjects may lose interest when data collection is made. collection is made.
4949
Test effectTest effect
If a research begins with a pretest, that If a research begins with a pretest, that test can affect performance during the test can affect performance during the treatment and on future tests. The test treatment and on future tests. The test alerts subjects as to what teachers expect alerts subjects as to what teachers expect them to learn. Then they may pay much them to learn. Then they may pay much attention to it during the course of the attention to it during the course of the research. research.
5050
external validityexternal validity
If a study has no internal validity, then it If a study has no internal validity, then it cannot have external validity. We cannot cannot have external validity. We cannot generalize from the data.generalize from the data.
When generalizing from the data set, we When generalizing from the data set, we are both concerned with IV but EV as well, are both concerned with IV but EV as well, namely how representative the data are namely how representative the data are for the group(s) to which we hope to for the group(s) to which we hope to generalize. generalize.
5151
Factors influencing the external Factors influencing the external validityvalidity
Population characteristicsPopulation characteristics
The effect of the research environmentThe effect of the research environment
Research effectsResearch effects
Data collection methodologyData collection methodology
5252
Population characteristicsPopulation characteristics
Research carried out with university Research carried out with university students may not be valid when applied to students may not be valid when applied to children. children.
Results from a study of English majors Results from a study of English majors may not be applicable to non-English may not be applicable to non-English majors.majors.
5353
The effect of the research The effect of the research environmentenvironment
Second language learners may become Second language learners may become more motivated simply because they are more motivated simply because they are told that they are participating in a study told that they are participating in a study that will help the researcher understand that will help the researcher understand the process of language learning. This is the process of language learning. This is called the “Hawthorne effect’. called the “Hawthorne effect’.
5454
Researcher effectResearcher effect
The researcher may unintentionally The researcher may unintentionally indicate to the subject the kind of response indicate to the subject the kind of response desired. desired.
5555
Data collection methodologyData collection methodology
Data obtained through structured Data obtained through structured interviews may differ qualitatively from interviews may differ qualitatively from data gathered through a test. It would be data gathered through a test. It would be difficult to compare the findings of these difficult to compare the findings of these studies since each approached the studies since each approached the problem in a different way. problem in a different way.