hbef4106-chp5

7/31/2019 HBEF4106-CHP5

1/22

58 OUM

TOPIC 5RESEARCH METHODOLOGY

TOPIC 5: RESEARCH METHODOLOGY

Assoc. Prof. Dr. Saat Md. Yasin

INTRODUCTION

This topic deals with the general concept in carrying out a research. It deals with the appropriatesteps in carrying out research methodology to ensure that a new researcher would not fall

into grave errors that could waste a lot of precious time and funding. This topic guides you inthe understanding of the process to fulfil certain objectives of this course.

OBJECTIVES

By the end of this topic, you should be able to:

1. plan a research in education successfully;

2. state the differences between qualitative and quantitative research designs;

3. describe a population and a sample;

4. select sample using random sampling;

5. develop research instruments;

6. conduct a pilot study;

7. explain how to establish the reliability and validity of the research instruments;

8. analyse quantitative and qualitative data; and

9. explain the differences between descriptive and inferential statistics.

7/31/2019 HBEF4106-CHP5

2/22

OUM 59

TOPIC 5 RESEARCH METHODOLOGY

TOPIC MIND MAP

5.1 WHAT IS RESEARCH?

Research is a formal and systematic application of the scientific and disciplined inquiry to thestudy of problems while methodology is a way or means of doing something such as the way

of making a scientific inquiry into a problem. Since the research methodology in educationalresearch is scientific and disciplined in nature, it has to adhere to certain procedures that

need to be strictly followed. First of all, any research to be carried out needs to be carefullyplanned to ensure the validity of the findings. We have to consider at least three important

factors to ensure that the research is properly implemented.

5.2 RESEARCH PLANNING

After identifying the area or the topic for the research, you need to formulate the purpose,objectives, questions and hypotheses of the research. You have to draw out the methods and

the procedures of the intended research. Therefore, you require expertise in a number of

7/31/2019 HBEF4106-CHP5

3/22

60 OUM


different areas. The research plan describes the nature of the participants in the research,the variables, the kind of data, the instruments to be used, the condition for data collection

and the techniques to analyse the data.

When identifying the area of study, you have to consider the type of research to be utilised. Is

it a quantitative or qualitative research? Which one is more suitable? Planning a researchusing the quantitative method is different from planning a qualitative research. You have to

consider many different factors when applying either the quantitative or the qualitative methods.

5.2.1 Research Questions and Data

Research questions are the questions that we ask to achieve the objectives for our research.

These questions can be answered through collecting of data and analysing them. After theresearch questions have been established and the topic of the study has been identified, you

have to describe the following procedures in executing the research.

(a) Describe and execute procedures to collect information. The procedures includeidentifying research participants, the strategies to collect data and activities describing,

how, when, why and from whom the data were collected. The detailed description would

enable you to smoothly carry out the research without major hitches if there were any.The flaws in carrying out the research might jeopardise the validity of the research findings.

(b) Analyse the collected data. The data would be analysed differently depending on thenature of the research. If the research was quantitative, then the data statistical analysis

might be needed to achieve the desired findings. On the other hand, if the research wasqualitative in nature, then the data analysis requires a descriptive explanation of the

findings. Lengthy notes might be scrutinised and summarised. The observations made

might be transcribed in coherent and easily understood explanations.

(c) After the data have been analysed, the result or the findings of the study has to be

presented coherently. They must be clearly stated and the implications of the findings

should also be included. The findings might be useful to interested parties for various

reasons. The government might want to improve whatever shortcomings pointed out bythe findings. The school principals might want to use them for school improvement.

1. What is the expertise needed in planning a research successfully?

2. What are the factors that you need to consider when applying either thequantitative or the qualitative methods?

7/31/2019 HBEF4106-CHP5

4/22

OUM 61


5.2.2 Selection of Research Design

A research design is a plan to carry out an intended research. According to Creswell (2002),

there are eight different research designs which are frequently used in educational research.

They are either quantitative or qualitative in nature or in some instances, the combination ofboth.

(a) Quantitative Research DesignsThe experimental and quasi-experimental designs will be explained separately.

(i) Experimental DesignsThey are usually referred to as group comparison studies because researchers

compare different groups in terms of one or more outcomes. Experiments are alsocalled intervention studies because you intervene or manipulate the variables.

The experimenters design and conduct experiments where they maximise theinfluence of the independent variable on the dependent variable, and at the same

time minimise the influence of the extraneous factors. There are five keycharacteristics of experimental research designs that are of importance to you:

Participants are selected and assigned at random to different groups, namely

control and experimental groups.

Intervention is applied to one or more groups

Outcomes are measured at the end of the experiment

Procedures are designed that address potential threats to validity

Statistical comparisons of different groups are conducted.

In the educational scene, experimental research could be used in various situations.One such situation is to find out the effectiveness of a new method of teaching over

a conventional one. Two groups of students were randomly selected. One group is

used as the experimental group using the new method of teaching, while the other

is the control group using the conventional method of teaching. The topic of thelesson is chosen from the syllabus and the different treatments are given. After the

Figure 5.1: Research design

RESEARCH DESIGN

Quantitative

1. Experimental2. Quasi-experimental3. Correlational4. Survey

Combination ofQuantitative andQualitative

1. Mixed Method2. Action Research

Qualitative

1.Grounded Theory2. Ethnographic3. Narrative

7/31/2019 HBEF4106-CHP5

5/22

62 OUM


treatments had been completed, the outcome of the experiment is measured. Theoutcome might be in the form of the academic achievement through the test results

of the students in the control group and the experimental group. This statistical

analysis would provide the answer whether the experimental group performedsignificantly better than the control group.

(ii) Quasi-Experimental DesignMost of the situations in schools need you to use intact groups such as in studying

a new English programme. This study may involve the students in the same grade

as respondents where one class is considered the control group while the other isconsidered as the experimental group. Randomly assigning students to the two

groups would disrupt classroom learning. Because teachers often use intact groupin experiments, quasi-experimental designs are usually used.

(iii) Correlational Designs

A science teacher wants to know whether the assumption that students who are

good in mathematics are also good in science is true or false. In order to find out,she compares the examination results of her students in the two subjects, themathematics and science. She correlates the mathematics test results and the

science test results of the students. If the students who scored high marks inmathematics also scored high marks in science, we can conclude that there is a

positive correlation between the mathematics and the science results, and if this

finding is consistent with other research findings in the same area, we could saythat the assumption is correct.

The above anecdote describes one of the correlational designs used in the

educational field. Correlational research in the above case uses a statisticalprocedure. Now, with sophisticated applications and explicit procedures of

correlations, such as regression, path analysis, and causal modelling, correlationalresearch is entrenched in the family of quantitative research. Correlational researchas a design, has two primary forms, namely explanation and prediction. Correlational

research helps to explain the association between two or more variables or to predict

an outcome.

(iv) Survey DesignsSuppose the Ministry of Education (MOE) wants to know what the parents and the

students feel about having one session schools where students begin school at8.00 a.m. and ends at 4.00 p.m. The research question might be: What are the

opinion of the students and their parents toward one session school and lengtheningof the school session?By using a survey design, the MOE would gather the opinions

of the students and their parents. The survey design is an economical way of doingresearch especially when the respondents are large. The MOE would randomly

select the respondents from the two groups: students and their parents, and sendthem a mailed questionnaire, analyse the results and generalised results from of

sample to the population. This is a survey that is commonly used in the school

system. Survey designs are also frequently used to:

describe trends;

determine opinions about policy issue, such as giving loans to all students in

tertiary institutions;

background of lecturers from private and public colleges and universities;

understand important attitudes and beliefs towards dating among secondary

school students;

7/31/2019 HBEF4106-CHP5

6/22

OUM 63


identify practices, such as carrying weapons to schools and gangsterism; and

evaluate programmes, such as using English for mathematics and science

subjects.

Survey designs are mainly of two types: cross-sectional and longitudinal. Cross-

sectional designs are used to collect data that reflect current attitudes, opinions or

beliefs. Longitudinal designs are used to study individuals for a period of time suchas studying people who were high achievers and low achievers in schools.

1. What research design do you use when you want to know the effectivenessof your new method of teaching oral English?

2. Explain how you implement the design.

3. What are the differences between Experimental Design and Quasi-

experimental Design?

(b) Qualitative Research DesignsThe qualitative types will be explained in the following paragraphs.

(i) Grounded Theory Designs

A teacher who had been teaching for the past twenty years had the experience ofteaching students from different socio-economic backgrounds. He found out that

most of the high achievers were from the higher socio-economic background andmost of the low achievers were from the lower socio-economic background. As a

result, he developed his own theory that students from the higher economicbackground have parents who are more concerned that their children must have

good grades. Their emphasis on discipline and spending more time in doing theirhomework enables their children to excel academically. On the other hand, students

from the lower socio-economic background spend less time on studying, lack of

parental supervision, and have non-conducive home environment for learning whichdo not help them to perform well. The teacher wanted to make a study on this

phenomenon which is grounded on a theory. He wanted to test his hypothesisthat includes the following:

Students from a higher socio-economic background spend more time studying.

Parents from a higher socio-economic background send their children to tuitionclasses.

Students from a higher socio-economic background have relatives who are high

achievers as their models.

The research that is most likely to be used in this situation is of qualitative type innature. The research based on theory, as in the above case, requires observation

as well as interviews. The data collected would be lengthy and qualitative in nature.The data would be analysed and a report of the findings has to be made. To know

how this Grounded Theory Designs work, let us examine one published groundedtheory studies (Creswell , 2002):

7/31/2019 HBEF4106-CHP5

7/22

64 OUM


Table 5.1: Nine Types of Ethnographic Designs

Realist ethnography An objective scientifically-written ethnography.

Confessional ethnography A report of the ethnographers fieldwork experiences.

Life history A study of one individual situated within the cultural context.

Auto-ethnography A reflective self examination by an individual set within hiscultural context.

Micro-ethnography A study focused on a specific aspect of a cultural group.

Ethnographic case study A case analysis of a person, event, activity, or process.

Critical ethnography A study of women and the cultural practices that serve todisempower and oppress them.

Postmordern ethnography An ethnography written to challenge the problems in our society.

Ethnographic novels A fictional work focused on cultural aspects of a group.

Richie, Fassinger, Linn and Johnson (1997), studied the career development process of18 high-achieving African American-Black and white women by generating a theoretical

model. They explored critical influences in the careers of these 18 women. From theinterview data, they developed a theoretical model that visually portrayed how socio-

cultural conditions, personal backgrounds, such as socio-economic background,influence the womens strength and passion for work and interconnectedness with others

in their careers.

(ii) Ethnographic DesignsThe term ethnography literally means writing about groups of people (LeCompte& Schensul, 1999: 21). Using this design, you can identify a group of people to

study, visit the setting of the group, note how they behave and interact with others,

how they think and develop a general picture of the group. It can provide a day today detailed picture of their activities. As an example, let us look at, The life of

residential school students in Malaysia. This study requires you to observe thestudents and make notes on their daily activities. You could become a participant

observer in order to know and understand the details of their daily life. By being withthem for a certain length of time, you would have an ethnographic view of the

students.

Ethnographic designs are qualitative research procedures for describing behaviour,beliefs, and language that they developed over a period of time. In the case of

residential school students in the example mentioned, they have their own patternof behaviour and their shared values which are likely to be different from other

students in non-residential schools. Van Maanen (1988); Denzin (1997); LeCompte,

Preissle and Tesch (1993) had classified nine (9) types of ethnographic designs asshown in Table 5.1.

7/31/2019 HBEF4106-CHP5

8/22

OUM 65


The different ethnographic designs stated above share similar characteristics whenreporting the findings. If an investigator intends to utilise one of the ethnographic

designs above, he/she has to read the subject matter further to ensure the most

appropriate approach for the research.

(iii) Narrative Research Designs

Narrative research is a design that is used by individuals to tell their personal, firsthandaccounts to researchers. In narrative research, investigators describe the lives of

individuals, collect and tell stories about peoples lives and write narratives ofindividual experiences (Connelly & Clandinin, 1990). As a distinct form of qualitativeresearch, a narrative usually focuses on studying a single person, gathering data

from the collection of stories.

Narrative research can assume multiple forms. Narratives may include a variety ofresearch practices as was compiled by Casey (1995, 1996). Examples of types of

narrative research forms are: Autobiographies.

Biographies.

Life Writing.

Personal Accounts.

Narrative Interviews.

Personal Documents.

Documents of Life.

Life Stories and Life Histories.

Oral Histories.

Ethnohistories.

Ethnobiographies

Autoethnographies.

Ethnopsychologies.

Person-centred Ethnographies

Popular Memories.

Latin-American testimonies.

Polish memoirs.

For teachers in Malaysian schools, not all of the above types of narrative research

could be applied. Only some of the most commonly used, such as autobiographiesand biographies, are usually utilised in research.

(c) A Combination of Qualitative and Quantitative Research Designs

The following paragraphs will elaborate on the combined qualitative and quantitativeresearch designs.

(i) Mixed Method Designs

A mixed method research design is a procedure of collecting both qualitative andquantitative data in a single study. Since the 1930s, educational and social science

7/31/2019 HBEF4106-CHP5

9/22

66 OUM


1. If a researcher wants to find out the reasons why some students achieved

excellent academic results, while others failed to do so even though the

initial academic potentials seem to be equal, what research design doyou suggest a researcher could choose? Explain.

2. What action research can a teacher do in school to improve his/her

teaching?

researchers have combined research methods of data collection in their studies(Sieber, 1973). Sieber suggested the combination of in-depth case studies with

surveys, creating a new style of research where quantitative and qualitative methods

were used within a single study. The rationale for carrying out the mixed methoddesigns in research is mainly to ensure that majority of the factors that influence

the situation could be taken care of. Majority of the variables have been looked intousing quantitative methods, but variables that could not be taken into account through

quantitative research, could be studied through qualitative research.

If a nationwide research is to be carried out where variables of various kinds, suchas the family, the school system, the curriculum, the students, and funding were to

be studied, the mixed method design would suit the purpose. Take study habitsas an example. When the details on how a student manages his/her time and the

method of how he/she learns, qualitative research such as ethnographic researchmight be better than just quantitative data, such as the number of hours spent on

studying. Other types of data that could easily be carried out quantitatively, such asfamily income, number of siblings, and academic background should be collected

quantitatively.

(ii) Action Research DesignsAction research addresses a specific, practical issue and seeks to obtain solutions

to the problem. Similar to the mixed method research, action research uses data

collection based on either quantitative or qualitative methods. Action researchprovides educators an opportunity to carry out research in the classroom to reflecttheir teaching practices. Action research offers a means for staff development; for

teachers development as professionals, and simultaneously addressing school-wide problems. As an example, an English teacher found out that her students

were weak in English from the students test results and her interaction with them.So, she decided to enquire from the students regarding their background such astheir families socio-economic background, the number of books they read in English,

whether the family members communicate in English with each other and theirattitude toward the English language. These variables could be researched

quantitatively as well as qualitatively to collect valid data and find out the variablesthat influenced the students command of the English language.

The findings of this action research would help the English teacher to know some

of the reasons why most of her students are weak in the English language. Thiswould help her to design a remedial programme to improve the students command

of the English language. If teachers are able to carry out action research to improve

their teaching, overall quality of the students learning would improve.

7/31/2019 HBEF4106-CHP5

10/22

OUM 67


5.2.3 Research Population and Sample

Figure 5.2: Research population and sample

What is the definition of population? Gay and Airasian (2003), defined population as The

population is the group of interest to the researcher, the group to which the results of the

study will ideally generalized. To determine a population is important in an educational research.The population determines how far and how much data should be collected and analysed.

The interpretation and inference of the data is also limited to the population of the study. Thepopulation also determines how much time and energy should be devoted to solve a problem.

Population also influences how much fund is needed to carry out the research. As an example,a study on the needs of the rural students to improve their academic performance in a certain

school is different from a study on the needs of rural students in Malaysia. The populations ofboth samples are different in size which also determine the types of research designs to be

carried out.

Another important factor for the researcher to consider is accessible oravailable population.In most studies, accessible population is generally a realistic one compared to the target

population which is more of an idealistic one. Although we can use the whole population forour research, it is not practical because it requires a large amount of funding, manpower and

time to carry it out successfully. Therefore, we have to devise a means that would do the

same job without distorting the findings. This is done through random sampling.

Selecting a sample is a very important step especially when conducting quantitative research.A good sample is one that is representative of the population from which it is selected. The

steps in sampling can be summarised as follows: identify the population;

determine the required sample size; and

select the sample.

Take an example in selecting a sample from the Form1 students in a school. You as a teacher,first of all, have to identify the population, that is the Form 1 students in the school. Afterwards,

you need to determine the sample size. Out of the 200 students for example, you need tosample 60 students; 30 percent of the population. You need to select randomly from the

cohort of Form 1 students of the school. You can use one of the sampling techniques. These

sampling techniques are all based on randomness in the selection of the sample. There arefour basic random sampling procedures, namely:

Simple Random Stratified

RESEARCH POPULATION AND SAMPLE

Selecting Sample

Cluster Systematic

7/31/2019 HBEF4106-CHP5

11/22

68 OUM


simple random sampling;

stratified sampling;

cluster sampling; and systematic sampling.

(a) Simple Random SamplingRandom sampling is a process of selecting a sample in such a way that all individuals inthe selected population have an equal chance to be selected into the sample. Random

sampling is the best way to obtain a representative sample. The procedures in simplerandom sampling involve the following specific steps:

(i) Identify the population and define it.

(ii) Determine the sample size.

(iii) List the names of all members of the population.

(iv) Assign all individuals on the list a consecutive number from zero to the requi re dnumber, for example, 000 to 200 where each individual has the same number ofdigits as the others.

(v) Select an arbitrary number from the table of random numbers. As an example, byclosing your eyes while pointing to a number in the list.

(vi) Go to the next number until the desired number of individuals has been se lect ed

for the sample. If 30 percent of the 200 members in the population are to be selected,then 60 students in the sample will fulfill the desired percentage.

(b) Stratified SamplingThis is a process of selecting a sample in such a way that identified sub-groups in thepopulation is represented in the sample in the same proportion that they exist in the

population. For example, if we want to find out the academic qualification of teachersteaching various subjects, the population of the teachers that are to be studied is not

uniform because some of these teachers have degrees while the others do not havethem. Simple random sampling cannot be applied in this situation due to the fact that it

does not represent the strata that exists in the population. Besides, simple random

sampling to the above population will increase sampling error. Therefore, simple stratifiedsampling should be used to reduce sampling error.

The first step in carrying out the study using stratified sampling is determining the strata

to be studied such as by categorising the graduate and non-graduateteachers. Thenfrom every stratum, respondents are selected at random to represent the teachers

population. Random sampling for every stratum could be generated using the samestrategy as simple random sampling. The allocation of sample size between strata canbe done proportionately. If 60% of the teachers are non-graduate and only 40% are

graduates, then the sample should consists of 60% non-graduate teachers and 40% for

graduate teachers.

(c) Cluster Sampling

Cluster sampling is a process of using a sample whenever the sampling unit is not theunit that represents the population but represents a group or a cluster within the population.

For example, we dont have a complete list of teachers. We can use cluster samplingby determining the number of clusters such as the teachers based on the states they

come from. Sampling based on clusters could be done randomly by using the table of

7/31/2019 HBEF4106-CHP5

12/22

OUM 69


random numbers. When the cluster that represents the population has been randomlyselected, we must use every unit in the cluster. After we have randomly selected the

teachers from one school district such as Putrajaya, all members within the cluster

must be used as the respondents for the study.For a clearer illustration of cluster sampling, note the following steps:

(i) Identify and define the population.

(ii) Determine the required sample size.

(iii) Identify and define a logical cluster (school, teachers in the district, primary

school teachers).

(iv) List all clusters that make up the population of clusters.

(v) Estimate the average number of population members by clusters.

(vi) Determine the number of clusters needed by dividing the sample size by theestimated size of a cluster.

(vii) Randomly select the required number of clusters.

(viii) Include all members in the selected cluster in your study.

(d) Systematic Sampling

It is a method of sampling where individuals are selected from a list taking every Nthname. If N here represents 5, then every 5 th name will be selected as the sample for the

study until the required numbers are met. This method of sampling though is not usedvery often.

The major difference between systematic sampling and the other types of sampling is

that all members of the population do not have an independent chance of being selectedfor the sample. Once the first name is selected, all the rest are automatically determined

for the sample. Therefore, systematic sampling is often not as good as random sampling.There is a possibility that students from a particular ethnic group having names that are

concentrated on certain letters could be excluded from the research through omission

by using systematic sampling.

Systematic sampling involves the following steps:

(i) Identify and determine the population

(ii) Determine the desired sample size.

(iii) List the names of the population.

(iv) Determine the N by dividing the size of the population by the desired sample.

(v) Start at some random place in the population list. Close your eyes and pointyour finger on a name.

(vi) Starting at that point, take every Nth name until the required sample size is

reached.

1. What do you understand by accessible population and target population?

2. When do you utilise systematic sampling?

3. Why is random sampling preferred over other sampling techniques?

7/31/2019 HBEF4106-CHP5

13/22

70 OUM


5.3 RESEARCH INSTRUMENTS

We are familiar with the terms test, examination, assessmentand measurement. A test or

examination is a formal, systematic procedure for gathering information about peoplescognitive, psychomotor and affective characteristics. A standardised test or examination isadministered, scored, and interpreted in the same way no matter where or when it is

administered. These standardised tests and examinations are developed by experts. For

example, the Ujian Penilian Sekolah Rendah (UPSR) or Primary School Assessment Test,Penilaian Menengah Rendah (PMR) or Lower Secondary Assessment, Sijil Pelajaran

Malaysia(SPM) or Malaysian Certificate of Education and the Sijil Tinggi Pelajaran Malaysiaor Malaysian Higher School Certificate are all standardised to ensure that all test - takers

experience the same conditions when taking them. Assessment is a broader term than testor instrument, and encompasses the process of collecting, synthesising and interpreting

information. Tests are a subset of measurement, as are observations and interviews.Measurement is the process of quantifying or scoring ones performance.

5.3.1 Developing Research Instruments

Before you carry out a research, you have to develop the research instruments. If it is to be atest, it should be planned and developed well to suit the purpose and objectives of the research.

A well-developed test would measure the variable of the research accurately. The accuratenessof the collected data is very important to ensure that the collected data could be analysed and

interpreted correctly.

(a) Determining the Objectives

The test should be constructed according to its objectives. The behavioural objectives

to be measured should be realistic and stated clearly. Objectives should also be statedbehaviourally and not in the form of learning activities. The statement of the objectives

should be linear whereby each and every statement should be related to only one process.

Figure 5.3: Developing research instruments

Determining Objectives

Determining Contents

DEVELOPING RESEARCH

INSTRUMENTS

7/31/2019 HBEF4106-CHP5

14/22

OUM 71


(b) Determining the ContentsThe contents of the area to be measured should be determined first. An effective test is

a test which has a high content validity. A test should cover all contents of the research

area. To ensure the test used covers all contents, you should first determine the contentsof the research area. The opinion of the expert in the area could be utilised to ensure the

test to be developed has high reliability and validity.

Usually the content of the area of study is wide, so you could use sampling techniques

of the content as representatives of the content. You need to determine content sampling

to represent the behaviour to be measured. If content sampling is used, you need toensure that they are representatives of the behaviour to be measured. If this method is

used, the researcher should ensure that the chosen contents could represent the overallcontents that are to be measured.

5.3.2 Reliability and Validity of Research Instruments

Both reliability and validity of research instruments are vital in any research. Validity is themost important characteristic of a test or a measuring instrument. Validity is concerned with

the appropriateness of the interpretations made from test scores. When we test, we test it fora purpose. For example, you may administer a questionnaire to find out the participants

opinions on the new methodology used in teaching mathematics compared to the traditionalmethod where talk and chalk were used. A key question for these test users is, Does this

test allow me to make the interpretation I wish to make? That is, will responses to thequestionnaire allow you to make appropriate interpretations about the respondents opinions?

Validity is important to all forms of research and all types of tests and measures. There are

four types of test validity:

content validity;

criterion-related validity;

construct validity; and

consequential validity.

(a) Content ValidityContent validity is the degree to which a test measures an intended content area. It

requires both item and sampling validity. Item validity is concerned with whether the

test items really measure the content area. Sampling validity is concerned with howwell the test samples the total content area being tested. Content validity is particularly

important for achievement tests. A test score cannot accurately reflect a students

achievement if it does not measure what the student was taught and supposed to learn.The validity of the content is determined by expert judgment. Experts in the topic covered

by the test are asked to assess its content validity. The judgment is made by comparingwhat was taught and what is being tested. The validity would be very strong when the

two coincides.

(b) Criterion-Related Validity

Criterion-related validity has two forms, namely concurrent and predictive validity.

(i) Concurrent validity is the degree to which scores on one test correlate to scoreson another equivalent test. For example, a test is developed that claims to do the

same thing as some other tests, except easier or faster. One way to determine

7/31/2019 HBEF4106-CHP5

15/22

72 OUM


whether the claim is true or otherwise is to administer the new and the old test to agroup of students and compare the scores. If there is a high correlation between

them, the concurrent validity of the new test is established and it may be utilised to

replace the older test.(ii) Predictive validity is the degree to which a test can predict how well individuals

will do in a future situation. Predictive validity is very important for tests that used toclassify or select individuals. In Malaysia, we use the test results of UPSR for

selection into residential schools. These test results are considered as having high

predictive validity. The selected students admitted into these schools are expectedto perform well in the next public examination three years later, namely the PMR.

No test would have perfect predictive validity, and therefore predictions based on

the scores of any test will be imperfect. However, predictions based on a combinationof several test scores will be more accurate. In order to establish the predictive

validity of a test, the criterion has to be carefully defined, which is the valid measure

of the performance to be predicted. Once the criterion has been identified, theprocedure for determining predictive validity is as follows:

Administer the test which is considered as the predictor to a group.

Wait until the behaviour to be predicted occurs.

Obtain measures of the criterion for the same group.

Correlate the two sets of score.

Evaluate the results.

Once the two results are correlated, and if the coefficient is high, the test has good

predictive validity.

(c) Construct ValidityConstruct validity is the most important form of validity because it asks the question:

Wha does this test measure? Constructs are non-observable traits, such as anxiety,

intelligence and honesty. You can only observe its effect. Constructs can explain certaindifferences between individuals. As an example, it was observed that some studentslearn faster than others or learn and remember more than others. To explain these

differences, a theory on intelligence that is related to learning was developed. This theoryexplains that students who have the above qualities are more intelligent than others.

They also perform better in examinations. Tests were developed to measure how much

intelligence a person has. The construct or theory explains that students who score

more tend to do better in school and other learning environment than those who haveless of it. Those with high scores are said to be more intelligent than others.

Research involving a construct are valid only to the extent that the instrument usedactually measures the intended construct. Determining construct validity is not the easy

and it usually involves gathering a number of pieces of evidence to demonstrate thevalidity. For example, if some psychologists construct an intelligence test instrument

called Smart Ali, the validity of the instrument should be subjected to various tests.First of all, we could see whether students who scored high on the Smart Ali test learned

faster, learn more and remember more than low scorers. We could correlate the scoresof the students at the beginning of the school year and at the end of the school year. We

could also correlate performance on the Smart Ali test with other well established

intelligence tests to observe whether the correlations were high. Scholars should alsobe invited to examine whether the test items represent the typical topics in the field of

7/31/2019 HBEF4106-CHP5

16/22

OUM 73


intelligence. In addition to this confirmatory evidence, we need to seek disconfirmatoryvalidity information. For example, we do not expect this intelligence test to be highly

correlated to self-confidence or anexiety tests.

(d) Definition of ReliabilityReliability is a synonym for consistency and replicability over time, over instruments and

groups of respondents (Creswell, 2002). It is about precision and accuracy. A reliable

research must show that if it were to be done on a similar group of respondents in asimilar context, than similar results would be obtained. There are three types of reliability,namely; stability, equivalence and internal consistency.

(i) Reliability as Stability

Reliability as stability is a measure of consistency over time and over similarsamples. It means that a reliable instrument will achieve similar data from similar

respondents over time. In the experimental model, for example, this would meanthat a test and then a retest were taken within an appropriate time span, similar

results would be obtained. Correlation coefficients can be calculated for the reliability

of the pre-and post-tests.

(ii) Reliability as EquivalenceReliability could be achieved through using equivalent or alternative forms of a test

or data gathering instrument. If the equivalent forms for the test is devised andproduces similar results, then the instrument could be said as having this form of

reliability.

(iii) Reliability as Internal ConsistencyInternal consistency demands that the instrument be run only once unlike the other

two mentioned earlier (stability & equivalence). Internal consistency could bedemonstrated by running the test once only through the split-half method. Let us

suppose that a test is to be administered to a group of students. Here, the testitems are divided into two halves matching in terms of difficulty and content. If the

test is to show split-half reliability, then the marks obtained in each half should becorrelated highly to the other half.

Figure 5.4: Three types of reliability

Reliability as

Stability

Reliability as

Equivalence

Reliability as

Internal

Consistency

RELIABILITY

7/31/2019 HBEF4106-CHP5

17/22

74 OUM


1. Why are the reliability and validity of research instruments vital?

2. What is the difference between reliability as equivalence and reliability

as internal consistency?

3. Explain the difference between predictive validity and concurrent validity.

5.3.3 Pilot Study

A pilot study is a pre-research study undertaken on similiar set of respondents: in terms of

location, nature of setting and profile of respondents (age, gender, etc). This is done to ensure

the validity of instruments and to resolve possible problems before the actual research isdone.

Pre-testing of the instruments to be used in a research is of vital importance. A pilot study hasseveral functions, mainly to increase the reliability, validity and practicability of the questionnaire

(Oppenheim, 1992; Morrison, 1993; Wilson and McLean, 1994). Some of the objectives of

piloting the questionnaire are (Cohen, Manion & Morrison, 2000):

To check the clarity of the questionnaires including instructions and layout.

To gain feedback on the validity of the questionnaire items.

To eliminate ambiguities.

To gain feedback on the type of question and its format.

To gain feedback on response categories for closed questions.

To gain feedback on the attractiveness of the questionnaire.

To gain feedback on the layout of the questionnaire.

To check the time taken to complete the questionnaire.

To identify redundant questions.

To identify commonly misunderstood items..

To try out the coding system for data analysis.

After the pilot study, you can refine the questionnaire based on the responses of the

respondents in the pilot study. This refinement would have eliminated all the shortcomings ofthe instruments. Then, you can proceed with the actual data collection for the research.

5.4 DATA ANALYSIS

Data analysis becomes very crucial after all the required data were collected. Data analysis

has to be done before we could make any inference. Numbers have to be crunched usingstatistical analysis and statistical values have to be interpreted. Quantitative and qualitativedata have to be treated differently using guidelines provided in this module. Let us look at

more information regarding data analysis.

7/31/2019 HBEF4106-CHP5

18/22

OUM 75


5.4.1 Measuring Research Variables

Data are collected from the respondents through a questionnaire or from test results. The

questionnaire items were generated from research variables. In educational research, thetypical variables are as follows:

Leadership styles of school principals.

Academic achievement of students.

Interpersonal communication skills.

Attitudes toward school.

Teaching styles of teachers.

Socio-economic background of parents.

Time spent on studying and doing home-work. Reading habits of students.

These variables are the main ideas that researchers seek to address in their studies. A variableis a characteristic or attribute of an individual or an organisation that you can measure orobserve. It also varies among individuals or organisations. When we talk about a variable

as a characteristic we mean personal aspects about the individuals, such as their age andincome level. An attribute represents how an individual in an organisation, feels, thinks or

behaves. When we discuss about individuals having self-confidence or displaying leadershiptraits, we are talking about their attributes.

Researchers measure these characteristics and attributes. Measurement means recordinginformation from individuals through:

Observation; or

asking questions.

In either case, student scores vary. The variables vary means the scores will have differentvalues depending on the respondents characteristics. For example:

SPM Examination results vary by nine possible scores: 1 being the highest, which isequivalent to a distinction, and 9 being the lowest which is equivalent to a failure.

Gender varies by two possible scores: male = 1 and female = 2.

In a quantitative study, most researchers used to record the scores by using the Likert Scale

on a 5 point scale. For example, we may state a statement: The teacher always usesexamples to explain difficult concepts. The 5 point scale used would most probably be:

5 = strongly agree

4 = agree

3 = moderately agree

2 = disagree

1 = strongly disagree

7/31/2019 HBEF4106-CHP5

19/22

76 OUM


Figure 5.5: The Qualitative Process of Data Analysis

Code the Text forDescription to be Usedin the Research Report

Code the Text forThemes to be Usedin the Research Report

SimultaneousIterative

text segments andCode the Data(i.e., locates

assigns a code label to them)

Read Through Data

(i.e., obtains a generalsense of material)

Prepare Data for Analysis(i.e., transcribes fieldnotes)

transcriptions, or

Collect the Datai.e., a text file, such as fieldnotes,optically scanned materials)

The respondents would choose one of the values to state their agreement or disagreementto the given statement. The high scores denote that there is a strong agreement to the

statement and vice versa. When a research variable in a construct it is usually measured by

taking the sum or average of scores of all related items in the questionnaire.

On the other hand, a qualitative study requires a different method of description. Instead ofutilising numbers to represent some ideas, the researcher might have to write lengthy

descriptive narrations, which later on have to be summarised and classified under differentselected variables. The analysis of qualitative and quantitative data will be explained in moredetails later.

5.4.2 Analysing Qualitative Data

Qualitative researchers use to analyse data by using the following steps:

Preparing and organising the data for analysis.

Describing and developing themes from the data.

Reporting the findings.

Interpreting the findings.

Validating the accuracy and credibility of the findings.

In a summary form, the qualitative process of data analysis was vividly presented by Creswell

(2002:258).

7/31/2019 HBEF4106-CHP5

20/22

OUM 77


Figure 5.5 summarises the general process of analysing qualitative data. Qualitative

researchers collect data first and then prepare it for data analysis by transcribing field notes.Your next step is to read through the data to have a general sense of the material. Since there

is a lot of data, you have to code the data for easy referral later on. Qualitative researchersneed to analyse their data by reading through several times and conduct an analysis each

time. In doing so the researcher would have sifted through a lot of information which could beuseful in writing the final report. A useful conceptualisation is to think about the process as

both simultaneous and iterative, which means going back and forth a few times. Therefore,there is not one single way to analyse qualitative data. It is a process in which you try to make

sense of the information.

5.4.3 Analysing Quantitative Data

Before quantitative data could be analysed, a researcher would have to prepare and organise

the data for analysis. The following steps would be useful as a guide:

(a) First of all, you have to assemble all data by transforming it into numeric score or value

to each response category for each item. Before conducting an analysis of scores, youshould consider what types of scores to be used. For example, a single item score isan individual score to each item for each participant in your study. The next one is

summed scores which is the scores of an individual added over several items that

measure the same variable. The other is the net ordifference scores which are scoresin a quantitative study that represent a difference or a change for each individual.

(b) The next step is selecting a statistical programme. With the presence of the computer

age, selecting a suitable programme for analysing quantitative data is really not a big

problem. If you are a student in a university, use the programme that is available at theuniversity.

(c) The third step is inputting data. This is a process when you transfer the data from theresponses on instruments to a computer file to be used for analysis.

(d) Exploring and Descriptively Analysing the Data is the fourth step. By doing this, you

engage in data analysis. This procedure consists of:

exploring and describing the data; and

conducting statistical tests on the data.

By exploring the data, you examine for errors and missing data so that decisions can be

made about the missing data.(e) Cleaning and Accounting for Missing Data

You must clean the data because of errors in scoring that you make when inputting thedata. This is a process of visually inspecting the data for scores outside the accepted

range. This will alert you on the errors which were made especially after a tiring process

of inputting data. Sometimes data to individual items are missing. Missing data isinformation that is not supplied by participants to specific items. Missing data will reduce

the statistical options available for your statistical analysis. Therefore, how would youhandle this problem? The obvious approach is to avoid having missing data by providingitems such that respondents are willing and able to answer. If your database contains

missing data, there are several technical procedures that can help you. One procedure

is eliminating respondents with missing scores from the data analysis, but this practicemay reduce the number of overall respondents. Some researchers recommend

7/31/2019 HBEF4106-CHP5

21/22

78 OUM


1. Why should you conduct a pilot study?

2. What would you do with missing data?

3. In qualitative research, how would you transcribe field notes?

4. What are the skills needed in analysing data from quantitative research?

substituting values for the missing cases. The researcher can have the computerprogramme to substitute a value for each missing score, such as the group mean (George

& Mallery, 2001).

(f) Calculating Descriptive and Inferential StatisticsDescriptive statistics present information that helps you in assessing the general trend

in the data and in answering descriptive research questions in the studies. Theseobjectives can be achieved by running the frequency distribution and its statistics. To

understand the statistics, you need to have some knowledge about them. These statistics

are the numbers derived from formulas to measure aspects of a set of data such as:

Measures of central tendency which are summary numbers that represent a single

value in a distribution of scores (Vogt, 1999). They are expressed as an average

score (the mean), the middle of a set of scores (the median) or the most frequentlyoccurring (the mode).

Measures of variability indicate the spread of the scores in a distribution (Creswell,2002). Variance, standard deviation, and range all indicate the amount of variability ofscores.

Measures of relative standing are statistics that describe one score relative to a

group of scores (Creswell, 2002). The percentile rank and z-score are two frequentlyused statistics.

The other basic types of statistics are inferential statistics which are briefly explained

here.

Inferential statistics refer to statistics that enable a researcher to draw conclusions,inferences, or generalisations from a sample to a population. Inferential statistics are

concerned with determining how likely it is that results based on a sample are similiar tothese obtained from the population (Gay & Airasian, 2003). The question that guides

inferential statistics is whether expected differences are real, significant ones or only the

result of sampling errors. Inferences concerning populations provide only probabilitystatements where you are never certain when making an inference about a population.

In inferential statistics, there are many other factors that need to be learnt. A course on

statistics would provide sufficient knowledge regarding inferential statistics. Other termsthat are important for inferential statistics are standard error, the null hypothesis, tests of

significance, the t-test, multiple regressions, chi square, and other more advancedstatistical analysis which can be found in statistics texts.

7/31/2019 HBEF4106-CHP5

22/22


SUMMARY

The topic on Research Methodology gives a brief explanation on the process of how to

implement a research successfully. This topic needs to be read with understanding withother topics written in this module. Some of the terms and procedures would be better

understood through reading other related topics. Questions are also given to help youunderstand the concepts and the process of the methodology of research. In order for you to

comprehend this topic well, you need to read other topics because all topics are inter-relatedwith each other.