Psychological testing

Spring 2015

Mikołaj Winiewski, PhD

Marcin Zajenkowski, PhD

Mikołaj Winiewski

mwiniewski@gmail.com

Marcin Zajenkowski

zajenkowski@psych.uw.edu.pl

COURSE PAGE

http://icelab.psych.uw.edu.pl/students-area/

Password: intelligence

Aim of the course

• Basic issues concerning construction and utilizing psychological tests.

• Fundamental vocabulary and logic of psychological measurement.

• Logic of design and psychometric properties behind any psychological test.

• This course is a LECTURE which means that presence in class IS NOT IS NOT MANDATORY.

• If you can read and understand assigned literature you should pass the exam

– around 70% of the exam questions will be based on assigned basic literature

Expectations and Class Etiquette

Our responsibility (as instructors) to you (as students) is to help you reach your full potential. As such, you may expect the following from us:

• We will come to class on time • We will have your tests quickly graded • We will listen to, and carefully consider any well

thought out question or concern you have about grading, or any other aspect of the class

• We will treat each one as a university student. That means the responsibility for doing well in this class lies with you, but We shall endeavor to assist you

Expectations and Class Etiquette

In order for this class to be a success, and for you to get as much as possible from it, We expect at least the following from you:

• You will come to class on time • You will not hold idle conversations during the lectures • You will not use mobile phones, game consoles or any other

stuff that is not necessary for participating in class • You will not eat in the class • In your dealings with any person in this class, you will

conduct yourself, always, with a minimum of respect, making sure to make your contributions in a manner that is neither rude nor offensive

Expectations and Class Etiquette

• There will be two summative exams: midterm and final worth in total 60 points. – Midterm exam (25 points) will cover material from

lectures 1 – 6

– Final exam (35 points) will cover material from lectures 7 - 15.

– Retake will cover all material.

• Exams will consist several multiple choice and open ended questions covering material from the literature and lectures.

Grading

Extra Credit

Group assignment worth additional 3 points.

• Select one of five proposed construct and create a draft of the tool to measure it.

• Presentations (5 minutes) – last class.

Notice: Your paper is to be the product of independent work. Plagiarism is unacceptable and will result in an automatic FAILURE (even thou it's an extra credit).

Extra credit

• Choose one psychological construct (e.g. altruism, verbal intelligence). Prepare presnetation and provide: – 10 items assessing the construct

– Which response type would be best for your test?

– Which method of determining reliability would be most adequate for your test? (justify your claim)

– Describe three possible studies that might examine the validity of your test – base on the psychological literature, scientific articles, studies reports.

-more details can be found on the course page-

Psychological constructs to choose

1. Sensation Seeking is a trait defined by the search for experiences and feelings, that are varied, new, complex and intense, and by the readiness of experiencing those physical, social, legal, and financial experiences.

2. Altruism is a concern for the welfare of others. Altruism is the opposite of selfishness. It can be distinguished from feelings of duty and loyalty. Altruism is a motivation to provide something of value to a party who must be anyone but the self, while duty focuses on a moral obligation towards a specific individual (for example, a god, a king), or collective (for example, a government). Pure altruism consists of sacrificing something for someone other than the self (e.g. sacrificing time, energy or possessions) with no expectation of any compensation or benefits, either direct, or indirect (for instance from recognition of the giving).

• 3. Math anxiety is an emotional reaction to mathematics based on a past unpleasant experience which harms future learning. A good experience learning mathematics can overcome these past feelings and success and future achievement in math can be attained.

4. Optimism is a mental attitude that interprets situations and events as being best (optimized), meaning that in some way for factors that may not be fully comprehended, the present moment is in an optimum state. The concept is typically extended to include the attitude of hope for future conditions unfolding as optimal as well. This understanding leads to a state of mind that believes everything is as it should be, and that the future will be as well.

• 5. Verbal intelligence is the ability to analyze information and solve problems using language-based reasoning. It is associated with the ability to listen to and recall spoken information; understanding the meaning of written or spoken information; solving language based problems of a literary, logical, or social type; the ability to perform complex language-based analysis.

Grading

POINTS GRADE

0-35 2

36-44 3

45-50 3,5

51-54 4

55-58 4,5

59-60 5

Book

Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing. Principles and applications, (6ed). Upper Saddle River, NJ.: Prentice-Hall, Inc.

1. Introduction Course description, Definition of psychological test, Item format 2. Psychometrics basic concepts Basic aspects of psychological testing - what is it, why we are using it and how

we are using it? Types of test, Standardization. • Chapter 1: Tests and Measurements - Murphy, K. R. & Davidshofer, C. O.

(2005). Psychological testing. Principles and applications, (6ed). Upper Saddle River, NJ.: Prentice-Hall, Inc.

3. Statistics – refresher Basic statistic - variable types, distributions, probabilities, sampling • Chapter 4. Basic Concepts in Measurement and Statistics - Murphy, K. R.

& Davidshofer, C. O. (2005). Psychological testing. Principles and applications.

Course schedule

4. Reliability - True score theory, methods of determining reliability: test-retest, alternate form

• Chapter 6. Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing. Principles and applications.

5. Reliability - Methods of determining reliability: internal consistency

• Chapter 7. Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing. Principles and applications.

6. Norms

Normalization, transformation of raw scores, types of normalized scales, criterion reference.

• Chapter 5. Scales, Transformations, and Norms - Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing. Principles and applications.

Course schedule

Course schedule

7. Midterm Exam

November 12th 2015

8. Test construction 1 Strategies of test development. • Chapter 11. The Process of Test Development - Murphy, K. R. &

Davidshofer, C. O. (2005). Psychological testing. Principles and applications.

9. Test construction 2

Item sampling, Structure exploration and confirmation, Item analysis: difficulty and discrimination, New approaches to item analysis - Item Response Theory (IRT).

• Chapter 10. Item Analysis - Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing. Principles and applications. (p.202-225)

• Chapter 11. The Process of Test Development - Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing. Principles and applications.

Course schedule

10. Validity

Definition of validity, content validity, examples • Chapter 7. Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing.

Principles and applications.

11. Validity

Criterion-related validity, examples, construcs validity, examples • Chapter 8. Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing.

Principles and applications.

12. Diagnosis

• Aspects of the psychological diagnosis. Interpretation of test scores.

• Chapter 2. and 3. Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing. Principles and applications. (p.20-51)

Course schedule

13. Testing in a Cross-Cultural Context - Tests in different cultures, tests adaptation process

• Chapter 11. Testing in a Cross-Cultural Context Domino, G., Domino M.L. (2006). Psychological Testing, An Introduction. Cambridge University Press.

Ethical principles in testing - Basic ethical issues in constructing and utilizing psychological tests.

• APPENDIX B. Ethical Principles of Psychologists and Code of Conduct - Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing. Principles and ap

Course schedule

Course schedule

14. Specific tests Tests from various areas of psychology and their specific features, examples of personality, intelligence, neuropsychological, affect tests

• Chapter s 13, 14, 17 . Murphy, K. R. & Davidshofer, C. O. (2005). Psychological testing. Principles and applications.

plications.

Introduction to Item Response Theory

15. Presentations of extra credit’s work January 21st

Short test…

Tests items examples

1. Describe briefly what a psychological test is?

Open-ended / essay

• What could be measured: knowledge, opinion.

• Easy to construct

• Encourages more appropriate study habits

• Measures higher-order outcomes (i.e., analysis, synthesis, or evaluation goals), creative thinking, writing ability

Open-ended / essay - Cons

• Hard to score

• Can yield great variety of responses

• Not efficient to test large bodies of content

• If you give the choice of three or four essay options, you can find out what people know, but not what they don't know

2. Which type of coffee do you like best?

a. black

b. white

c. latte

d. cappuccino

e. don’t like coffee

Multiple Choice

• What could be measured: abilities, knowledge personality, prefernces.

• Which scale of measure is used?

• More answer options (4-5) reduce the chance of guessing that an item is correct

• Difficult to write four or five reasonable choices

3. Knowledge about psychological tests is important in psychological practice.

• Strongly disagree

• Disagree

• Neither agree nor disagree

• Agree

• Strongly agree

Likert scale

• What can be measured: personality, all constructs assessed with self-report methods.

• Which scale of measure is used?

• Involved in research that employs questionnaires.

• Named after its inventor, Rensis Likert.

• The range captures the intensity of respondents feelings for a given item.

4. Mark anywhere along the line to show how much do you like to wake up at 7 am?

It’s horrible It’s terrific

Graphic rating scale

• What can be measured: prefernces.

• Which scale of measure is used?

• Respones are converted to numerical form.

• Also: semantic differential, e.g. hot-cold.

5. Complete:

Toe is to Foot as Finger is to ______?

Gas is to Car as Wood is to ______?

Supplied Response

• What can be measured: knowledge and fact outcomes, terminology, formulas.

• Which scale of measure is used?

• Chances of guessing reduced.

6. Which object should be next?

Multiple Choice

• With correct and incorrect anwers.

• What can be measured: knowledge, cognitive performance, abilities.

• Which scale of measure is used?

7. Mark whether it is True (T) or False (F) for you:

Things have been going very well for me lately. T F

True/false

• What can be measured: preferences (e.g. personality), cognitive performance (e.g. intelligence).

• Can present many items at once

• Easy to score

• Increase the chance of guessing

8. Mark whether you like (L), dislike (D), or are unsure (?) about the following activity:

Partying with friends. D L ?

Muliple Choice

• Sometimes an even-point scale is used, where the middle option is not available.

• This is sometimes called a "forced choice" method

Psychological test - definition

What is a psychological test

• „An evaluative device or procedure in which a sample of an examinee’s behavior in a specified domain is obtained and subsequently evaluated and scored using a standardized process” AERA et al. 1999

• „Test – objective and standardized measure of a sample of behaviour” Anastasi (1988)

• „A psychological test is a systematic procedure for obtaining samples of behaviors, relevant to cognitive or affective functioning, and for scoring and evaluating those samples according to standards” Urbina (2004)

What is a psychological test

• „An evaluative device or procedure in which a sample of an examinee’s behavior in a specified domain is obtained and subsequently evaluated and scored

” AERA et al. 1999

• „Test – objective and measure of a sample of behaviour” Anastasi (1988)

• „A psychological test is a systematic procedure for obtaining samples of behaviors, relevant to cognitive or affective functioning, and for scoring and evaluating those samples ” Urbina (2004)

What is a psychological test

• „An in which a sample of an examinee’s behavior in a specified domain is obtained and subsequently evaluated and scored using a standardized process” AERA et al. 1999

• „Test – objective and standardized of a sample of behaviour” Anastasi (1988)

• „A psychological test is a for obtaining samples of behaviors, relevant to cognitive or affective functioning, and for scoring and evaluating those samples according to standards” Urbina (2004)

What is a psychological test

• „An evaluative device or procedure in which a sample of an examinee’s behavior in a specified domain is obtained and subsequently using a standardized process” AERA et al. 1999

• „Test – and standardized measure of a sample of behaviour” Anastasi (1988)

• „A psychological test is a systematic procedure for obtaining samples of behaviors, relevant to cognitive or affective functioning, and for those samples according to standards” Urbina (2004)

What is a psychological test

– EXPLANATION → planning, uniformity and thoroughness

– RATIONALE → objective and fair to use. • No mater who is administrating, scoring or interpreting results.

– EXPLANATION → small (but significant) subsets of a larger whole.

– RATIONALE → efficiency • Time, resources and cognitive abilities (of test taker and administrator) are limited.

What is a psychological test

– EXPLANATION → way of assessing test results (applying common yardstick:)

– RATIONALE → providing meaning to the test results • criteria, comparison between people

– EXPLANATION → some sort of categorical or numerical system of understanding test results based on preestablished rules.

– RATIONALE → providing base for any sort of inference

What is a psychological test