SPE 501 Class 11, Online Module 6

Do Now • Download the “Score reports” file from the website Choose a score report based on your grade level and/or interest.

• Look over the report and try to find any of the following. Label them on the score report.• Raw score• Standard/scaled score• Confidence intervals or score range• Standard error of measure• Percentile/Percentile rank• Grade/age equivalent

• Highlight other scores/words you encounter that you are not familiar with.

Objectives

RWBAT…• Define terms related to standardized assessment• Read a standardized assessment score report to gain

information about a student• Translate a score report into student and parent-friendly

language• Describe how accommodations are used in district/state

assessments

What are standardized assessments?

• Also known as Formal Assessments• Designed by people who are experts in test construction and distributed by companies

• Test administration kept consistent • Objective• Strict standards for scoring• Often multiple choice

Types of Formal assessments• Aptitude—used to predict performance in the future (ex.

ACT, IQ tests)• Achievement—used to evaluate academic skills (ex.

ISAT)

Reasons for administering achievement tests:• Determine eligibility for special education• Identify students in need of support• Compare students across schools, districts, etc.• Evaluate schools, teachers, curriculum, etc.

Norm vs. Criterion Referenced

• Norm-referenced assessments compare students with others who took the test. This group is called the norm or reference group.

• Criterion-referenced assessments compare students with a pre-determined set of skills. These tests do not compare students with each other but focus on describing a individual student’s academic performance.

Reporting Scores

Ways to report scores on standardized tests:• Raw score• Standard/scale score• Percentile• Stanine• Confidence intervals• Cut scores

Helpful Terms• Mean

• average score (*used the most)

• Median• the middle score

• Mode• the score the occurs the most often

• Standard deviation• measure of variability, shows the average deviation of

scores from the mean

• Example: Find the mean, median, and mode• 60, 78, 80, 80, 85, 92, 96

Example

Given scores of 60, 78, 80, 80, 85, 92, 96

Mean = 81.6

Median = 80

Mode = 80

Standard Deviation = 11.6

The Normal Curve

• The “normal distribution” is a distribution of scores that are distributed evenly around the mean score. It describes many naturally occurring physical and social phenomena. Many scores will cluster around the mean. Fewer scores fall at the end points or at the higher and lower occurrences.

• Sometimes referred to as a “bell curve”

The Normal Distribution

Properties of the Normal Distribution• The mean is the midpoint of the distribution.• The mean, the median, and the mode are all at the same

point.• 68% of all scores are located in the area of plus or minus

one standard deviation from the mean.• About 16% of the scores are higher or lower than one

standard deviation above the mean.• Only 2% of the scores are more than two standard

deviations above or below the mean.

• Example: If you scored 700 on the SAT math test, you did well, because only about 2% of the people who take the test make that score.

Raw Score

• The number of questions answered correctly • Always reference the number of items (questions) when interpreting the raw score

• Never try to interpret raw scores across subtests or between tests because the subtests & tests differ in difficulty and in the number of questions asked

• Example: A CPS reading benchmark test gives raw scores out of 45

Standard Score

• Also sometimes called a scaled score• Way of converting raw scores to a number that makes more sense (i.e. does not take into account the number of items)

• Uses the mean and standard deviation to convert scores.

• T-scores and Z-scores are each types of standard score.

• Many psychological tests use standard scores with a mean of 100 and a SD of 15 (ex. IQ tests—100 is average)

Percentile Rank• These scores show the percentage of students in a

normed sample who score at, below, or above a particular raw score.

• A student scoring better than three-fourths of the students in the normed sample would be at the 75th percentile or have a percentile rank of 75. A percentile rank of 50 means the student scored as well, or better than 50% of the students in the normed sample.

• Different from % correct. Percents are only used with raw scores.

Stanine

• Divides the normal curve into 9 sections. • Range from stanine 1 (lowest) to stanine 9 (highest)

Grade Equivalent Scores (GE)• GE scores are obtained from separately normed samples

for each grade level.• They are reported in numbers such as 3.8, 5.4, 11.2. The

whole number equals the grade and the decimal the month.

• GE scores are often misleading and misinterpreted. For example, if a 7th grade student has a GE of 10, should she be promoted to the 10th grade? • No! A GE score of 10 does not mean that the student can do 10th grade

level work. This score means that the 7th grade student scored similar to what a 10th grade student would have scored on the 7th grade test. The high scores may simply represent superior mastery of material at the 7th grade level.

Confidence Interval• This is a “standard error band” within which a student’s score can fall. It is sometimes referred to as a score range on assessment reports.

• It is based on the standard error of measurement (SEM), and is an estimation made by test developers who take into consideration how much a student’s score would vary over repeated testing sessions.

• You might see this as a shaded area surrounding a graph of scores

Cut Scores

• Some tests use cut scores to divide students into classification groups (e.g. meeting standards, below standards, etc.)

• Cut scores are normally use the standard/scale score instead of percentiles

• Examples:• http://www.isbe.state.il.us/assessment/htmls/psae_cut_points.htm•

http://www.isbe.state.il.us/assessment/pdfs/2013/isat/cut-scores13.pdf

RIT Scores• Part of the NWEA/MAPS test• Basically a standard score used by the test makers• Grade-independent: a score of 235 is the same no matter

what grade so a 4th grade student and a 5th grade student both with a score of 235 are on the same level

• Equal intervals between scores: a student who goes from 150 to 170 has made the same amount of growth as a student who goes from 170 to 190.

• Presented in Descartes Ranges: way of using cut scores to put students in different groups for instructional planning purposes. Descartes ranges connected to criterion information.

Standardized Assessment Reports

• Review your score report: • Any new terms/scores you can identify?• Lingering questions?

Communicating Assessment Results with Parents and Students

Tyesha, a 7th grade student in your class, just took a standardized assessment and received the score report below. Her mother emails you asking you to explain what the different numbers mean.

Tyesha’s Standardized Test Scores for Math

 

Standard score 182

Grade equivalent 5.5

Percentile Rank (district normed) 43

Stanine 5

Standard score 182

Her score on the test converted to some scale used by the test makers. That does not mean that she got 182 questions correct. (It would help to show the parent the scale used for the test – what does 182 mean?)

Grade equivalent 5.5

Tyesha scored as well as a fifth grader in the fourth month of school would have scored on this 7th grade test. It doesn’t mean she’s at the 5th grade level in math but it does mean she is a bit below where she should be.

Percentile Rank (district normed) 43

Tyesha scored better than 43% of the students who took the test. She is slightly below the average.

Stanine 5

The stanine divides scores into 9 groups. Five is the middle stanine so Tyesha’s score is in the middle range.

What can we say overall?

Communicating Assessment Results with Parents and Students• Avoid using statistical terms – standard deviation, mean,

etc.• Avoid terms that could be triggers for parents – bell curve,

“normal” curve• Keep things simple – ex. You don’t have to explain how

we get a standard score• Be positive but also honest!• Talk about next steps – what do the results tell you about

where you (and the student) need to go

Evaluating Standardized TestsAll forms of assessment should have evidence of reliability and validity.• Reliability refers to the consistency of the test results, or

how likely it is that a person will get the same score each time the test is taken.

• Validity refers to the adequacy and appropriateness of interpretations based on the test results

It is also important to consider who is the in the norming group. • How do the characteristics (i.e. race, gender, geographic

region, disability) of the population used as the norming sample compare with your population of students

Reliability

How reliability is judged (done by test manufacturers):• Test-retest: give a the same test at two different times

• Alternate form: give two versions of the test• Split-half: break test into two sections • Internal consistency – statistical measure of reliability

• Interrater: have two different people score a test (only works for tests that involve ratings of performance, i.e. not multiple choice)

Validity• Content

• How well a test’s items reflect a particular body of knowledge and skill• Ex. Does the math section on the 3rd grade ISAT reflect what is covered in 3rd

grade math?

• Criterion• How well a test predicts a student’s future behavior or compare to current

performance on other measures • Ex. How well does the ACT predict college success?

• Construct• How well a test measures some internal attribute of a person • Ex. If a test is measuring student verbal reasoning, how is that term

defined/measured?

• Consequence• What decisions are being made based on the test results• Do the test results accomplish their intended purposes and avoid unintended

consequences?

Validity of standardized assessments for students with disabilities

• Issues with construct-irrelevant variance - Is the test measuring something other than it intends? For example, if a math test has a heavy reading load, does it accurately measure the math ability of a student with a severe reading disability?

• Use of accommodations – Do the accommodations being used impact the test in such a way that is no longer measuring what they intend to?

• Consequential validity – Are the test results being used appropriately for students with disabilities? For example, if a student is being retained based on ISAT results, is that appropriate for a student with a severe learning disability?

• Predictive power – If a test is designed to predict future performance (e.g. the SAT and college grades) does it still hold true for SWD?

Accommodations for District/State Assessments

• Remember the 5 types:• Presentation • Response • Setting • Timing • Scheduling

• Located in Section 10c of the IEP

Accommodations for District/State Assessments

• Not always the same as accommodations used for classroom assessments. Different accommodations are allowed for different tests.

• For a student to receive accommodations on standardized tests they must be listed in the IEP and have been used previously by the student for classroom assessments. (Ex. You can’t create new accommodations just for the ISAT if a student hasn’t used them in classroom assessments.)

Allowable Accommodations• Each district/state test has accommodations that they

allow to be used by students with disabilities. They also may have procedures for schools to apply and get approval for individual students to use accommodations on an assessment.

• The school counselor or case manager usually handles these accommodations.

• http://www.isbe.state.il.us/assessment/pdfs/2013/IEP_504_guidance.pdf

Accommodations on the NWEA• NWEA lays out guideline for accommodations:

http://www.nwea.org/node/4615 • Not as strict as ISAT and PSAE since they are not used

for high-stakes decisions at the district or state level.• Download the accommodations checklist (on the web

page above) to keep track of acc used for students

Accommodations on the ISAT• List of allowable accommodations for SWD and

ELLs(English Language Learners):• http://www.isbe.state.il.us/assessment/pdfs/

ISAT_accommodations_grid.pdf

Accommodations on the PSAE (ACT)• The ACT breaks accommodations into two types:

• ACT accommodations – if a student uses these accommodations their scores can still be reported to colleges

• State allowed accommodations – these are more intensive acc that a student may use if the test is going to be used for state accountability purposes (like the PSAE in Illinois). If a student uses state allowed acc their ACT scores cannot be reported to colleges.

• Differences between ACT and State Allowed acc: • http://www.isbe.state.il.us/assessment/pdfs/2009/

ACT_Apvd_State_Alwd.pdf

• More detailed info: look at p. 1 for the difference between ACT and State allowed accommodations, p. 2 for examples, and p. 6 for specific disabilities and accommodations allowed• http://www.isbe.state.il.us/assessment/pdfs/2013/psae/accom-procedu

res13.pdf

Illinois Alternate Assessment• For students with most severe cognitive disabilities for

whom taking the ISAT or PSAE even with accommodations is not appropriate

• Determined by the IEP team• http://www.isbe.state.il.us/assessment/pdfs/2012/iaa/

partic_guidelines_pres.pdf

• Administered in a one-on-one setting. Student given prompts (questions) and evaluated on the amount of support they need to respond to the prompt• http://www.isbe.state.il.us/assessment/pdfs/2013/iaa/

test_admin_pres.pdf

For more information• ISBE info on standardized tests:

http://www.isbe.state.il.us/assessment/ • Information about tests for parents (good review of info):

http://www.pbs.org/wgbh/pages/frontline/shows/schools/etc/guide.html

Next steps…

Module 6 activity:• Download module 6 activity from the website. Choose

the activity that corresponds with your teaching assignment (primary or secondary). You do not need to do both!

• Complete module 6 activity by typing directly on the document. Save and upload to your google folder by 11:59pm on 3/24.

Course feedback:• Click on the link for the google form to give feedback on

the course

Finishing the course• This is the last module!

• Remaining assignments:• IEP Assessment Report Review – due by 11:59pm on 3/24

• Adapting assessments assignment – Due by 11:59pm on 3/29!