Technology Assessment

Performance- and Knowledge-Based Items

The 21st Century Skills Assessment uses a psychometrically validated blend of interactive, performance-based questions that allow students to authentically perform complex tasks in simulated applications, and multiple choice, knowledge-based questions.

Reports that Inform Instruction

All 21st Century Skills Assessment reports provide proficiency data across four levels (Below Basic, Basic, Proficient, Advanced) not only for overall proficiency, but also for each of the NETS-S 2007 strands, and for each standard measured by the project.

Aligned to ISTE NETS-S 2007 Refreshed Standards

The assessment is aligned to all 24 standards in the following six strands:

  • Students gathered around a laptopCreativity and Innovation
  • Communication and Collaboration
  • Research and Information Fluency
  • Critical Thinking, Problem Solving and Decision Making
  • Digital Citizenship
  • Technology Operations and Concepts

View the ISTE Standards

 

Contact Information

  • Holly Walsh – RIDE
    (401-222-8457
    holly.walsh@ride.ri.gov
  • Martha Ketsdever – Learning.com
    (800) 580-4640 x3214
    mketsdever@learning.com
  • Mary Huff - Learning.com
    (800) 580-4640 x4565
    mhuff@learning.com

Rhode Island 21st Century Skills Assessment Testing Windows

Students take the 21st Century Skills Assessment one time during the 8th grade. The majority of districts give the assessment in two class periods during computer class minimizing the impact on the rest of the school population. Other arrangements are made when a formal computer class is not available to students. With that understanding and to accommodate for unique situations, there are two testing windows, one in December to accommodate semester one students and one in May to accommodate semester two students. Schools that have a year-long computer class will assess their students in May.

2013/14 School Year

  • Nov. 25-Dec 24, 2013
  • April 28-May 23, 2014

RI Training Opportunities

Face to Face training sessions are offered each fall.  
Training session are designed for those who administer, proctoring or coordinating the 21st Century Skills Assessment for their 8th grade students.  Participants learn details about the assessment, proctoring, setting up classes and reporting. Please bring class lists to the training session.

Location: 
Lippitt School
30 Almy St.,
Warwick, RI 02886
(driving directions)

Fall 2013 Training Session Dates:
Wednesday, November 6, 2013

NOTE:  You may use either an existing eRIDE user account to login or create a workshop registration account by clicking on the links below.

Online webinars are also available throughout the year.  Select "Delivering Online Assessment".


A moment to reflect Entrance Survey

Technical Requirements

  • Internet Connection
  • Windows, Mac OS, or Linux
  • Browser: Firefox 3.6 or later, Internet Explorer 7 or later, Safari 3 or later
  • Some content requires Adobe Flash Player 9 or later and Adobe Reader 7.0 or higher
  • Some content requires speakers or headphones

21st Century Skills Assessment

(Aligned to the 2007 ISTE Refreshed NETS Standards)

Eighth Grade Technology Survey Results

TechLiteracy Assessment

(Aligned to early version of ISTE NETS Standards)

21st Century Skills Assessment Frequently Asked Questions

  • What is the address for the Rhode Island Resource page?
    http://www.learning.com/resources/ri/

  • I am not getting information about the new 8th Grade TechLiteracy Assessment. How can I be added to the contact list?
    Send your request to be added to the 8TECHLIT-L list to Stacy Mello.

Assessment Questions

  • Where do my students go to take the assessment?
    http://www.learning.com
      
  • Should my students take the sample assessment?
    It is highly recommended that students take a "practice test" for two important reasons.
    •  To prepare students for this type of online testing
    •  To identify technical difficulties well in advance of the testing windows
     
  • What is the address for the 8 item sample assessment?
    http://www.learning.com/21st-century-skills-assessment/samples/middle-school/  
    The first 8 windows or slides of the Sample Assessment instruct the test taker on how to navigate and what to expect on the assessment. Be patient and wait approximately 10-15 seconds on the eighth slide. A second "Start" button will appear to start the Sample Assessment.

  • Is it best for a student to answer a question or to skip it?
    It is best to answer the question. The student will be penalized if they skip a question.

  • Why is the “right click” not working during a performance based question on the assessment?
    The right click is disabled because the options would be different in different applications. Students should use the toolbars to complete the task.

  • What is the cut off score for a student to be considered Technology Literate?
    300

  • Can a student get partial credit in the performance items?
    No. They receive full credit or no credit for a multi step item. Students should pay attention to detail.

  • What accommodations can be made for ELL students and students with IEPs on the assessment?
    Scenario:
    Whole class of ELL students or students with IEP modifications:
    The test administrator may turn off the randomization of the test items and read the questions to the whole group of students. The test administrator should not point to the screen or give other cues to the students that would influence test scores.

    Scenario:
    Individual ELL student or student with IEP modifications:
    The test administrator may read the test questions to the individual student. Test administrator may determine whether to administer the test to the individual student in a different location or quietly off to the side so as not to disturb the other students in the class. The test administrator should not point to the screen or give other cues to the students that would influence test scores. The randomization function can be turned on and off for a whole "class" within the Learning.com management system but not individual students within that "class".

Roster Questions

  • Is there a way to print only the ID cards for those taking the test in Dec. instead of the entire list? I tried clicking in front of their names and generating a report but that didn't seem to work.
    Unfortunately, no. You will have to print out the cards and sort them out.

  • What do I do if a student is sitting in front of me but they do not show up on my roster?
    Email Mary Huff at Learning.com. Include information such as the student’s name and State Assigned Student ID (SASID) and the school in which the student should be enrolled.

  • I know how to add a student but what if a student switched schools and still appears on my list?
    You don't have to do anything. Simply ignore the name. Leaving the student on your list will not affect scores for your school.

  • What do I do if a student transfers into the district and it says “the student ID is already in use” when I try to enroll that student?
    Email Mary Huff at Learning.com. Include the student’s name, the school where the student is coming from, and the new school in which they should be enrolled.

  • What should I do if a student moves out of state?
    You can simply ignore a student name on your roster who has moved to another state. This will not affect your scores in any way.

  • Are there any students that do not have to take the assessment in 8th grade?
    Yes, the following students do NOT have to take the TechLiteracy Assessment:
    •  Students who take the Alternate Assessment
    •  ELL students who are “new to the country”, that is any student who enrolled in a US school on or after Oct. 1and are exempt from taking the NECAP Reading/Writing assessments
    •  Students who enroll in the school after the school's designated testing window

  • Why do my ELL students have to take the assessment?
    ELL students who are "new to the country" are exempt from taking this assessment. See above.

    All other ELL students must take the assessment. Since the TechLiteracy Assessment is not a reading assessment, the majority of all ELL students are not exempt from this assessment just as they are not exempt from the NECAP Math and NECAP Science assessments regardless of when they entered the US.

  • Do students that take the Alternative Assessment have to take the 8th Grade TechLiteracy Assessment?
    No. See above.

  • What should I do if a student is out due to an extended illness?
    Both testing windows are three week opportunities to assess students. Hopefully the student will be back to school within the two weeks. The alternative is to test the student during the second window in the spring. Best effort prevails in this situation. Use the Proctor Notes area within the Learning.com management system to explain extenuating circumstances.

Scoring Questions

  • What is the significance of the 4 levels of proficiency?
    Below Basic:
    The student is lacking basic skills for their grade level.
    Basic: The student has the skills that are appropriate for entering their grade level, but is not prepared for moving to the next grade level (6th grade or 9th grade).
    Proficient: The student is ready to advance to the next grade level.
    Advanced: The student has skills that are beyond the next grade level.

  • How was the proficiency standard set?
    The proficiency benchmarks for students were created using the 24 ISTE NETS-S 2007 performance indicators and NETS-S profiles. When standards serve as educational goals, they often need to be revised into statements of achievement before they can be measured. This requires breaking standards down into component parts and linking them to specific actions.

    Items were then tested with students in field studies in different states and among different demographic populations.

    The prevalent standards, items, and student performance data were then scrutinized by a panel of three co-authors of the ISTE NETS-S 2007, and four technology instruction experts with classroom, district level, and state level experience. This panel, in conjunction with expert psychometricians, examined the data and made two determinations. First, they confirmed that 21st Century Skills Assessment does effectively measure student skills and knowledge in technology. Second, they determined where the bar for proficiency in technology literacy should lie for both the elementary and middle school national student populations. This determined the Proficiency Standard used in 21st century Skills Assessment. The panel then determined the cut-marks between below basic and basic, and between proficient and advanced.

  • Was the purpose of the psychometric analysis to find distracter and poorly worded questions, and to determine the general difficulty of each question, along with how well it identified proficient students?
    The purpose of psychometric analysis is to ensure that an assessment’s scores will be reliable and valid. Psychometric analysis is critical when test results will potentially be used for high stakes decisions. Some benefits are indeed to identify unusable questions, determine the general difficulty of each question, and determine how well each question identified proficient students, for example, the analysis shows how students that answered specific questions correctly or incorrectly did on the test as a whole. Item Response Theory was used for the analysis.

  • Why is a score between 100 and 500 produced?
    The score begins at 100 to prevent confusion with percentage scores of 0 to 100%. It extends from 100 to 500 to prevent confusion resulting from trying to draw inaccurate relationships with other, unrelated assessments, and to provide 4 levels of proficiency.

  • Why is the Proficiency Standard set at 300?
    The purpose of the scale score is to provide a readable, consistent method of comparing scores and showing proficiency that does not change between tests, even as the number of items required to be correct will change for each test based upon the psychometric analysis of the items.
    300 was chosen to enhance readability of scores.

  • Are the first right answers worth more points than the last right answers?
    Please note that the scaled scores indicate proficiency and are the only scores that are comparable to each other from one test to the next, not point values per item. 21st Century Skills Assessment uses a fixed cut mark at 300, or 200 points, to indicate proficiency. For each new version of the assessment (the pre and post test for an upcoming year) the combination of items is analyzed by psychometricians using Item Response Theory to determine the test characteristic curve which sets the new cutoff. The scaled passing score will always be 300; however, depending on the number of correct answers needed to obtain that score, the number of points each question is worth will vary from form to form, as will the percentage of answers that need to be correct before a student is minimally proficient.

    This means that if the combination of items were shown to have a higher level of difficulty than before, less items will need to be correct to show. Or, if the analysis has determined that the items were less difficult than before, more items must be correct to achieve proficiency. Points per item are determined by psychometric analysis of each new test and usually vary between different cut-marks.

  • If each question is weighted equally, how much is each question worth?
    Please note that the scaled scores indicate proficiency and are the only scores that are comparable to each other from one test to the next, not point values per item. While the number of points is calculated without numerical weighting, each new assessment has questions of varying difficulty. The combination of items for each new test is analyzed by psychometricians using Item Response Theory to determine the test characteristic curve which sets the new cutoff. The scaled passing score will always be 300; however, depending on the number of correct answers needed to obtain that score, the number of points each question is worth will vary from form to form, as will the percentage of answers that need to be correct before a student is minimally proficient.

    This means that if the combination of items were shown to have a higher level of difficulty than before, less items will need to be correct to show proficiency. Or, if the analysis has determined that the items were less difficult than before, more items must be correct to achieve proficiency. Points per item are determined by psychometric analysis of each new test and usually vary between different cut-marks.

Site Documents

Documents on this site require the use of the following programs:

DOC - Microsoft Word

PDF - Acrobat Reader

PPT - Microsoft PowerPoint

XLS - Microsoft Excel