In summer 2022, an "alternative access plan" was filed with and approved by Technology Accessibility Review Committee. This will be reviewed again in Summer 2023.

See https://answers.uillinois.edu/114833 for more information about Gradescope and support.

The content of that alternative access plan is shared here.

Description of Gradescope

This plan involves the use of Gradescope, https://gradescope.com, an online tool for courses to assist with managing homework/quiz assignment distribution, collection, and grading. Whether the students upload their own PDFs, or course staff scan written in documents in, Gradescope provides an administrative tool for course staff to manage the grading and re-grading of assignments in a SaaS web platform.

Gradescope runs independently at https://gradescope.com/ with SSO enabled for University of Illinois at Urbana-Champaign logins. The intention is that Gradescope will be LTI 1.3 integrated with Canvas, offering both an access point to service discovery and setup, along with advanced integration features like roster and gradebook passing between Gradescope and Canvas.

Gradescope is run by TurnItIn, the same company that does our plagiarism detection for Canvas via their flagship product, TurnItIn.

Scope of usage

It is the intention that Gradescope is available to any University of Illinois at Urbana-Champaign course, regardless of department or college. The license and administrative support is there; the integration with Canvas will help make it easy for new courses to use Gradescope.

Gradescope has been used in the Grainger College of Engineering as a solution for online homework management in large scale since 2020 and piloted by CS courses prior to that. Gradescope was used for 250 courses in Spring 2022, primarily in Engineering but also in Math and Statistics.

Gradescope was purchased by Engineering via a Sole Source Purchase valid through 2026. The license is at the unlimited use tier so there are no barriers to using it outside of Engineering. Cost sharing or recovery can be explored between different units if usage changes, but currently the Engineering demand would carry the rest of the campus.

Access barriers in the product

The accessibility review of Gradescope identified three main areas of accessibility concern that will be addressed in this document. Thanks to Technology Services staff who performed the audit over late 2021, early 2022. Gradescope has been receptive to feedback and the product is evolving to be more accessible because of this process.

B1: Visual nature of Gradescope

Gradescope is designed to take images of student work and make it available for digital distributed grading. The majority of the student content to be evaluated is submitted as PDF representations of images of hand-written work. (Gradescope has an iOS and Android app to assist with imaging and uploading.) Gradescope also supports some online data entry that isn’t PDF, like filling forms or submitting textareas, but mainly Gradescope is used with images.

As we know, untreated content inside images is very difficult for screen reading or other assistive technology. Even if the interface to manipulate those images was fully supportive of low vision or keyboard mobility (see other barriers), the content inside the images may not be comprehendible even with assistive technologies. This is the nature of the product, and is similar to grading hand-written paper documents.

B2: Keyboard accessibility for “date picker” interface in instructor screens

The accessibility review of Gradescope done in January 2022 identified one “show stopper” level of severity, involving keyboard navigation in a calendar “date picker” interface in instructor assignment screens. We identified poor usability with screen readers or keyboard navigation with respect to content on the screen not accurately representing user choices and updating information on the screen in a way that screen readers did not report the change.

Gradescope was reactive to this feedback and has since made a change. This should be formally reevaluated in the future, but a quick re-evaluation showed the access severity of the bug was reduced. We appreciate Gradescope for making rapid changes and working with Illinois to evaluate and confirm the changes addressed our needs.

B3: Other minor known accessibility concerns in student and instructor interfaces

The accessibility review of Gradescope done in January 2022 identified a number of smaller accessibility infractions of lessor priority than the show stopper mentioned in B2. Many of these accessibility concerns were in instructor screens. Gradescope acknowledged their previous accessibility focus had been on student-facing interfaces and their instructor side has known limitations and in need of revision.

Gradescope disclosed a summer 2022 planned extensive accessibility audit, followed by a period of Gradescope engineers making any necessary remediations in the fall, and culminating with an updated VPAT by the end of 2022. We anticipate most of the lower priority accessibility issues, which we shared with Gradescope, to be addressed in this next round of remediations. We plan for a spring 2023 accessibility audit, and will update this plan based on the evolution of the platform.

Alternative methods of access

Instructors or course staff with vision disabilities may not be well served by Gradescope.

Alternative #1: not using Gradescope. Other methods of assignment structuring and collection may provide accessibility data to graders with accessibility tools. Examples include assignment collection via LMS’s like Canvas, Moodle, PrairieLearn, etc.

Alternative #2: delegate Gradescope operation to course staff who can process the data in the way Gradescope presents it. In large courses, where the use of tools like Gradescope increase efficiencies and those courses often have a team of course staff(consisting of instructors, teaching assistants, graders, undergrad course assistants, etc., responsibilities can be shared between course staff for collective success.

Alternative #3: courses may require certain types of PDF uploads which have more accessible data inside them. This might be difficult to train students to produce and to enforce in the software, but it shifts the responsibility of accessible documents into the course and students who create them and away from the digital tool that helps manage them. This makes Gradescope similar to other document management tools like Canvas, Box.com, shared folders, etc.

Alternative #4: wait for future updates. Gradescope has proven to be receptive and responsive to our accessibility concerns and the product is evolving. If there is an accessibility challenge in the current software, that should be reported to service managers so that we can track the issue with Gradescope. Future releases may address these concerns, so revisiting the use of Gradescope in a later semester may resolve any outstanding accessibility concerns.

Communication:

The known accessibility barriers and their remediations will be tracked and shared in this knowledgebase entry. That page will be searchable/findable in the knowledgebase. It will be updated with known issues and their current status.

When the integration is with Canvas is made generally available, the Gradescope service managers (currently Dave Mussulman from Engineering, potentially adding more) will email all of the known UIUC Gradescope instructors to let them know about this change. We’ll include the knowledgebase pages for support and accessibility. If desired, we can repeat this process if more frequent reminders or access disclaimers are required.