An accessibility analysis of web pages from every college and university in the United States (nearly 4,000 higher education institutions).
The 4k project considers the automatically detectable errors that can be identified by WAVE.
Annual Reports
- Higher Ed in 4k: Nov 2023 report
- Higher Ed in 4k: Nov 2020 report (current report)
- Higher Ed in 4k: Nov 2019 report
Purpose
The main purpose of the 4k Project is to document the progress of United States Higher Education Institutions’ web accessibility.
Interpreting the results
All automated tools, including WAVE, have limitations—only 25% to 35% of possible conformance failures can be automatically detected. The absence of detectable errors does not indicate that a site is accessible or compliant. Still, the data presented in this project provide a meaningful representation of the state of web accessibility in Higher Ed. “Errors” are WAVE-detected, accessibility barriers that have notable end-user impact and are likely WCAG 2 Level A/AA conformance failures.
What this isn’t
This project is not a condemnation of Higher Ed in the US, remember that .edu, .us, and .gov had the lowest number of average accessibility errors of all common-level domains (TLDs) in the WebAIM Million project. Before learning about web accessibility, some of the team members at Pope Tech worked in higher education on websites and created some of the same types of accessibility errors (perhaps some of those errors still exist on websites and made it into this study!).
Automated testing isn’t a silver bullet, automated tools can’t detect everything or even close to everything. Only a human can determine true accessibility. WAVE is a suite of tools to help in this process and not the end goal.
We believe web accessibility in higher education is important to track which is why we perform this study.
Background
The Higher Ed in 4k Project was inspired by the WebAIM Million Project. After the WebAIM Million project launched, one of our takeaways was the opportunity of projects like this to help make websites more accessible in a significant way.
With this in mind, we asked ourselves:
What would happen if we went deeper than the home page, if we focused on one group (Higher Education in the US)?
The Higher Ed in 4k Project is more of a living experiment than a one-time study.
Methods
For context, we have included the key summary points of the methodology:
- Approximately 4,000 (4k) higher education institutions from IPEDS list were included
- Up to 100 pages (all within 4 links from the main home page) were sampled from each institution
- The accessibility evaluation was performed by Pope Tech using the WAVE engine
- Automatically detectable WAVE errors (Errors & Contrast Errors) were used to calculate the average errors per page for each institution
- The institutions average errors per page were used for the analyses and rankings
- We rescanned the same pages in 2020 that were scanned in 2019
- Web pages or websites that were missing or invalid were dropped from the 2020 analyses
A more comprehensive description of the 4k Project methodology is provided in our Nov 2019 Report.
2020 Results
Overall, Higher Education has historically done better than most industries with regard to web accessibility. While many industries and business sectors have been slower to respond to improving web accessibility, higher education continues to improve. Our 4k Project documents the improvements made by higher education institutions in the United States between 2019 and 2020.
Key 4k Project findings (Overall)
- In 2020, 5,642,199 detectable errors remain (an average of 20 errors per page)
- 8.16% of pages have no detectable errors
- The 2020 results had an average of 4 fewer errors per page than the 2019 results (a 17% decline!)
- Alerts stayed nearly the same between 2019 to 2020 with an average of 27 alerts per page each year
- In 2020, 7 institutions had 0 detectable errors on the pages we sampled (+2 from 2019)
- Comparing 2019 to 2020 results, the sampled pages for 63.8% of the institutions decreased in the average errors per page
Our 4k Study results are consistent with WebAIM’s reported trends for higher education. WebAIM found that while non-higher education websites tended to get slightly more errors per page between 2019 and 2020, higher education pages improved across the same period.
Key Take Away Points
- Higher education tends to do better than other industries in web accessibility
- There was a 17% reduction in the average errors per page between 2019 and 2020.
- Just under ⅔ of the higher education institutions made progress on detectable errors on the pages we analyzed. This means, however, that over ⅓ of the institutions did not make any progress or regressed.
The “Average” Institution (2020)
The average institution in the 2020 data set had the following:
- 75 pages scanned
- 20 errors per page
- 27 alerts per page
- a user would encounter a detectable error on 1 in every 40 elements on a page
5 most common errors
| Error | Count | % of Errors |
|---|---|---|
| Very Low Contrast | 3,737,103 | 66.1% |
| Empty Link | 673,602 | 11.9% |
| Missing Alternative Text | 457,340 | 8.4% |
| Missing Form Label | 221,687 | 3.9% |
| Empty Button | 147,393 | 2.6% |
It is important to realize that if the 5 most common errors detected in this project were resolved, it would remove 93% (5.3 million) of the detectable errors from the pages we scanned.
Concurrently with this report, we released a blog about the most common errors detected in this study, as well as well as suggestions on how to fix them.
State/territory rankings
For the second year in a row, Alaska has the lowest amount of detectable accessibility errors with only 7.8 errors per page, followed by Hawaii, Wyoming, Alabama, and Montana which were all below 13.5 errors per page. While these states tend to have lower population levels, the overall state population didn’t correlate with the number of errors. Rhode Island and Pennsylvania were right at the average with 20 errors per page.
The 5 states with the highest average number of accessibility errors across their higher education institutions were Colorado, Kentucky, New Mexico, South Dakota, and Utah with over 25 errors per page each. Forty-seven of the 50 states average error counts across their higher education institutions decreased.
When we initially calculated the state-average scores, we did so using the mean (average) of all the institutions in that state. . We discovered that there were some outlier institutions with very high error per page counts that were skewing the results. Based on this, we used each state’s median institution’s errors per page as the average for the state rankings.
On our State Rankings page, you can find each state’s ranking, median, and errors per page. We provided two ranks, an overall ranking with all institutions (public and private) and a separate ranking for just public institutions.
Rankings by classifications
Using the IPED List data, and other higher education institution databases, we also were able to compare institutions based on different classifications beyond just overall rankings.
Below are some of the interesting comparisons we found.
Highest Degree Offered
| Type | 2019 Errors per page | 2020 Errors per page | % change |
|---|---|---|---|
| up to Associate degree | 24.8 | 20.0 | 19.6% decline |
| up to Bachelor’s degree | 25.1 | 20.3 | 19.1% decline |
| Post Bachelor degrees | 23.3 | 19.4 | 16.4% decline |
Private vs. public institutions
| Type | 2019 Errors per page | 2020 Errors per page | % change |
|---|---|---|---|
| Public | 20.21 | 15.4 | 23.8% decline |
| Private not-for-profit | 25.3 | 21.2 | 16.0% decline |
| Private for-profit | 30.1 | 26.2 | 12.9% decline |
Public institutions averaged 15 errors per page, and have fewer accessible errors per page than private institutions. Similarly, Private not-for-profit institutions have fewer detectable errors (with 21 errors per page) than private for-profit institutions which have 26 errors per page.
It would make sense that public institutions would be better as they have additional laws beyond the Americans with Disability Act including Section 508.
Student enrollment
| Type | 2019 Errors per page | 2020 Errors per page | % change |
|---|---|---|---|
| Enrollment under 1,000 | 28.3 | 24.1 | 14.8% decline |
| Enrollment 1,000 – 4,999 | 24.1 | 19.2 | 20.4% decline |
| Enrollment 5,000 – 9,999 | 19.0 | 15.4 | 19.3% decline |
| Enrollment 10,000 – 19,999 | 19.0 | 14.3 | 24.7% decline |
| Enrollment 20,000 and above | 14.0 | 11.2 | 20.2% decline |
There was a direct correlation between student enrollment numbers and accessibility errors per page. The more students, the fewer detectable errors.
This could be because the larger institutions have more resources and budgets. Institutions with over 20,000 students enrolled had only 11 detectable accessibility errors per page.
Land Grant
| Type | 2019 Errors per page | 2020 Errors per page | % change |
|---|---|---|---|
| Land Grant institutions | 15.4 | 13.8 | 10.4% decline |
| Non-Land Grant institutions | 24.3 | 19.9 | 18.1% decline |
Land Grant Universities had fewer detectable errors (13.8 per page) than non-land grant institutions (19.9 errors per page).
In a system
Many higher education institutions are part of a state or private school system with many intuitions in the school system.
| Type | 2019 Errors per page | 2020 Errors per page | % change |
|---|---|---|---|
| In a system | 23.1 | 17.8 | 22.9% decline |
| Not in a system | 24.6 | 20.8 | 15.4% decline |
Institutions in a school system had fewer errors per page (17.8) and resolved a higher percentage of issues between 2019-2020 (22.9% decline) than institutions not in a school system (20.8 errors per page and a 15.4% decrease in errors).
Carnegie classifications
When comparing Carnegie classifications, it is important to understand that not all institutions had a classification in the IPED List. The results only reflects those with a specified classification. Additionally, we only included a subset of the Carnegie classifications.
| Type | 2019 Errors per page | 2020 Errors per page | % change |
|---|---|---|---|
| Doctoral/Research Universities–Extensive | 15.3 | 12.4 | 19.0% decline |
| Masters Colleges and Universities I | 19.1 | 15.4 | 19.4% decline |
| Baccalaureate Colleges–Liberal Arts | 20.8 | 15.4 | 26.0% decline |
| Associates Colleges | 21.5 | 17.2 | 20.0% decline |
| Baccalaureate Colleges–General | 24.4 | 19.3 | 20.9% decline |
| Medical schools and medical centers | 27.8 | 19.8 | 28.8% decline |
| Schools of law | 30.7 | 32.1 | 4.6% increase |
Doctoral/Research Universities had the fewest accessibility errors averaging only 12 errors per page. It is interesting that the two Carnegie classifications with the highest number of detectable errors were Medical schools with 20 errors per page and Schools of Law with over 32 errors per page. Of the different Carnegie classifications, only Schools of law increased in average errors per page from 2019 to 2020 (a 4.6% increase). All the remainder classifications we evaluated had declines in detectable errors.
UCEDD vs. non UCEDD
UCEDD stands for University Centers for Excellence in Developmental Disabilities Education. The vision of the UCEDD program is, “a nation in which all Americans, including Americans with disabilities, participate fully in their communities.” There is at least one in every US state and territory housed inside a host university.
| Type | 2019 Errors per page | 2020 Errors per page | % change |
|---|---|---|---|
| UCEDD | 13.8 | 11.9 | 13.8% decline |
| Non-UCEDD | 24.3 | 19.9 | 18.1% decline |
The host UCEDD institutions have just under 12 detectable errors per page, which is 8 detectable errors per page fewer than the average higher education institution (19.9 errors per page).
Institutions who requested their 4k account
Between November 2019 and November 2020, eighty-seven higher education institutions took us up on our offer to access their 4k project account in our platform. This has allowed those higher education institutions the opportunity to review their institution’s results, fix issues, and even rescan to update results. The updated results are included on our live project page and rankings are recalculated.
Interesting and random tidbits
In 2019, we reported on some other interesting findings from our project. Below, we have included our 2020 updates on each of these different findings.
Skip Links
In 2020, we found there were 1,991 skip links that didn’t have a target across the sample of pages in our project. This means that someone went through the effort to add a skip link but then either never tested it or it was broken with a template update. The good news here is this was a substantial decline from the 6,964 skip links without targets we detected in 2019 on the same pages.
Marquee tags
In 2020, in our sample of pages, there were 46 marquee tags still around (a substantial drop from the 170 we detected in 2019).
Layout tables
We found 296,891 layout tables (in 2020) compared to 335,183 found in 2019 and 47,810 data tables (in 2020) compared to 53,715 found in 2019. A data table is classified as a data table if it is a properly structured table with the proper heading rows. Realistically, we suspect that there are relatively few true layout tables.
PDFs
We found 619,755 links to PDFs in 2020 or just over 2 per page (similar to 2019). These may or may not be accessible. But as we know from the 2019 WebAIM screen reader survey, 74% of screen reader users are either Very Likely or Somewhat Likely to encounter significant issues accessing a PDF document.
Conclusion
While there is still significant work to be done to ensure Higher Education websites are accessible to everyone, we are encouraged by how much better Higher Education websites were this year compared to last year. Collectively, they did resolve over 17% of the average errors per page. Additionally, nearly ⅔ of the institutions had improvements in their overall error/page averages. This suggests that the majority of higher education institutions are making efforts to improve web accessibility.
We are hopeful that this project and other endeavors by the Higher Education web accessibility community will continue to bring more awareness to web accessibility. are optimistic that over time we will see additional improvement.