Common Accessibility Issues of Norwegian Mobile Banking Applications | IDEA-lab Cerovac
This is a webpage for my Universal Design 2024 (UD2024) conference project where I was using parts of EN 301 549 and WCAG to check how (in)accessible are iOS and Android mobile applications from 4 prominent Norwegian banks.
Between the lines
Academic articles sometimes need additional interpretation. This page tries to provide that and add some context to the original study.
To sum it up really quickly - not a single app conforms to Web Content Accessibility Guidelines version 2.0 (or 2.1 / 2.2) - and with that also with EN 301 549 version 3.2.1. This is perhaps not something new to people that understand the requirements and it doesn't mean that apps are totally inaccessible. Conforming to standards also does not mean that apps will be usable for people with disabilities. But it can serve as a good indication and, most importantly, it can teach us to have even more focus on accessibility when we deliver products and services.
I will not share the names of the banks, that is not the point. The point is that some people can't really use their banking applications, or at least experience barriers when trying to do so. The point of the study is to advocate for accessibility and universal design and to point out that we need better accessibility maturity in organizations!
It's quite obvious that designers, developers and content providers considered accessibility partially, but there are components and content in applications that was obviously not created or at least tested properly and we need more awareness to improve that.
Results with additional comments
The most accessible bank failed 9 out of 44 tested WCAG (and EN 301 549) success criteria.
Summarized results
All four tested bank applications failed the following WCAG success criteria:
- 1.1.1 - Non-text Content - Level A
- 1.3.1 - Info and Relationships - Level A
- 2.5.3 - Label in Name - Level A
- 4.1.2 - Name, Role, Value - Level A
- 4.1.3 - Status Messages - Level AA
Basically, this means that users of assistive technologies like screen readers and voice control users will face barriers, or in some cases they will be totally blocked and will need the help of others to complete tasks.
Two out of four banks cannot be fully operated by keyboard alone (failing WCAG 2.1.1 Keyboard - Level A or other keyboard related success criteria).
Out of 44 selected WCAG 2.2 success criteria, relevant especially for mobile applications, all banks failed at least 9 of them;
- Bank A failed 9 out of 44 tested WCAG success criteria.
- Bank B failed 12 out of 44 tested WCAG success criteria.
- Bank C failed 15 out of 44 tested WCAG success criteria.
- Bank D failed 15 out of 44 tested WCAG success criteria.
Detailed results
Please check public Google sheet table with detailed results if you want to make abstractions of the data.
| # | Success criterion (SC) | SC short | Bank D | Bank C | Bank B | Bank A | SUM per row |
|---|---|---|---|---|---|---|---|
| 1 | 1.1.1 - Non-text Content - Level A | 1.1.1 | N | N | N | N | 4 |
| 2 | 1.3.1 - Info and Relationships - Level A | 1.3.1 | N | N | N | N | 4 |
| 3 | 1.3.2 - Meaningful Sequence - Level A | 1.3.2 | Y | Y | Y | Y | 0 |
| 4 | 1.3.3 - Sensory Characteristics - Level A | 1.3.3 | Y | Y | Y | Y | 0 |
| 5 | 1.3.4 - Orientation - Level AA | 1.3.4 | Y | Y | N | N | 2 |
| 6 | 1.3.5 - Identify Input Purpose - Level AA | 1.3.5 | N | N | Y | Y | 2 |
| 7 | 1.4.1 - Use of Color - Level A | 1.4.1 | Y | Y | Y | Y | 0 |
| 8 | 1.4.3 - Contrast (Minimum) - Level AA | 1.4.3 | Y | Y | Y | Y | 0 |
| 9 | 1.4.4 - Resize Text - Level AA | 1.4.4 | Y | Y | Y | N | 1 |
| 10 | 1.4.5 - Images of Text - Level AA | 1.4.5 | Y | Y | Y | Y | 0 |
| 11 | 1.4.10 - Reflow - Level AA | 1.4.10 | Y | Y | Y | Y | 0 |
| 12 | 1.4.11 - Non-text Contrast - Level AA | 1.4.11 | Y | N | Y | Y | 1 |
| 13 | 1.4.12 - Text Spacing - Level AA | 1.4.12 | Y | Y | Y | Y | 0 |
| 14 | 1.4.13 - Content on Hover or Focus - Level AA | 1.4.13 | Y | Y | Y | Y | 0 |
| 15 | 2.1.1 - Keyboard - Level A | 2.1.1 | N | Y | Y | N | 2 |
| 16 | 2.1.2 - No Keyboard Trap - Level A | 2.1.2 | Y | N | Y | Y | 1 |
| 17 | 2.1.4 - Character Key Shortcuts - Level A | 2.1.4 | Y | Y | Y | Y | 0 |
| 18 | 2.2.1 - Timing Adjustable - Level A | 2.2.1 | Y | Y | Y | Y | 0 |
| 19 | 2.2.2 - Pause, Stop, Hide - Level A | 2.2.2 | Y | Y | Y | Y | 0 |
| 20 | 2.3.1 - Three Flashes or Below Threshold - Level A | 2.3.1 | Y | Y | Y | Y | 0 |
| 21 | 2.4.3 - Focus Order - Level A | 2.4.3 | N | N | N | Y | 3 |
| 22 | 2.4.4 - Link Purpose (In Context) - Level A | 2.4.4 | N | N | Y | Y | 2 |
| 23 | 2.4.6 - Headings and Labels - Level AA | 2.4.6 | Y | N | Y | Y | 1 |
| 24 | 2.4.7 - Focus Visible - Level AA | 2.4.7 | Y | Y | Y | Y | 0 |
| 25 | 2.4.11 - Focus Appearance (Minimum) (Level AA) WCAG 2.2 | 2.4.11 | N | N | N | Y | 3 |
| 26 | 2.4.12 - Focus Not Obscured (Minimum) (Level AA) WCAG 2.2 | 2.4.12 | Y | Y | N | N | 2 |
| 27 | 2.5.1 - Pointer Gestures | 2.5.1 | Y | Y | Y | Y | 0 |
| 28 | 2.5.2 - Pointer Cancellation - Level A | 2.5.2 | Y | Y | Y | Y | 0 |
| 29 | 2.5.3 - Label in Name - Level A | 2.5.3 | N | N | N | N | 4 |
| 30 | 2.5.4 - Motion Actuation - Level A | 2.5.4 | Y | Y | Y | Y | 0 |
| 31 | 2.5.7 - Dragging Movements (Level AA) WCAG 2.2 | 2.5.7 | N | Y | N | Y | 2 |
| 32 | 2.5.8 - Target Size (Minimum) (Level AA) WCAG 2.2 | 2.5.8 | N | N | N | Y | 3 |
| 33 | 3.1.1 - Language of Page - Level A | 3.1.1 | Y | Y | Y | Y | 0 |
| 34 | 3.2.1 - On Focus - Level A | 3.2.1 | Y | Y | Y | Y | 0 |
| 35 | 3.2.2 - On Input - Level A | 3.2.2 | Y | Y | Y | Y | 0 |
| 36 | 3.2.6 - Consistent Help (Level A) WCAG 2.2 | 3.2.6 | Y | Y | Y | Y | 0 |
| 37 | 3.3.1 - Error Identification - Level A | 3.3.1 | N | N | N | Y | 3 |
| 38 | 3.3.2 - Labels or Instructions - Level A | 3.3.2 | N | N | Y | Y | 2 |
| 39 | 3.3.3 - Error Suggestion - Level AA | 3.3.3 | N | Y | Y | Y | 1 |
| 40 | 3.3.4 - Error Prevention (Legal, Financial, Data) - Level AA | 3.3.4 | Y | Y | Y | Y | 0 |
| 41 | 3.3.7 - Accessible Authentication (Level AA) WCAG 2.2 | 3.3.7 | Y | Y | Y | Y | 0 |
| 42 | 3.3.9 - Redundant Entry (Level A) WCAG 2.2 | 3.3.9 | Y | Y | Y | Y | 0 |
| 43 | 4.1.2 - Name, Role, Value - Level A | 4.1.2 | N | N | N | N | 4 |
| 44 | 4.1.3 - Status Messages - Level AA | 4.1.3 | N | N | N | N | 4 |
Most interesting findings
Some of the most interesting findings:
- PIN codes provided as images with not alternatives and automatically disappearing, causing even Artificial Intelligence to fail to recognize them. Meaning that people will need to reach out for help and get others to read them their PIN number, making them vulnerable to abuse.
- Partially supporting physical keyboard where all elements are accessible with keyboard, besides writing in amounts. This has also implications on other types of assistive technologies (like switch control based assistive technologies).
- Most of PDF documents like contracts, invoices, transaction lists are not accessible. I suspect that they are often generated automatically and were not tested with assistive technology.
- I was limiting time for the manual testing, and with more time I would most likely find more issues...
UD2024 compendium
I did an extended abstract and a poster for this project, please feel free to find it on page 159 in the Conference Compendium of the Seventh International Conference on Universal Design (UD2024), Shaping a Sustainable, Equitable and Resilient Future for All.