BUG: I can't use Flow Launcher with screen readers

[MesterPerfect]

Checks

• I have checked that this issue has not already been reported.

Problem Description

I am a blind user
I use NVDA/JAWS screen readers to navigate Windows
I can't use Flow Launcher with screen readers
I found it very difficult to install Flow Launcher using an NVDA screen reader
After installing Flow Launcher, I can't open settings, browse search results, or anything

To Reproduce

1. install Flow Launcher
2. I can't access it in the system tray

Screenshots

No response

Flow Launcher Version

v1.13.0

Windows Build Number

windows 10 Version 22H2 (OS Build 19045.2604)

Error Log

2023-02-18.txt

[onesounds]

Hi,

I tested it and found something wrong. I changed it to be readable by a screen reader. It hasn't gone live yet, so we need to make sure it's working properly. The following URL is my modified version, and you can get a portable version to test.
https://ci.appveyor.com/project/JohnTheGr8/flow-launcher/builds/46271078/artifacts

The current UI has a lot of arbitrary changes I've made, so there are parts of it that don't read properly in screen readers (toggle switches, for example), and I don't expect you to be happy with it. I will fix the preference pane as much as I can, and if there are other things you think I should fix, please let me know and I'll do my best to incorporate them.

[MesterPerfect]

Hi,

I tested it and found something wrong. I changed it to be readable by a screen reader. It hasn't gone live yet, so we need to make sure it's working properly. The following URL is my modified version, and you can get a portable version to test. https://ci.appveyor.com/project/JohnTheGr8/flow-launcher/builds/46271078/artifacts

The current UI has a lot of arbitrary changes I've made, so there are parts of it that don't read properly in screen readers (toggle switches, for example), and I don't expect you to be happy with it. I will fix the preference pane as much as I can, and if there are other things you think I should fix, please let me know and I'll do my best to incorporate them.

Thank you for responding
I have downloaded your modified version
Installed by clicking on it
But I couldn't access settings to test changes you made
The right-click menu is still not compatible
I mean menu on system tray
I can attach photos and videos if you want

[onesounds]

ok please attach video.

[MesterPerfect]

ok please attach video.

ok
I cannot navigate system tray menu items using keyboard
But I can do this with mouse
I had some difficulty navigating settings
But it can be bypassed
Also I can't view search results

Untitled.mp4

[onesounds]

I'll ask again to confirm.

1. You mean you can't move the focus of the menu that appears when you right-click on the tray icon to the keyboard? I've seen this issue, I'll fix it.
2. No problem getting to open the tray menu (not navigating), In other words, I don't have to do any of the entry steps up to this point. right?
3. What do you mean the ' I can't view search results'? In your video, you didn't try to search.

Additionally, the settings window can be opened with CTRL+I while the flow search bar is focused, which means you don't have to use the tray menu. Surely, I'll fix the keyboard navigation in the tray menu.

[MesterPerfect]

I'll ask again to confirm.

1. You mean you can't move the focus of the menu that appears when you right-click on the tray icon to the keyboard? I've seen this issue, I'll fix it.

yes

2. No problem getting to open the tray menu (not navigating), In other words, I don't have to do any of the entry steps up to this point. right?

Yes, right.

3. What do you mean the ' I can't view search results'? In your video, you didn't try to search.

When I search for something using Flow Launcher some results are displayed, I can't navigate these results using the arrows to choose from search results

Additionally, the settings window can be opened with CTRL+I while the flow search bar is focused, which means you don't have to use the tray menu. Surely, I'll fix the keyboard navigation in the tray menu.

ok thank you

[onesounds]

https://ci.appveyor.com/project/JohnTheGr8/flow-launcher/builds/46274886/artifacts

I've fixed the tray issue first (pressing TAB activates keyboard focus. At this point, you can move the cursor with the arrows.), but I have two problems. These two are harder to fix.

1. Arrow key to activate keyboard focus is impossible. I want to activate the keyboard with the arrow keys as well, like the tray menu in other programs.
2. The positioning of the context menu relative to the mouse.

We don't recommend using the tray menu due to the nature of the program, and all functions except resetting positions can be accessed via hotkeys, so we think this should be sufficient. If you need anything else, please let us know.

---

I don't understand the comment about not being able to navigate search results, currently you can move selections with the arrows up and down. (Enter to launch) In my case, I've found that Flow searches work well, and each item is readable on a screen reader (NVDA).

If you happen to have your mouse cursor over the flow launcher, it will automatically focus on the mouse cursor location and keyboard navigation may not work well.

If this means that tab doesn't navigate, that's because it's being used as an autocomplete.

Tell us more about what you're having trouble with in the steps of entering a search term, getting search results, and selecting an item. Example > "After getting search results, the focus doesn't move when I press the down arrow."

[MesterPerfect]

"After getting search results, the focus doesn't move when I press the down arrow."

Yes that's right, this problem really exists.

Also, there is a problem navigating within the settings
When i focus on any combobox and move around using the arrows,
Elements are read using nvda like this
Flow.Launcher.ViewModel.SettingWindowViewModel+SearchWindowPosition

I will attach another video for you to hear how nvda reads

[MesterPerfect]

Untitled.mp4

[onesounds]

1. Is the tray issue resolved?

2. I still don't understand the navigation issue you are talking about, can't you search and open notepad or sticky note? In your video, you are doing the search correctly. It looks like you're also using the up and down keys to change the selection (although I can't tell exactly what you're pressing). Make sure to move the mouse away from the flow window. Trigger the flow with alt+space and try navigating through the results again with the keyboard. If necessary, use the alt+number shortcut to select items. Verify that this behavior works correctly. If moving the mouse away solves this problem, I will add a setting to ignore mouse manipulations on the search screen. OK. What you mean is that the navigation works, but the screen reader reads each item indistinguishably. Am I understanding you correctly?

3. some menu items are not being read correctly. This has been confirmed for other menu items as well. There are currently issues with window navigation other than "General".

I want to fix two issues first, the tray issue and the menu navigation issue, and work on the rest. I basically want to create a usable state and fix (2nd problem) the third problem. This is because making too many modifications in one PR at once is a code management problem. Once issues 1 and 2 are resolved, we'll mark this issue as resolved, and open issue 3 as a separate issue.

p.s Would a high-contrast theme and large fonts help with your testing? If so, we'll provide you with a relevant theme.

[MesterPerfect]

Is the tray issue resolved?

Yes, the tray menu can be activated and then press 'tab' key from keyboard, after that, menu items can be navigated using arrows.

Regarding problem of navigating search results, in this video I opened on screen keyboard 'osk' and did what I do from the external keyboard
I think this will be clearer for you

Uploading Untitled.mp4…

OK. What you mean is that the navigation works, but the screen reader reads each item indistinguishably. Am I understanding you correctly?

Yes that's right.

[MesterPerfect]

@onesounds

[onesounds]

I've identified areas that are beyond my skill set. I need someone(@taooceros )else to help me, but they're busy. Please be patient.