Automatically move mouse with flat review

[nvaccessAuto]

Reported by aleksey_s on 2010-07-27 09:42
A mode should be implemented that, when switched on, will move the mouse automatically when user reviews the flat representation. It will save a few keypresses when one wants to click a lot in the application.
Blocked by #58

[nvaccessAuto]

Comment 1 by aleksey_s on 2010-07-27 09:45
Since it seems to be not very difficult to implement, but it will bring a great convenience for users, I would like to see it taken for 2010.2.

[nvaccessAuto]

Comment 2 by mdcurran on 2010-07-27 13:19
Would you be happy simply with a "Mouse follows review cursor" checkbox in mouse settings dialog?

I agree it would be very easy to do.

Though I am worried that some of this stuff will be scattered across dialogs.
In VoiceOver all the navigation routing checkboxes are in the one dialog.
Do we wish to consider doing this?
However this is out of scope for this ticket, for now I guess we can only go with the checkbox either in the mouse settings dialog or the review cursor dialog. Which one is best?
Perhaps the Review Cursor dialog suits better now I think about it.

[nvaccessAuto]

Comment 3 by jteh on 2010-07-27 22:33
I vote Review cursor dialog.

[nvaccessAuto]

Comment 4 by mdcurran on 2010-07-30 03:39
Having played with this idea a bit, I am not too sure its all that useful.
There is a rather annoying side-affect when dealing with menus, and anything else that react when the mouse is moved over them.
Specifically for menus, if the mouse moves when the review cursor moves (and if focus moves navigator object is enabled, which it is by default) then the mouse moves when the focus moves. However, when arrowing down a menu and you highlight a submenu, that submenu opens and the focus is placed on the first item. If that item is itself a submenu, it happens again.
This is because if you run the mouse down a menu, each item is highlighted, and any submenus are opened automatically.

I think, its easy enough for the user to move the review cursor where they want, and then press move navigator object to mouse.
At least then they are physically making the decision to deal with any consequence that arrises from placing the mouse at that point.
Alternatively, we could just move the mouse from the actual scripts, rather than in setReviewPosition/setNavigatorObject, which would I guess have the same affect, but not confuse the focus.
But, this would mean quite a bit of code duplication... unlesss perhaps setReviewPosition and setNavigatorObject could take a keyword argument... something like interactive=True, meaning that the user deliberately set it, rather than say focus/selection.

[nvaccessAuto]

Comment 5 by jteh on 2010-07-30 03:44
Alternatively, we could implement the setting and document that it shouldn't be used in conjunction with focus moves navigator/review.

How does !VoiceOver on the Mac deal with this?

[nvaccessAuto]

Comment 6 by aleksey_s (in reply to comment 4) on 2010-07-30 18:54
Replying to mdcurran:

Having played with this idea a bit, I am not too sure its all that useful.

I am sure it is useful, however, we should decide how to deal with cases you found.

Specifically for menus, if the mouse moves when the review cursor moves (and if focus moves navigator object is enabled, which it is by default) then the mouse moves when the focus moves.

This brings a new problem. When the application decides to set focus on something when user is working with the flat review, NVDA will "turn off" the flat review and bounce the review cursor to the object in focus. This behavior may be confusing for the user. Or do i misunderstand it? What happens with the position of the review cursor in the flat review when the navigator object moves?

However, when arrowing down a menu and you highlight a submenu, that submenu opens and the focus is placed on the first item. If that item is itself a submenu, it happens again.

This is how I see it:

• The "Review cursor moves mouse" mode is turned on
• user moves review cursor to the menu item with submenu
• since mouse is set to the menu item with submenu, the submenu opens automatically
• the focus is set to the first item in the submenu, but the review cursor (and mouse cursor too) stays on the top level item, since user hasn't moved it from there
• Therefore, second submenu doesn't open
  This behavior I saw in jaws with the jaws-cursor.

  I think, its easy enough for the user to move the review cursor where they want, and then press move navigator object to mouse.

You mean "move mouse to the navigator object", right?

At least then they are physically making the decision to deal with any consequence that arrises from placing the mouse at that point.

I think by enabling that mode users already made the decision. Anyway, I find myself dealing with menus is mutch easier with using traditional windows way (arrowing) rather than moving around with the flat review.

unlesss perhaps setReviewPosition and setNavigatorObject could take a keyword argument... something like interactive=True, meaning that the user deliberately set it, rather than say focus/selection.

Will it behave as I described above then?

[nvaccessAuto]

Comment 7 by jteh (in reply to comment 6) on 2010-07-31 04:32
Replying to aleksey_s:

This brings a new problem. When the application decides to set focus on something when user is working with the flat review, NVDA will "turn off" the flat review and bounce the review cursor to the object in focus. This behavior may be confusing for the user.

The idea is that flat review isn't a "mode" as such. Rather, it is something the user chooses to do at a particular point in time. We did this because we still believe that, particularly with modern applications, object navigation is still a more effective way of accessing controls. Also, flat review won't work everywhere, so having it enabled all the time doesn't make sense.

• The "Review cursor moves mouse" mode is turned on
• user moves review cursor to the menu item with submenu
• since mouse is set to the menu item with submenu, the submenu opens automatically
• the focus is set to the first item in the submenu, but the review cursor (and mouse cursor too) stays on the top level item, since user hasn't moved it from there

This behavior I saw in jaws with the jaws-cursor.

That's because JAWS doesn't tether its mouse (JAWS) cursor to the focus (PC cursor) by default. This is actually the other alternative: we implement a Mouse moves review cursor option, but make it clear that it is not wise to have both it and Focus moves navigator enabled simultaneously.

[nvaccessAuto]

Comment 8 by mdcurran (in reply to comment 7) on 2010-07-31 06:24
Replying to jteh:
This is actually the other alternative: we implement a Mouse moves review cursor option, but make it clear that it is not wise to have both it and Focus moves navigator enabled simultaneously.
Um, we already have that option: Review cursor dialog -> follow mouse checkbox. Unless I'm not following what you're saying. However, if the mouse moves with this option enabled, the review cursor will always be thrown in to the deepest object at the point of the mouse, rather than staying on the flat review object if thats where you were.
I think, we do need to consider the idea of a mode a bit more I think... before this is perfect.

[nvaccessAuto]

Comment 9 by jteh (in reply to comment 8) on 2010-07-31 06:32
Replying to mdcurran:

This is actually the other alternative: we implement a Mouse moves review cursor option, but make it clear that it is not wise to have both it and Focus moves navigator enabled simultaneously.

Um, we already have that option: Review cursor dialog -> follow mouse checkbox.

Err, I meant Review cursor moves mouse. Sorry.

[nvaccessAuto]

Comment 10 by Diamond on 2013-09-05 08:32
Are there any plans to add or is there any progress on implementing this feature? I have to say that the lack of such a feature is one of the things which I found initially made using NVDA less intuitive than other screen readers. I suppose it is possible to adjust to the lack, but it would still be useful to have.

[nvaccessAuto]

Comment 11 by nvdakor on 2014-04-28 23:35
Hi,
In case this gets implemented, how would this fit in with screen review mode?

[nvaccessAuto]

Comment 12 by nvdakor on 2014-12-12 07:14
Hi,
Update: A few weeks ago, somebody asked about moving the mouse with a keyboard (such a utility does exist), with the devs saying you can route the mouse to the current object once you found what you are looking for. With the implementation of screen review, I feel the mouse routing can be done anywhere, thus sort of making this suggestion come alive (in an unexpected way).
Thanks.

[LeonarddeR]

It might be good to set a priority for this, though it will probably not get higher than a p4.

[jcsteh]

I'd say this is a feature and we don't prioritise most features here now (only fixes). I'll apply the feature label.

[Adriani90]

cc: @josephsl as far as I know this is somehow possible with your golden cursor addon by pressing windows+nvda+m which allows moving the mouse cursor by pressing the arrow keys.

[josephsl]

Hi,

Five years later...

I don't maintain Golden Cursor - it was never my add-on to begin with.

I think this request might be a bit harder to implement as modern apps (particularly Microsoft Store apps) cannot be navigated via screen review. The closest we could offer is an option to move the system focus and/or mouse cursor when using flattened view of the UI tree. Using visual highlight to highlight navigator object might be a workaround, but the core of the issue is mouse movement.

Thanks.

[josephsl]

Hi,

2025 update: can we say that keyboard-based mouse movement (in recent NVDA) be somewhat of a close substitute?