Say all for displayModelEditable controls

[nvaccessAuto]

Reported by aleksey_s on 2010-05-16 13:35
Currently, say all for those controls is impossible.
ERROR - queueHandler.pumpAll (16:30:48):
error in generator 472
Traceback (most recent call last):
File "D:\nvda\main\source\queueHandler.py", line 67, in pumpAll
next(gen)
File "D:\nvda\main\source\sayAllHandler.py", line 129, in readTextHelper_generator
updater.updateCaret()
File "D:\nvda\main\source\textInfos\offsets.py", line 412, in updateCaret
return self._setCaretOffset(self._startOffset)
File "D:\nvda\main\source\textInfos\offsets.py", line 155, in _setCaretOffset
raise NotImplementedError
NotImplementedError

[nvaccessAuto]

Comment 1 by jteh on 2010-08-31 22:42
Can we just look up the screen coordinates for the required offset and then click the left mouse button there? That should cover a lot of cases. However, we might have been concerned that this would click a link.

[nvaccessAuto]

Comment 2 by mdcurran (in reply to comment 1) on 2010-08-31 23:34
Replying to jteh:

Can we just look up the screen coordinates for the required offset and then click the left mouse button there? That should cover a lot of cases. However, we might have been concerned that this would click a link.

I think clicking is too dangerous myself. However, I would suggest that we create a custom window message, which we can here call wm_setCaretPos, and we would send this from NVDA to the Window in question. We would then have code in nvdaHelperRemote, registered with our registerWindowsHook function, for WH_CALLWNDPROC, which would look for this message, and if seen, call SetCaretPos, using the coordinates passed in with the window message.

Only question is where exactly to put this code in nvdaHelperRemote (i.e. what file).

A variant of this idea, if we were uncomfortable sending window messages from NVDA, is to do it via rpc, however the remote implementation of that rpc call would still have to SendMessage anyway, so that setCaretPos was called from the correct thread.
But the advantage would be that we would just include it in the displayModel interface, and registerWindowsHook and unregisterWindowsHook could be called at will, around the SendMessage call, therefore not needing its own initialization and termination code.

I like that idea best from a code organization point of view. From a performance point of view not using rpc would be better... but, we're already using it for heeps of stuff anyways...

[nvaccessAuto]

Comment 3 by mdcurran on 2010-09-01 03:00
Hmm, thinking about this, calling setCaretPos may move the caret as far as the OS understands (visually, and it will cause MSAA locationChange, and getCaretPos will give back the new coords), but the application itself will not know the caret has moved, and therefore it will be out of synch, and any edits will happen at the wrong point.

Clicking is a better idea. But, the link problem definitly does exist. Tested in windbg, clicking once on the links for "uf /c /D /O KERNEL32!Sleep" activates the links, rather than just moving the caret to the clicked position.

However, as the display model code for NVDA has always been thought of as a last resort, perhaps we can overlook this issue -- links don't really appear much in many application edit fields.

I do notice though that, at least for richEdit, when the mouse is on a link, its shape is no longer an edit cursor (iBeem), but instead is some custom shape. Also, in textPad, which is a custom edit field needing displayModel, when the mouse is inside the edit field it switches its shape to the standard edit cursor (iBeem). If this is always the case, perhaps we could disallow setting caretPos if when moving the mouse there the mouse is not an edit cursor.

[nvaccessAuto]

Comment 4 by aleksey_s on 2010-09-01 07:39
that mouse solution doesn't fix the initial issue (say all). We can't click on the text that actually is not on the screen, we need to scroll there at first. So I feel solution introduced by cbd4e3b isn't a fix for this ticket. We should investigate whether setCaretPos really makes problems for some apps as described by Mick, and then (if no other ideas come around) consider a mixed way: e.g. do setCaretPos first to scroll the text field and make a click to notify the app about caret movement.

[nvaccessAuto]

Comment 5 by jteh (in reply to comment 4) on 2010-09-01 08:21
Replying to aleksey_s:

e.g. do setCaretPos first to scroll the text field

It's highly possible that !SetCaretPos literally just sets the position of the visible caret on screen; i.e. that is what the app calls when it wants the drawn caret to move. Certainly, this won't help for off-screen scrolling, as the point needs to be on-screen to move the caret there. Anyway, this is all supposition; the idea needs to be tested to be certain. Note that Mick's fix does fix braille cursor routing, so if you meant this ticket to be for say all only, we'll change the summary and move it to 2010.3.

[nvaccessAuto]

Comment by mdcurran on 2010-09-20 18:17
(In #653) Fixed in 6d51c85.

[nvaccessAuto]

Comment 8 by jteh on 2012-01-19 00:06
Changes:
Changed title from "Implement _setCaretOffset for displayModelEditable controls" to "Say all for displayModelEditable controls"

[LeonarddeR]

Do I understand it correctly that this issue should already have been closed, given https://github.com/nvaccess/nvda/issues/659#issuecomment-155282532/nvaccess/nvda/issues/653#issuecomment-155282479 ?

cc @michaelDCurran

Edit: This is not my day! Sorry for first posting in #653

[jcsteh]

NVDA now moves the caret in these controls by clicking. However, this probably isn't sufficient for say all because it doesn't scroll the window, so we'd never get to the new text. So no, I'm pretty sure this is not fixed.

[jcsteh]

One way to fix this might be to use the page turning stuff (see textInfos.DocumentWithPageTurns) to scroll the control once the cursor hits the end of the text. The question is how to scroll and how to detect that scrolling was successful. Do we just send page down? The problem with that is that page down might not scroll a full screen.

I'm setting this to p4, as we're always going to have reliability problems in controls that require screen scraping, modern controls use proper text implementations anyway, and this will be costly to implement and test.

[josephsl]

Hi,

More than seven years later...

Since @LeonarddeR mentioned Python 3 and display model+say all, can we say that the mentioned issue closes this one, or are there additional work to do?