7

The code works for Word and Outlook but fails with PowerPoint in that only the first character or first word of the textbox ever gets selected. Is this a bug? Is there any workaround? Try this on a simple PowerPoint slide in PowerPoint 2013.

private static async Task<string> getText(double x, double y)
{
    string result = null;

    try
    {
        var location = new System.Windows.Point(x, y);
        AutomationElement element = AutomationElement.FromPoint(location);

        object patternObj;
        if (element.TryGetCurrentPattern(TextPattern.Pattern, out patternObj))
        {
            var textPattern = (TextPattern)patternObj;

            var range = textPattern.RangeFromPoint(location);
            range.ExpandToEnclosingUnit(TextUnit.Word);
            range.Select();

            var text = range.GetText(-1).TrimEnd('\r');
            return text.Trim();
        }
        else
        {
            return "no text found";
        }
    }
    catch (Exception ex)
    {
        return ex.Message;
    }
}

You cannot see it from the screenshot, but the mouse is on "first" not "stuck", but regardless of where the mouse is placed, it always is stuck. Maybe this is fixed in PowerPoint 2016?

enter image description here

When I look at the bounding box for the range it is always the whole element, rather than the selected word. That could be part of the problem of why RangeToPoint is not working.

Original posted in MSDN but no response...

Update. If I use 

text = printRange(range, text);
while (range.Move(TextUnit.Word, 1) > 0)
{
    text += Environment.NewLine;
    text = printRange(range, text);
}

I get

enter image description here

tofutim
  • 22,664
  • 20
  • 87
  • 148
  • Why not use COM interface of Power Point? You can access it as `PowerPoint.Application` object. Every MS Office product has great object model available as COM interface. – Vasily Ryabov Sep 15 '15 at 14:54
  • I have no time for writing C# code, but in Python I could get text so: `pp.Presentations[0].Slides[8].Shapes[0].TextFrame.TextRange.Text` (caption of the 9th slide). – Vasily Ryabov Sep 15 '15 at 15:09
  • Where `pp` is an instance of `PowerPoint.Application`. – Vasily Ryabov Sep 15 '15 at 15:10

2 Answers2

3

This behavior is probably due to a limitation in PowerPoint 2013, and I expect you can't work around it using UIA. When you call RangeFromPoint(), the UIA provider hit beneath the mouse, (ie the one that's implementing IUIAutomationTextPattern::RangeFromPoint(),) is meant to return a degenerative (ie empty) range where the mouse cursor is. Then the UIA client can expand the returned range to get the surrounding character, word, line or paragraph.

However, as you point out, PowerPoint 2013 isn't doing that. I've just written the test code below, (using a managed wrapper for the native Windows UIA API generated by tlbimp.exe,) and found that PowerPoint apparently returns a TextRange for the entire text box beneath the cursor. When I ran the code, I found that I did get the expected word beneath the cursor in WordPad, Word 2013 and PowerPoint OnLine, but not PowerPoint 2013. I got the same results when I ran the Text Explorer tool that's part of the Inspect SDK tool. The image below shows Text Explorer reporting that the text returned from PowerPoint 2013 is the entire text in the a text box, when the mouse is hovering over one of those words.

(I should add that for the test code below to work at all, I think the current display scaling setting needs to be at 100%. I've not added code to account for some other scaling being active.)

I don't know if this is fixed in PowerPoint 2016, I'll try to look into that and let you know.

Thanks,

Guy

enter image description here

private void buttonGetTheText_Click(object sender, EventArgs e)
{
    labelText.Text = "No text found.";

    IUIAutomation uiAutomation = new CUIAutomation8();

    Point ptCursor = Cursor.Position;

    tagPOINT pt;
    pt.x = ptCursor.X;
    pt.y = ptCursor.Y;

    // Cache the Text pattern that's available through the element beneath
    // the mouse cursor, (if the Text pattern's supported by the element,) in
    // order to avoid another cross-process call to get the pattern later.
    int patternIdText = 10014; // UIA_TextPatternId
    IUIAutomationCacheRequest cacheRequestTextPattern =
        uiAutomation.CreateCacheRequest();
    cacheRequestTextPattern.AddPattern(patternIdText);

    // Now get the element beneath the mouse.
    IUIAutomationElement element = 
        uiAutomation.ElementFromPointBuildCache(pt, cacheRequestTextPattern);

    // Does the element support the Text pattern?
    IUIAutomationTextPattern textPattern =
        element.GetCachedPattern(patternIdText);
    if (textPattern != null)
    {
        // Now get the degenerative TextRange where the mouse is.
        IUIAutomationTextRange range = textPattern.RangeFromPoint(pt);
        if (range != null)
        {
            // Expand the range to include the word surrounding 
            // the point where the mouse is.
            range.ExpandToEnclosingUnit(TextUnit.TextUnit_Word);

            // Show the word in the test app.
            labelText.Text = "Text is: \"" + range.GetText(256) + "\"";
        }
    }
}
Guy Barker - Microsoft
  • 574
  • 2
  • 3
  • Thank you for looking into this Guy. Any chance we can track down someone at Microsoft that can fix or offer a workaround? – tofutim Sep 13 '15 at 04:32
  • I'll see if I can find out anything more about this, and let you know. I installed the Office 2016 Preview this morning, and pointed Inspect's Text Explorer to PowerPoint 2016, and it seemed to behave in the same way as PowerPoint 2013. That is, when I try to find the text range beneath the mouse cursor, PowerPoint returns the entire text for the text box beneath the cursor, rather than the degenerative range at the point where the cursor is. – Guy Barker - Microsoft Sep 13 '15 at 17:37
  • Thank you for testing that for me. Any help would be much appreciated. – tofutim Sep 13 '15 at 22:11
  • Any new news or leads? – tofutim Sep 15 '15 at 14:41
  • Guy, I have received word from a PowerPoint dev that this has been replicated and filed as a bug. Hopefully they will find a fix for it soon. – tofutim Sep 16 '15 at 22:10
  • @GuyBarker-Microsoft Hi Guy, is there a possibility that you can shed some light on this issue: http://stackoverflow.com/questions/32347734/ui-automation-events-stop-being-received-after-a-while-monitoring-an-application/32515554?noredirect=1#comment53234158_32515554 ? tofutim, sorry for my lack of originality ;) – o_weisman Sep 24 '15 at 07:25
  • Hi Guy, any update or pressure you can put on the PowerPoint team? – tofutim Dec 15 '15 at 02:25
0

I can suggest only Python code getting caption text of the slide (for example). Sorry, I have no time to re-write it on C#. You can play with the PowerPoint.Application COM object and MSDN example of Power Point automation.

from __future__ import print_function
import win32com.client as com
pp = com.Dispatch('PowerPoint.Application')
print(pp.Presentations[0].Slides[8].Shapes[0].TextFrame.TextRange.Text)
Vasily Ryabov
  • 9,386
  • 6
  • 25
  • 78