Detect location of specific text

Question

Appium 1.5.0 
Xcode  7.3 
IOS 10.11.4 OSX EI Capitan
Using on:Simulator iOS 8.1

Suppose we have a text field To see more information clickhereand to get more click here.

So I identified the text using xpath and I can get its location and size. Now can we somehow get the location of here .I need to click it, but because it is not an element I cannot search for it. So somehow if I can get its location, I can tap at that.

But how can we get its location and still keep it generic so that it runs on different devices?

You can refer this http://stackoverflow.com/questions/21349725/character-index-at-touch-point-for-uilabel — Proton, Jun 28 '16 at 11:48
I have an idea but I don't know Appium, can you use Objective C with it ? — AnthoPak, Jun 28 '16 at 12:09
@AnthoninC. unfortunately have to use appium...still can you share the idea.....as an answer...i will try if it works....m doing it in python — vks, Jun 28 '16 at 12:11
@DungProton i need it in python....through `appium-python` client....do you have any idea how to achieve the same in python — vks, Jun 28 '16 at 12:15
if you call `source_window` it will return all the elements and their position — juhlila, Jul 29 '16 at 19:12
@juhlila the text is embedded but it's not an identifiable element — vks, Jul 29 '16 at 19:13

score 1 · Answer 1 · answered Jun 28 '16 at 12:23

The idea using Objective C (I know the OP uses Appium but he told me to answer) is to use library such as TTTAttributedLabel and using it this way :

Create TTTAttributedLabel (UILabel subclass), and put text content in it, as if it was a UILabel. Set its delegate to self. Then, add a link to a word this way :

NSRange rangeWord = [attributedLabel.text rangeOfString:@"here"];
[attributedLabel addLinkToURL:[NSURL URLWithString:@"anActionOnClickHere"] withRange:rangeUser];

On clicking words it will call this method in which you can handle the click :

- (void)attributedLabel:(__unused TTTAttributedLabel *)label
   didSelectLinkWithURL:(NSURL *)url {
    NSString *urlToString = [url absoluteString];

    if ([urlToString containsString:@"anActionOnClickHere"]) { //perform segue for example
        [self performSegueWithIdentifier:@"hereSegue" sender:self];
    }
}

I absolutely don't know about Python, hope you will find a equivalent method.

Thanx....will dive deep though at the outset it seems this might not be possible in python.....will update u — vks, Jun 28 '16 at 12:25

score 1 · Accepted Answer · edited May 23 '17 at 11:51

There isn't any direct way to achieve this(no API available in Appium itself). However, my solution is generic and will work on all devices(Android and iOS), all programming language implementation.

[You have already done] get the location and size of the element.
Take a screenshot of the current screen.
Crop image from the screenshot using the location and size of the image.[This step can be skipped if there are not ambiguous targets]
[You need to do this only one time manually] Crop out the image portion you want to get the location.
Search the location of the cropped image, get the location and perform a precise tap/click.

For image search refer here : How to find an image within another image using python

I marked this though its not totally generic as when devices change,the one time manual work will have to be done again... — vks, Jul 20 '16 at 19:45

Detect location of specific text

2 Answers2