2

This is a inverse question of

$driver.get_element_by_xpath(....)

Background

I have a series of dynamically generated pages to parse. The target element doesn't have a good locator, xpath, text, or id. However, the neighbor element has a unique text to match. My plan is to locate the neighbor element and use its xpath to come up with the target xpath.

$neighbor_element = $driver.get_element_by_text("unique text"); # or some other way
$neighbor_xpath = $neighbor_element.xpath; # this step is the question
$target_xpath = modify($neighbor_xpath); # this is my function
$target_element = $driver.get_element_by_xpath($target_xpath);

I have searched around. cannot find a function or method to get xpath from an element, Perl or Python.

Update

I apologize that I cannot post the example page because it is owned by a company, but I hope the question is straightforward enough. When I use chrome devtools to inspect, I see the xpath are related

neighbor xpath = //*[@id="lable_ni.dynmic_string 123456"]/lable/span[2]
target   xpath = //*[@id="dynmic_string 123456"]

Update2

This the dummy test file

<!DOCTYPE html>
<html>

<head></head>

<body>
    <div>
        <div id="lable_ni.dynmic_string 123456">
            <label id="lable_ni.dynmic_string 123456">
                <!-- I use this as neighbour element -->
                <span>unique text</span>
            </label>
        </div>
        <div>
            <!-- target element is here -->
            <textarea id="dynmic_string 123456">target text is here</textarea>
        </div>
    </div>
</body>

</html>

This is how it looks like

enter image description here

My strategy was to

  1. find the xpath of "unique text". (this text is known beforehand)
  2. convert it to the xpath of target text
  3. get the target text

I stuck at the first step

oldpride
  • 761
  • 7
  • 15
  • Update the question with the text based HTML of the desired element and the neighbour element. – undetected Selenium Apr 12 '22 at 21:12
  • If I understand right, you are looking for an example HTML page. Sorry, the original pages belong to a company which I cannot post here, but the question is generic enough I believe. – oldpride Apr 12 '22 at 21:18
  • What does it mean a "neighbor" element? A sibling? LIke there's a `

    ` and there's another `

    ` right under/above (etc) it in source?

     – zdim Apr 12 '22 at 21:19
  • Thank you for checking. I just updated with xpath examples – oldpride Apr 12 '22 at 21:25
  • @zdim, your understanding is correct – oldpride Apr 12 '22 at 21:32
  • Your real problem might belong to someone else, but you can recreate the general situation in a fake example. – brian d foy Apr 13 '22 at 16:43
  • @briandfoy, thank you. I will type a dummy example soon. Your are a known Perl guru, so maybe you know from Perl perspective at least: why selenium (Python or Perl) doesn't provide a method to the element object to dump to get the xpath or css, which are very basic info and shouldn't take too much time/memory to keep track when parsing the HTML? – oldpride Apr 13 '22 at 20:13

2 Answers2

2

Find the parent of that "known-neighbor" then their select child, or children that are next in the list to the known one.

I am a little confused by what is known about the target. The original text says that

The target element doesn't have a good locator, xpath, text, or id.

but the edit shows

target xpath = //*[@id="dynmic_string 123456"]

Since there's got to be some way to tell I'll take it as a fact that something is known about the target. Another way would be to get all children (so, all siblings) and browse through them and locate the ones next to the known one.

Here is an example with Perl. For this very page, take that known-neighbor to be a <p> with some text and find the target which is a sibling <p> and which has some given text in it. (As an equivalent of finding a sibling with that id given in the question's edit.)

use warnings;
use strict;
use feature 'say';

use Selenium::Chrome;

my $url_SO = q(https://stackoverflow.com/questions/71849162/)
    . q(how-to-find-the-neighbour-element-of-an-active-element-)
    . q(using-selenium-with-pytho);

my $drv = Selenium::Chrome->new( 'extra_capabilities' => 
    { 'goog:chromeOptions' => { args => [ 'headless' ] }}  );

$drv->get($url_SO);
say "\nPage title: ", $drv->get_title, "\n";

# Our "neighbor": <p> with text 'This is...'
# Get parent with: 'element-spec/..'
my $parent = eval { 
    $drv->find_element(
        '//p[text()="This is a inverse question of"]/..') };
if ($@) { die "Error on <p>'s parent: $@" }
say "known-<p>'s parent tag: ", $parent->get_tag_name;
say "known-<p>'s parent text:";
say '-'x50, "\n", $parent->get_text, "\n", '-'x50;

# Target: <p> with text that contains word 'searched'
my $tgt = eval { 
    $drv->find_child_element( 
        $parent, q(./p[contains(text(), 'searched')]) ) };    
if ($@) { die "find-child error: $@" }
say "target text: ", $tgt->get_text;

This does as expected. I can't really post a closer match to the question because not much is given. If I misunderstood that that shown id of the target is known then use find_child_elements on the $parent and go down the list probing for the known neighbor. The target should be the one before or after (and you better know which :)

If the target isn't in fact a true sibling, but is rather a child of a further ascendant (than the immediate parent), the xpath expressoin can go up the hierarchy with the additional /../.. (etc).

This uses the (server-less) Selenium::Chrome, with methods in Selenium::Remote::Driver and Selenium::Remote::WebElement

(It should be possible to do the whole known-element->parent->child thing with a single XPath expression, or perhaps even look directly for siblings, I'll look once there's time.)

zdim
  • 64,580
  • 5
  • 52
  • 81
  • Zdim, thank you for the suggestion. The ".." to get parent is very useful. I will use that. Regarding to your question, "dynmic_string 123456" is dynamically generated and therefore, I don't know it beforehand and I cannot start with using it in xpath search. However the neighbor element (nearby but not siblings) and target element share the same unique string. Therefore, if I can find the neighbor, then the neighbour's xpath, and then I can derive the target's xpath. What you are suggesting is to use the parent ".." (relative xpath) instead of absolute xpath. I got it. – oldpride Apr 13 '22 at 13:05
  • @oldpride I'll look at this again today, for one thing to use the neighbor's xpath (had no time yesterday). So the target element has that same unique string? I didn't realize that. – zdim Apr 13 '22 at 16:19
  • I applied the parent notation '..' into //span[text()="unique text"]/../../../div[2]/textarea . this worked for me. – oldpride Apr 14 '22 at 04:10
1

Using Selenium Python clients incase you are able to locate the neighbor element as:

neighbor_element = driver.find_element(By.XPATH, "unique text")

Depending upon the location of the target element with respect to neighbor_element you can use either of the following locator strategies:

  • Incase the target_element is a <span> element located left to the neighbor_element:

    target_element = driver.find_element(locate_with(By.TAG_NAME, "span").to_left_of(neighbor_element))

You have to include the following imports:

from selenium.webdriver.support.relative_locator import locate_with
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • 1
    Thank you. This is a bonus solution as it is out of my expectation; so is the locator strategies. They both will reshape my selenium programming. Saying that, it is still puzzling why Selenium couldn't provide a method to print the element's xpath. I noticed that you even changed the title of this thread, removing find-xpath-by-element, is it some kind of cover-up? LOL – oldpride Apr 12 '22 at 23:34
  • @oldpride The `relative-locator` concept is new with Selenium4 and have no direct dependency on _`xpath`_. _`TAG_NAME`_ is used legitimately. – undetected Selenium Apr 12 '22 at 23:38