extract a specific word between two values

Question

I curl a html page and stock output into variable, so I try to extract a word between two value, but I failed.

 </tr> <tr> <td><a <a href="https://test/one/AAA">AAA</a></td>
 <td>Thu Aug 30 09:59:36 UTC 2018</td> <td align="right"> 2247366 </td>
 <td></td> </tr> <tr> <td><a
 href="https://test/one/1.1.22">1.1.22</a></td> <td>Thu Aug 30 09:59:36
 UTC 2018</td> <td align="right"> 5 </td> <td></td> </tr> </table>
 </body> </html>

 content=$(curl -s https://test/one/)
 echo $content | sed -E 's_.*one/([^"]+).*_\1_'

I try to catch value after one/ and before ", so I want to extract AAA, 1.1.22,...

Use xpath to do so – Mévatlavé Kraspek Aug 30 '18 at 14:06 — Mévatlavé Kraspek, Aug 30 '18 at 14:06
Show us what you have tried and failed. – iamauser Aug 30 '18 at 14:07 — iamauser, Aug 30 '18 at 14:07
echo $content | sed 's/.*one/=$.*$".*/\1/' – locklockM Aug 30 '18 at 14:20 — locklockM, Aug 30 '18 at 14:20

karakfa · Accepted Answer · 2018-08-30T14:59:28.017

0

$ ... | sed -E 's_.*one/([^"]+).*_\1_'

AAA
BBB

since you have slash in your content, better to choose a different delimiter, here I used _.

UPDATE Since you changed the input file format dramatically, here is the updated script

$ echo "$contents" | sed -nE '/one/s_.*one/([^"]+).*_\1_p'
AAA
1.1.22

edited Aug 30 '18 at 14:59

answered Aug 30 '18 at 14:25

karakfa

66,216
7
41
56

Never parse HTML with regex – Gilles Quénot Aug 30 '18 at 14:27
1

That's rather dogmatic. HTML is text and if you don't need the tree structure as in this case, there is nothing wrong with text tools. – karakfa Aug 30 '18 at 14:28
Thanks my friends!! but it gives me just AAA, because AAA, BBB, it is juste an example, more precision, AAA, can be 0.01, 0.2.1, 3.33.1... – locklockM Aug 30 '18 at 14:31
doesn't matter what the content is as long as it doesn't contain `sed` delimiter and quote sign. – karakfa Aug 30 '18 at 14:32
so how I can have the rest ? I have just the first one plz – locklockM Aug 30 '18 at 14:35
what is your test case? The example you posted works as shown. Please update the question with more realistic input. – karakfa Aug 30 '18 at 14:36
OK, the reason is you lose newlines since you didn't double quote `$content2`. The `sed` command as written will only extract one per line. See the update – karakfa Aug 30 '18 at 14:59
thanks it works, It is imposible for me do to by myself. thanks!! – locklockM Aug 30 '18 at 15:07

score 0 · Answer 2 · answered Aug 30 '18 at 14:28

Don't parse XML/HTML with regex, use a proper XML/HTML parser and a powerful xpath query.

theory :

According to the compiling theory, XML/HTML can't be parsed using regex based on finite state machine. Due to hierarchical construction of XML/HTML you need to use a pushdown automaton and manipulate LALR grammar using tool like YACC.

realLife©®™ everyday tool in a shell :

You can use one of the following :

xmllint often installed by default with libxml2, xpath1 (check my wrapper to have newlines delimited output

xmlstarlet can edit, select, transform... Not installed by default, xpath1

xpath installed via perl's module XML::XPath, xpath1

xidel xpath3

saxon-lint my own project, wrapper over @Michael Kay's Saxon-HE Java library, xpath3

or you can use high level languages and proper libs, I think of :

python's lxml (from lxml import etree)

perl's XML::LibXML, XML::XPath, XML::Twig::XPath, HTML::TreeBuilder::XPath

ruby nokogiri, check this example

php DOMXpath, check this example

Check: Using regular expressions with HTML tags

Example using xpath :

//a[contains(@href, "https://test/sites/two/one")]

extract a specific word between two values

2 Answers2

theory :

realLife©®™ everyday tool in a shell :

or you can use high level languages and proper libs, I think of :

Example using xpath :