How to find the URL`s from the content reading from a file

Question

I have a below data present in the file with multiple paragraphs, I want to find the URL`s presented in each paragraph separately.

Licenses: 

Prasd Licence v2.1


Prasd Free License
=====================


v. 2.1
------

This prasad Free License (the "License") applies to any original work of
authorship (the "Original Work") whose owner (the "Licensor") has placed the
following notice immediately following the copyright notice for the Original
Work:

  Licensed under the prasad Free License version 2.1

---

balaji Software Rights Notice
(balaji 2.7.7)

balaji License
=============


We encourage users to develop software with balaji. However, we do ask that credit
is given to us for developing balaji. By "credit", we mean that if you use balaji
or incorporate any source code into one of your programs (commercial product,
research project, or otherwise) that you acknowledge this fact somewhere in the
documentation, research report, etc... If you like balaji and have developed a
nice tool with the output, please mention that you developed it using balaji. In
addition, we ask that the headers remain intact in our source code. As long as
these guidelines are kept, we expect to continue enhancing this system and expect
to make other tools available as they are completed.



---

mogga 1.1

mogga Software License
=======================


Version 1.1
-----------


      "This product includes software developed by the mogga Software Foundation
      (http://www.mogga.org/)."



--------------------------------------------------------------------------------

This software consists of voluntary contributions made by many individuals on
behalf of the mogga Software Foundation. For more information on the mogga
Software Foundation, please see <http://www.mogga.org/>.

jfhwifwefwe

---

Praveen 2.0

Version 2.0, January 2004
=========================


http://www.praveen.org/licenses/

TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION

1. Definitions.

    with the License. You may obtain a copy of the License at
  http://www.praveen.org/licenses/LICENSE-2.0 Unless required by applicable law
  or agreed to in writing, software distributed under the License is
 

---

You could try reading the file and doing re.findall("(http[s]?:\/\/www\.[\S]+)", line). — palvarez, May 23 '19 at 14:19
findall will give all the URL`s but I need the URL for each paragraph separately with key as paragraph name and value as URL of that particular paragraph — Praveen Mandadi, May 23 '19 at 14:20
Your file is in HTML or markdown? How can you tell you have a new paragraph? — palvarez, May 23 '19 at 14:26
the data is in .txt file and it has a specified format as I mentioned in the body — Praveen Mandadi, May 23 '19 at 14:32
Does this answer your question? [Find all urls in file](https://stackoverflow.com/questions/60903661/find-all-urls-in-file) — oxfn, Apr 06 '21 at 19:24

score 1 · Accepted Answer · answered May 23 '19 at 14:55

Is this what you wanted maybe? Would be nice to have the expected result in your question.

URL_REGEX Reference

import re

URL_REGEX = r'((?:(?:http|ftp|https):\/\/)?[-\w@:%.\+~#=]{2,256}\.[a-z]{2,6}\b[-\w@:%\+.~#?&\/=]*)'
result = {}

with open("text.txt") as f:
  par_name = ""
  lines = f.readlines()
  urls = []
  for i, line in enumerate(lines):
    line = line.strip()
    # New paragraph
    if {"="} & set(line):
      if par_name or urls:
        result[par_name] = urls
      urls = []
      par_name = lines[i-1].strip()
      continue
    urls.extend(re.findall(URL_REGEX, line))
print(result)

Results in:

{
  'Prasd Free License': [],
  'balaji License': [],
  'mogga Software License': ['http://www.mogga.org/', 'http://www.mogga.org/']
}

How to find the URL`s from the content reading from a file

1 Answers1