0

I'm trying to use a file I've created called my_addqueue.py its located here

src\
    blog\
       my_addqueue.py

and the function I' am trying to use is in the same directory

src\
    blog\
       my_addqueue.py
       my_scrapy.py

I am trying to learn how to use redis with my django app. But when I run

python blog/my_addqueue.py

I get the following error message

Parent module '' not loaded, cannot perform relative import.

here is my code

my_scraps.py

   def p_panties():
        def swappo():
            user_one = ' "Mozilla/5.0 (Windows NT 6.0; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0" '
            user_two = ' "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5)" '
            user_thr = ' "Mozilla/5.0 (Windows NT 6.3; Trident/7.0; rv:11.0) like Gecko" '
            user_for = ' "Mozilla/5.0 (Macintosh; Intel Mac OS X x.y; rv:10.0) Gecko/20100101 Firefox/10.0" '

            agent_list = [user_one, user_two, user_thr, user_for]
            a = random.choice(agent_list)
            return a

        headers = {
            "user-agent": swappo(),
            "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
            "accept-charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.3",
            "accept-encoding": "gzip,deflate,sdch",
            "accept-language": "en-US,en;q=0.8",
        }

        pan_url = 'http://www.examop.com'
        shtml = requests.get(pan_url, headers=headers)
        soup = BeautifulSoup(shtml.text, 'html5lib')
        video_row = soup.find_all('div', {'class': 'post-start'})
        name = 'pan videos'

        if os.getenv('_system_name') == 'OSX':
            author = User.objects.get(id=2)
        else:
            author = User.objects.get(id=3)

        def youtube_link(url):
            youtube_page = requests.get(url, headers=headers)
            soupdata = BeautifulSoup(youtube_page.text, 'html5lib')
            video_row = soupdata.find_all('p')[0]
            entries = [{'text': div,
                        } for div in video_row]
            tubby = str(entries[0]['text'])
            urls = re.findall('http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+', tubby)
            cleaned_url = urls[0].replace('?&autoplay=1', '')
            return cleaned_url

        def yt_id(code):
            the_id = code
            youtube_id = the_id.replace('https://www.youtube.com/embed/', '')
            return youtube_id

        def strip_hd(hd, move):
            str = hd
            new_hd = str.replace(move, '')
            return new_hd

        entries = [{'href': div.a.get('href'),
                    'text': strip_hd(strip_hd(div.h2.text, '– Official video HD'), '– Oficial video HD').lstrip(),
                    'embed': youtube_link(div.a.get('href')), #embed
                    'comments': strip_hd(strip_hd(div.h2.text, '– Official video HD'), '– Oficial video HD').lstrip(),
                    'src': 'https://i.ytimg.com/vi/' + yt_id(youtube_link(div.a.get('href'))) + '/maxresdefault.jpg', #image
                    'name': name,
                    'url': div.a.get('href'),
                    'author': author,
                    'video': True

                    } for div in video_row][:13]

        for entry in entries:
            post = Post()
            post.title = entry['text']
            title = post.title
            if not Post.objects.filter(title=title):
                post.title = entry['text']
                post.name = entry['name']
                post.url = entry['url']
                post.body = entry['comments']
                post.image_url = entry['src']
                post.video_path = entry['embed']
                post.author = entry['author']
                post.video = entry['video']
                post.status = 'draft'
                post.save()
                post.tags.add("video", "Musica")
        return entries

my_addqueue.py

from rq import Queue
from redis import Redis
from .my_scraps import p_panties

redis_con = Redis()
q = Queue('important', connection=redis_con)

task = q.enqueue(p_panties())
print('noted')

How do i fix this?

EDIT file structure

enter image description here

losee
  • 2,190
  • 3
  • 29
  • 55
  • Do you have a file called "\_\_init__.py" in your blog dir? It can be an empty file – nkhumphreys Aug 08 '16 at 21:28
  • @nkhumphreysno I don't I was thinking of adding it but I am so new to python I didn't want this to break my app. Are you familiar with redis? what is the typical file structure for using redis and queue? – losee Aug 08 '16 at 21:31
  • This isn't related to redis. You need to tell Python to treat your dir as a module. The presence of this file does that – nkhumphreys Aug 08 '16 at 21:32
  • I already have an empty init there – losee Aug 08 '16 at 21:34
  • Can you post the full directory structure? – nkhumphreys Aug 08 '16 at 21:38
  • Try to cd into the blog dir and then run "Python my_addqueue.py" does that run? – nkhumphreys Aug 08 '16 at 21:49
  • Does it give the same error? – nkhumphreys Aug 08 '16 at 22:05
  • which python version are you using? – an0o0nym Aug 09 '16 at 03:17
  • I.m using 3.5 is that an issue? i know it's one with google api. because I was trying to use that, but it maxes out at 3.4 – losee Aug 09 '16 at 14:30
  • what I ended up doing is moving my_task.py into my blog directory but it gave me a whole different trace back which I am about to post. my_addqueue.py is in the project folder. I teseted smaller functions like the word counter on herkus site and it works but for some reason mine wont – losee Aug 09 '16 at 14:47
  • Possible duplicate of [Relative imports in Python 3](http://stackoverflow.com/questions/16981921/relative-imports-in-python-3) – an0o0nym Aug 09 '16 at 20:47

0 Answers0