Running a for loop to create functions with exec, inside a class, inside a function

Question

I know it sounds complicated and it may not be possible, but I thought I'd try anyway.

So I'm doing some web scraping with Selenium, and any time that I want to run some jQuery on a page, instead of having

driver.execute_script("$('button').parent().click()")

I want to tidy up my code and just have

j('button').parent().click()

I'm attempting to accomplish this by having a class j and then when you have a function like parent it just returns another instance of that class.

In these examples I just have it printing the string that will get executed so that it can be tested more easily. And the way I have it the class exists inside a function.

So in this example everything works fine when all the class functions are defined normally, but if you uncomment the for loop section and define the .parent() function with exec then you get the error NameError: name 'j' is not defined:

def browser():

    class j:

        def __init__(self, str):
            if str[0] == '$':
                self.jq = str
            else:
                self.jq = f"$('{str}')"

        def click(self):
            output = f"{self.jq}.click()"
            print(output)
            return j(output)

        def parent(self):
            return j(f'{self.jq}.parent()')

        # functions = 'parent, next, prev'
        # for fn in functions.split(', '):
        #     exec(
        #         f"def {fn} (self):\n" +
        #         f"    return j(f'{{self.jq}}.{fn}()')"
        #     )

    j('button').parent().click()

browser()

However, if you move everything outside of the browser function and run everything then there's no error that way also, and that's if you define the functions either way.

I also tried doing exec(code, globals()) but that just gave me a different error message instead: AttributeError: 'j' object has no attribute 'parent'

Is there some way to define functions with exec in this way and do what I'm try to do?

Edit: This is the entire error message:

Traceback (most recent call last):
  File "C:\Users\Rob\Desktop\site\mysite\py\z.py", line 30, in <module>
    browser()
  File "C:\Users\Rob\Desktop\site\mysite\py\z.py", line 28, in browser
    j('button').parent().click()
  File "<string>", line 2, in parent
NameError: name 'j' is not defined

[Dynamic/runtime method creation (code generation) in Python](https://stackoverflow.com/questions/533382/dynamic-runtime-method-creation-code-generation-in-python) — furas, Feb 21 '20 at 05:25
Instead of dynamically generating those methods, just implement `__getattr__` or similar magic methods and handle the call to undefined methods that way. — deceze, Feb 21 '20 at 05:57

furas · Answer 1 · 2020-02-21T07:15:39.947

Using setattr and examples from Dynamic/runtime method creation (code generation) in Python I made:

def browser():

    class j:

        def __init__(self, str):
            if str[0] == '$':
                self.jq = str
            else:
                self.jq = f"$('{str}')"

        def click(self):
            output = f"{self.jq}.click()"
            print('click:', output)
            return j(output)

        def fun(self, name):
            return j(f'{self.jq}.{name}()')

        def fun2(self, name):
            self.jq += f'.{name}()'
            return self

    # - after class -

    functions = 'parent, next, prev'
    for fn in functions.split(', '):
         #setattr(j, fn, lambda cls, fun=fn: j(f'{cls.jq}.{fun}()'))
         #setattr(j, fn, lambda cls, fun=fn: j.fun(cls, fun))
         setattr(j, fn, lambda cls, fun=fn: cls.fun(fun))

    print('test:', j('button').parent().next().click().fun2('a').fun('b').jq)

browser()

But it is not ideal - it creates something like next = fun("next") and it may run it as next("x") and add .x() instead of .next()

I tested also return self instead of creating new instance - and it also works.

Beacuse lambda is use in for-loop so it needs fun=fn to correctly get value from fn . Without this it will take value from fn when it will be executed and then all function get the same value from fn - last value assigned to fn in loop.

EDIT: Instead of running directly setattr with lambda I run function which runs setattr and use inner function instead of lambda - this way it can create function without second argument so now .next("x") raise error.

def browser():

    class j:

        def __init__(self, str):
            if str[0] == '$':
                self.jq = str
            else:
                self.jq = f"$('{str}')"

        def click(self):
            output = f"{self.jq}.click()"
            print('click:', output)
            return j(output)

        def fun(self, name):
            return j(f'{self.jq}.{name}()')

        def fun2(self, name):
            self.jq += f'.{name}()'
            return self

        @classmethod
        def add_function(cls, name):
            def func(cls):
                cls.jq += f'.{name}()'
                return cls
            func.__name__ = name
            setattr(j, name, func)

    # - after class -

    functions = 'parent, next, prev'
    for fn in functions.split(', '):
         #setattr(j, fn, lambda cls, fun=fn: j(f'{cls.jq}.{fun}()'))
         #setattr(j, fn, lambda cls, fun=fn: j.fun(cls, fun))
         #setattr(j, fn, lambda cls, fun=fn: cls.fun(fun))
         #setattr(j, fn, lambda cls: cls.fun(x))
         j.add_function(fn)

    item = j('button').parent().next().click().fun2('a').fun('b')
    print('test:', item.jq)

    j.add_function('other')

    item = item.other()
    print('test:', item.jq)

    item = j('input').other().click()
    print('test:', item.jq)

    print('name:', j.next.__name__) # gives `next`, without `func.__name__ = name` it gives `func`

browser()

I tested also __getattr__

def browser():

    class j:


        def __init__(self, str):
            if str[0] == '$':
                self.jq = str
            else:
                self.jq = f"$('{str}')"

        def click(self):
            return self.fun('click')
            #self.jq += f".click()"
            #return self

        def fun(self, name):
            self.jq += f'.{name}()'
            return self

        functions = ['parent', 'next', 'prev']

        #def __getattr__(self, name):
        #    '''acceptes every name'''
        #    return lambda: self.fun(name) # I use `lambda` because it needs function to use `()`

        #def __getattr__(self, name):
        #    '''___name__ doesn't change its name in error message and it shows `<lambda>`'''
        #    a = lambda: self.fun(name)
        #    a.__name__ = name
        #    return a

        def __getattr__(self, name):
            '''acceptes only name from list `functions`'''
            if name in self.functions:
                return lambda: self.fun(name) # I use `lambda` because it needs function to use `()`
            else:
                raise AttributeError(f"'j' object has no attribute '{name}'")

        #def __getattr__(self, name):
        #    '''test running without `fun()`'''
        #    self.jq += f'.{name}()'
        #    return lambda:self # I use `lambda` because it needs function to use `()`


    # - after class -

    item = j('button').parent().next().click().fun('a').fun('b')
    print('test:', item.jq)

    j.functions.append('other')
    item = item.other()
    print('test:', item.jq)

    item = j('input').other().click()
    print('test:', item.jq)

    #print('name:', j.next.__name__) # doesn't work

    print(j('a').tests().jq) 

browser()

I created new code which does't run directly `setattr` with `lambda` but it runs other function which runs `setattr` and it doesn't need `lambda` — furas, Feb 21 '20 at 06:26
Thanks for the improvements. This got me most of the way there. I've worked on your answer and managed to get to pretty much 100% of what I was trying for. Do you mind if I post an updated version of this answer? — Rob Kwasowski, Feb 21 '20 at 06:48
I tested also magic function `__getattr__` and I put it in answer. — furas, Feb 21 '20 at 07:16

Rob Kwasowski · Accepted Answer · 2020-02-22T06:54:05.560

Furas's second answer got me most of the way there so he deserves the upvotes. But I managed to improve on it and get it to do everything I was looking for.

So here is my version that can take arguments and checks type and puts quotes around strings but not numbers.

def browser():

    class j:

        def __init__(self, str):
            self.jq = f"$('{str}')"

        def click(self):
            self.jq += ".click()"
            print(self.jq)
            return self

        @classmethod
        def quotes(cls, arg):
            q = "'" * (type(arg) == str and len(arg) > 0)
            output = f'{q}{arg}{q}'
            return output

        @classmethod
        def add_function(cls, name):
            def func(cls, arg=''):
                cls.jq += f'.{name}({j.quotes(arg)})'
                return cls
            func.__name__ = name
            setattr(j, name, func)

    j_functions = 'closest, eq, find, next, parent, parents, prev'
    for fn in j_functions.split(', '):
        j.add_function(fn)

    j('button').eq(0).next('br').prev().click()

browser()


>> $('button').eq(0).next('br').prev().click()

Running a for loop to create functions with exec, inside a class, inside a function

2 Answers2