0

I'm currently testing out a web crawler using a pre built shell, and during the process, I kind of got stuck at this part, where at the def create_data_files part it complains about "formal parameter name expected.

As referenced here's the part of the code so far (Im using PyCharm Community Edition 2017.1.1).

# Each website is a separate project (folder)
def create_project_dir(directory):
    if not os.path.exists(directory):
        print('Creating directory ' + directory)
        os.makedirs(directory)
create_project_dir('test')

# Create queue and crawled files (if not created)
def create_data_files(test,'www.startlap.com/'):
    queue = os.path.join(project_name , 'queue.txt')
    crawled = os.path.join(project_name,"crawled.txt")
    if not os.path.isfile(queue):
        write_file(queue, base_url)
    if not os.path.isfile(crawled):
        write_file(crawled, '')


# Create a new file
def write_file(path, data):
    with open(path, 'w') as f:
        f.write(data)



# Add data onto an existing file
def append_to_file(path, data):
    with open(path, 'a') as file:
        file.write(data + '\n')


# Delete the contents of a file
def delete_file_contents(path):
    open(path, 'w').close()


# Read a file and convert each line to set items
def file_to_set(file_name):
    results = set()
    with open(file_name, 'rt') as f:
        for line in f:
            results.add(line.replace('\n', ''))
    return results


# Iterate through a set, each item will be a line in a file
def set_to_file(links, file_name):
    with open(file_name,"w") as f:
        for l in sorted(links):
            f.write(l+"\n")
Draken
  • 3,134
  • 13
  • 34
  • 54
Grassroot Programmer
  • 11
  • 1
  • 1
  • 2
  • you can't define a function like that, the parameter should be a name, not a value. definition: ```def create_data_files(test, url)``` call: ```def create_data_files("sss", "www.sss.ss")``` – xiaobing Apr 21 '17 at 08:54
  • sadly even with this it still displays an error. made a screenshot about it for convenience sake: http://imgur.com/a/x1RLc – Grassroot Programmer Apr 21 '17 at 09:29
  • sorry i made a mistake, you should write `def create_data_files(test, url)`when you define a function, here `test` and `url` are parameters, then can it somewhere else `create_data_files("arg1", "arg2")`, here the value you pass in are arguments. – xiaobing Apr 21 '17 at 09:41
  • difference between parameters and arguments. http://stackoverflow.com/questions/156767/whats-the-difference-between-an-argument-and-a-parameter – xiaobing Apr 21 '17 at 09:42

1 Answers1

1

You need to change your def create_data_files(test,'www.startlap.com/'): line with something like def create_data_files(test, url='www.startlap.com/'):

Ali Cirik
  • 1,475
  • 13
  • 21