Trouble with duplicate lines of logs in log file

Question

I've been trying to get this working correctly all day, its nearly complete just a strange issue I'm getting. Every result that is found in the search query gets logged as expected but the first result gets logged once, the second gets logged twice, the third gets logged three times etc.

Any ideas how to get rid of the duplicates? Example of logs

#!/usr/bin/python
import urllib
import simplejson 
import logging
from logging.handlers import SysLogHandler

query = urllib.urlencode({'q' : 'test'})
url = 'http://ajax.googleapis.com/ajax/services/search/web?v=1.0&%s' \
      % (query)
search_results = urllib.urlopen(url)
json = simplejson.loads(search_results.read())
results = json['responseData']['results']
for i in results:
    logger = logging.getLogger()
    logger.addHandler(SysLogHandler(address=('192.168.0.2', 514)))
    logger.addHandler(logging.FileHandler("hits.log"))
    logging.warn(i['url'])
    print i['url']

same question as http://stackoverflow.com/questions/6729268/python-logging-messages-appearing-twice — gurney alex, Jan 23 '12 at 17:08
@Gurney: I disagree, that question has every line doubled. The _answer_ might be relatively similar but they are different enough questions that I think the "duplicate" isn't quite right. — sarnold, Jan 24 '12 at 00:09
@sarnold: well, I won't argue with you there, but IMO it's the same question: the code adds a handler more than one time, and as a result you get duplicated log output. The way you call addHandler several times matters little in this regard. — gurney alex, Jan 24 '12 at 13:13

score 5 · Answer 1 · edited Jul 29 '15 at 18:33

5

I had similar problem, but I needed to add a new handler each time in the for loop. So removing handler inside a loop didn't help me.

When you create handler like this:

hdl = logging.FileHandler("hits.log")

you need to remove it like this:

logger.removeHandler(hdl)

edited Jul 29 '15 at 18:33

DilithiumMatrix

17,795
22
77
119

answered Nov 16 '12 at 15:32

Marko Trajkov

101
1
5

score 5 · Accepted Answer · answered Jan 23 '12 at 15:16

5

Because you're adding a new handler each time in the for loop. Do this outside the loop, then only do the actual logging.warn inside the loop.

answered Jan 23 '12 at 15:16

Daniel Roseman

588,541
66
880
895

well its not answered yet thats why its not shown as such if i only have the logging.warn(i['url']) in the loop it failes to write to the syslog – H20 Jan 23 '12 at 19:25

score 4 · Answer 3 · answered Jan 23 '12 at 16:57

4

As you have not accepted the answer, as Daniel said, you need to have

logger = logging.getLogger('')
logger.addHandler(logging.FileHandler("hits.log"))
logger.addHandler(SysLogHandler(address=('192.168.0.2', 514)))

outside the for loop.

answered Jan 23 '12 at 16:57

shadyabhi

16,675
26
80
131

Trouble with duplicate lines of logs in log file

3 Answers3