The simple solution is just as you stated: loop through the files and search through each one.
Naive approach
for file in files:
for line in file:
if line contains pattern:
print file.name
If you wanted to be a little better, you could immediately bail out of the file as soon as you found a match.
Slightly better
for file in files:
for line in file:
if line contains pattern:
print file.name
break # found what we were looking for. continue to next file
At this point you could attempt to distribute the problem across multiple threads. You will probably be IO bound and may even see worse performance because multiple threads are trying to read different parts of the disk at the same time
Threaded approach
for file in files:
# create new worker thread which does...
for line in file:
if line contains pattern:
# insert filename into data structure
break # found what we were looking for. continue to next file
# wait for all threads to finish, collect and display data
But if you are concerned about performance, you should either use grep or copy how it works. It saves time by reading the files as raw binary (rather than break it up line by line) and makes use of a string searching algorithm called the Boyer–Moore algorithm. Refer to this other SO about how grep runs fast.
Probably What You Want™ approach
grep -l pattern files