2

I am trying to get my images from the folder to an array and processing them one by one. But I want to do it in parallel. I tried to implement each image in a different process and execute but this does not seem to work

my code:

import numpy as np
import os
import cv2
from multiprocessing import Process
import time

file_path=[]
path="C:/Users/User/Desktop/Dataset_Screens/picture_dataset_Small/"
for root,dir,files in os.walk(path):
    for f in files:
        file_path.append(os.path.join(root,f))


path1="C:/Users/User/Desktop/Dataset_Screens/picture_dataset_Big/"
for root1,dir1,files1 in os.walk(path1):
     for f1 in files:
        file_path.append(os.path.join(root1,f1))

print(len(file_path))


def get_image(inputfilepath):
    img_original= cv2.imread(inputfilepath)
    img_array=np.asarray(img_original)
    blur= cv2.pyrMeanShiftFiltering(img_original,21,49)
    gray_image= cv2.cvtColor(blur, cv2.COLOR_BGR2GRAY)
    ret,thresh= cv2.threshold(gray_image,70,255,cv2.THRESH_BINARY)    
    _, contours,hierarchy =cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
    countourimage=cv2.drawContours(img_original,contours,-1,0,3)

    largest_area=0
    print('I am inside the loop')
    for i,c in enumerate(contours):
        contour_areas=cv2.contourArea(c)
        if(contour_areas>largest_area):
            largest_area= contour_areas
            x_rect,y_rect,w_rect,h_rect=cv2.boundingRect(c)
            cropped=img_original[y_rect:y_rect+h_rect,x_rect:x_rect+w_rect]

    cv2.imwrite('C:/Users/User/Anaconda3/contourimage.jpg',cropped)

t= time.time()      
if __name__ == '__main__':

    for i in range(4):
        p = Process(target=get_image, args=(file_path[i],))
        p.start()

print('Done in ', time.time()-t)
print('We are DONEEEEEEE!!!!!!')

I am using Spyder which does not seem to work either, so i ran on conda prompt and the outputs are shown below:

The output in spyder:

132
Done in  0.022994279861450195
DONEEE!!

The output in conda prompt:

C:\Users\User\Anaconda3>python examples.py
132
Done in  0.05248904228210449
DONEEE!!
132
Done in  0.0
DONEEE!!
132
Done in  0.0
DONEEE!!
132
Done in  0.0
DONEEE!!
132
Done in  0.0
DONEEE!!
Corrupt JPEG data: premature end of data segment
Corrupt JPEG data: premature end of data segment
Corrupt JPEG data: premature end of data segment
Corrupt JPEG data: premature end of data segment
I am inside the loop
I am inside the loop
I am inside the loop
I am inside the loop

I am trying to convert my python program in to parallel to decrease the time taken

I have added p.join() to the function

if __name__ == '__main__':
    for i in range(4):
        p = multiprocessing.Process(target=get_image, args=(file_path[i],))
        p.start()
        p.join()

and the output in spyder:

132
Done in  30.90500521659851
DONEEE!!

and conda prompt:

C:\Users\User\Anaconda3>python examples.py
132
132
Done in  0.0
DONEEE!!
Corrupt JPEG data: premature end of data segment
I am inside the loop
132
Done in  0.0
DONEEE!!
Corrupt JPEG data: premature end of data segment
I am inside the loop
132
Done in  0.0
DONEEE!!
Corrupt JPEG data: premature end of data segment
I am inside the loop
132
Done in  0.0
DONEEE!!
Corrupt JPEG data: premature end of data segment
I am inside the loop
Done in  30.726163625717163
DONEEE!!

I am an absolute beginner with parallel processing..Please suggest..Thanks in advance!!

  • Please check these. Seems that this is not an issue with python's parallel processing: https://stackoverflow.com/questions/30237613/opencv-read-image-premature-end-of-jpeg-file and https://github.com/Haneke/Haneke/issues/4 – Rambarun Komaljeet Oct 09 '18 at 12:55
  • Thanks for the response.. I went through the links but does not seem to answer my query because i think if the image data was the issue, spyder should have also thrown an error... but in spyder the function does not get called at all –  Oct 09 '18 at 14:48
  • 1
    How do you determine that the function does not get called at all? I think it does get called, it is just that the secondary processes do not get access to the terminal and cannot print. Try to generate some other effects such as file IO and see if that has any effect. – JohanL Oct 10 '18 at 18:40
  • I could wrong as well... but i have tried to put print statement as the first statement in the defined method but it never printed so thought it does not call at all... I am working on spyder which did not work, so i tried to run through the cmd prompt and still the same.. –  Oct 11 '18 at 06:42

0 Answers0