How can I extract the prediction from multiprocessing in python?

Question

I am trying to multiprocess the result of svm.

And I tried the following simple method.

clf = svm.SVC(C=1.0, cache_size=1000000, class_weight=None, coef0=0.0,
    decision_function_shape='ovo', degree=3, gamma='auto', kernel='rbf',
    max_iter=-1, probability=False, random_state=None, shrinking=True,
    tol=0.001, verbose=False)

clf.fit(x, y)

def predict(x):
    clf.predict(x)

from multiprocessing import Pool

pool= Pool(processes = 4)


setting = np.loadtxt('~/test.csv', delimiter=',')
x=setting[:,0:3]
y=setting[:,3]

x[i]
[cols,rows] = setting.shape
i = 0
while i < rows:
    k = x[i]
    pool.map(predict,[[k]])
    print(pool.map(predict,[[k]]))
    i = i+1

This code will run, but it does not seem to have the right results.

I want to return or print the predicted value.

I would be grateful if you could provide a method or code to get predicted value results.

fordy · Accepted Answer · 2018-06-25T12:37:03.497

1

setting.shape will return a tuple (m, n), where m is the number of rows, and n is the number of columns. Looks like you have it the other way round.

Trying changing your line to

[rows, cols] = setting.shape

Also, you might want to try splitting your prediction data into equally sized matrices each with rows/processes entries and mapping over them. The overhead of pooling row-by-row might not be worth it.

edited Jun 25 '18 at 12:37

answered Jun 25 '18 at 12:13

fordy

2,520
1
14
22

Could you please give me some code to try the method you recommended? – 송준석 Jun 25 '18 at 12:20
and when i try this code, predict values are all None. is there any problem to input type or method? – 송준석 Jun 25 '18 at 12:24
See this answer: https://stackoverflow.com/questions/31449291/how-to-parallelise-predict-method-of-a-scikit-learn-svm-svc-classifier – fordy Jun 25 '18 at 12:27

How can I extract the prediction from multiprocessing in python?

1 Answers1