13

I would like to use an Attribute-Relation File Format with scikit-learn to do some NLP task, is this possible? How can use an .arff file with scikit-learn?

Matthew Spencer
  • 2,265
  • 1
  • 23
  • 28
tumbleweed
  • 4,624
  • 12
  • 50
  • 81

5 Answers5

24

I really recommend liac-arff. It doesn't load directly to numpy, but the conversion is simple:

import arff, numpy as np
dataset = arff.load(open('mydataset.arff', 'rb'))
data = np.array(dataset['data'])
renatopp
  • 1,275
  • 10
  • 17
  • Thanks for the feedback. Any idea of how can i use this conversation to classify?. – tumbleweed Dec 08 '14 at 18:49
  • I all ready know that for instance with SVM the basic idea for classification is:`from sklearn import svm s = svm.SVC() lables = [label1, label2] s.fit(training_data, labels)` How could i present an `arff` file to a classification algorithm? – tumbleweed Dec 08 '14 at 21:26
  • Were you able parse TF-IDF scores from .arff file and use it in sklearn? – Nithish Inpursuit Ofhappiness Jun 20 '17 at 23:25
  • 1
    I get the following exception when loading an arff file using liac-arff: `BadAttributeType: Bad @ATTRIBUTE type, at line 21.` I think the reason is the presence of "relational" attributes in my arff file. Has anyone a solution? Thank you. – Florent F May 08 '19 at 16:49
10

I found that scipy has a loader for arff files loadarff() to load them as numpy record arrays. I am not 100% sure that those arrays are suitable for direct consumption by scikit-learn but that should get your started.

Royi
  • 4,640
  • 6
  • 46
  • 64
ogrisel
  • 39,309
  • 12
  • 116
  • 125
  • Do you think i'll need to parse those numpy arrays?... What kind of preprocess would i need to do in order to feed some classification algorithm in scikit-learn? – tumbleweed Dec 08 '14 at 21:31
  • Those looking for code to use scipy import scipy.io.arff as arff data = arff.loadarff(open(file_path,'rt')) – Jeevan Dec 22 '20 at 10:22
3

Follow renatopp's answer: assume your data is the iris dataset, there should be 5 dimensional with last one is the class label column.

s = svm.SVC()
data_input = data[:,0:4]
labels = data[:,4] # this is the class column
s.fit(data_input, labels)

I think this is something you want.

Teng Fu
  • 27
  • 3
3

Solution with scipy.arff

Code:


from scipy.io import arff
import pandas as pd

data = arff.loadarff('file.arff')
df = pd.DataFrame(data[0])
df.head()
Hissaan Ali
  • 2,229
  • 4
  • 25
  • 51
0

If your "arff" file is a text file, try the following code instead:

import arff, numpy as np
dataset = arff.loads(open('mydataset.arff', 'rt'))
data = np.array(dataset['data'])
Mehmet
  • 341
  • 2
  • 7