1

I am trying to cluster my data. I load my data form excel file as you see below in my code:

dfAB1 = pd.read_excel('kucukverideneme_dfAB1.xlsx')

Then I tried to implement Complete Linkage clustering algorithm.

from scipy.cluster.hierarchy import linkage
from scipy.cluster.hierarchy import dendrogram
hc_complete = linkage(dfAB1, "complete")

model = AgglomerativeClustering(n_clusters=3, affinity='euclidean', linkage='complete')
model.fit(dfAB1)
labels_ = model.labels_

I got this error:

"MemoryError:Unable to allocate292.GİB for an array with shape (279953,279953) and data type float 64"

Afterwards, I researched the problem on Stack Overflow and I applied those below: Unable to allocate array with shape and data type.

The, after I run the algorithm again, I didn't received this error but my computer was frozen. It was not finished.

Does anyone have an idea about what should I do?

Adrian Mole
  • 49,934
  • 160
  • 51
  • 83
ALPER AYTATLI
  • 11
  • 3
  • Float, String, uint32_t, Long etc. is'nt an default data type, need handle data types before proccesing it. why using float64 ? Your system native float unit probably *FLOAT32* but you trying a lot OWERFLOW methot. Another opinion `Python not a best computing language !` – dsgdfg Sep 30 '21 at 07:35
  • My system is 64 bit thats why i used float 64, and i need not integer values for analysis. Thats why i used float 64. What would you offer as datatype to use? Float32? i can try. Secondly, as a best computing language, what would you offer? – ALPER AYTATLI Sep 30 '21 at 08:37
  • `Σ` operations some time required `owerflow` handle, so if you try compute whole data at same time need becarefuly `system and variable` limits. indeed assembly language never allow multiple float vs long or another non-equal types. You using 64bit OS but are you sure all modules or packages is 64 bit ? If haven't any problem divide your data to `N` parts maybe will be work. `C` is computing language (you can't use any *pandora box* on as variable parameters). My test `CPython + C` 5 times faster than `Python`. i hope helpfull, have nice day. – dsgdfg Oct 04 '21 at 06:20
  • I'll take into consideration. I'm also trying to use virtual machines such as Colab. Thank you. – ALPER AYTATLI Oct 05 '21 at 12:50

0 Answers0