Why is it illegal to call mean on this 3x2 matrix of doubles in Scala Breeze?

Question

I am new to Breeze (formerly Scalala), and can't figure out why the following simple program throws an exception. i am using Scala 2.9.2 and Breeze 0.1:

import breeze.linalg._
val m = DenseMatrix((3.0, 1.0, 2.0), (-2.0, 1.0, 3.0))
val n = mean(m, Axis._1)

It works as expected if I use a 2x2 or a 3x2 matrix, but a 2x3 matrix causes the following exception:

-- org.jblas ERROR Couldn't load copied link file: java.lang.UnsatisfiedLinkError: C:\Users\daved\AppData\Local\Temp\jblas8588491482847885553jblas.dll: Can't find dependent libraries.

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 6
    at breeze.linalg.package$$anon$1.apply$mcD$sp(package.scala:203)
    at breeze.linalg.package$$anon$1.apply(package.scala:196)
    at breeze.linalg.package$$anon$1.apply(package.scala:186)
    at breeze.linalg.DenseVector.ureduce(DenseVector.scala:91)
    at breeze.linalg.Tensor$$anon$1.apply(Tensor.scala:149)
    at breeze.generic.UReduceable$class.apply$mcD$sp(URFunc.scala:56)
    at breeze.linalg.Tensor$$anon$1.apply$mcD$sp(Tensor.scala:148)
    at breeze.generic.URFunc$mcD$sp$class.apply$mcD$sp(URFunc.scala:32)
    at breeze.linalg.package$$anon$1.apply$mcD$sp(package.scala:186)
    at breeze.generic.URFunc$mcD$sp$$anonfun$apply$mcD$sp$1.apply(URFunc.scala:36)
    at breeze.linalg.LowPriorityDenseMatrix1$$anon$17.apply(DenseMatrix.scala:444)
    at breeze.linalg.LowPriorityDenseMatrix1$$anon$17.apply(DenseMatrix.scala:440)
    at breeze.generic.URFunc$mcD$sp$class.apply$mcD$sp(URFunc.scala:36)
    at breeze.linalg.package$$anon$1.apply$mcD$sp(package.scala:186)
    at com.tennisedge.opencv.BreezePlay$delayedInit$body.apply(BreezePlay.scala:24)

Please show where in the documentation your input is legal, until then this is a bug in your code, not Breeze's. See http://www.codinghorror.com/blog/2008/03/the-first-rule-of-programming-its-always-your-fault.html — djechlin, Feb 15 '13 at 13:44
In the scaladoc for Breeze at http://www.scalanlp.org/api/#breeze.linalg.package it indicates `mean` is a `URFunc`. Since a DenseMatrix[Double] is reduceable, it should be legal to call this. If it is a bug in my code, I change my question to be "Which matrices are legal to call `mean` on" — Dave DeCaprio, Feb 15 '13 at 13:49
so variable m is valid, instantiated object and the problem is in the 3 line (mean) - correct? — xhudik, Feb 15 '13 at 15:06
This works fine for me on osx using SBT. It sounds like you have a dependency problem since it can't find jblas, or at least it can't access it. — Noah, Feb 15 '13 at 15:15
yes, this was my 1 guess, but when I read that it was working with some other dimensions... - not sure ... — xhudik, Feb 15 '13 at 15:18
Are correctly configured native libraries a requirement for correct operation of Breeze? — Dave DeCaprio, Feb 15 '13 at 15:55
Ok, so debugging down into this, it looks to me like a bug in DenseMatrix. Line 449: implicit def canCollapseCols[V, R:ClassManifest] = new CanCollapseAxis[DenseMatrix[V], Axis._1.type, DenseVector[V], R, DenseVector[R]] { def apply(from: DenseMatrix[V], axis: Axis._1.type)(f: (DenseVector[V]) => R): DenseVector[R] = { val result = DenseVector.zeros[R](from.rows) val t = from.t for(r <- 0 until from.cols) { result(r) = f(t(::, r)) } result } } The `0 until from.cols` should be `0 until t.cols` — Dave DeCaprio, Feb 15 '13 at 16:10

score 3 · Answer 1 · answered Feb 15 '13 at 16:23

I believe this is a bug in Breeze. In particular, the DenseMatrix canCollapseCols implicit conversion.

If I define my own implicit conversion in the local file, everyting works as expected:

implicit def myCanCollapseCols[V, R:ClassManifest] = new CanCollapseAxis[DenseMatrix[V], Axis._1.type, DenseVector[V], R, DenseVector[R]] {
  def apply(from: DenseMatrix[V], axis: Axis._1.type)(f: (DenseVector[V]) => R): DenseVector[R] = {
    val result = DenseVector.zeros[R](from.rows)
    val t = from.t
    for(r <- 0 until t.cols) { // BUG WAS HERE: was from.cols in original defintion.
      result(r) = f(t(::, r))
    }
    result
  }
}

val m = breeze.linalg.DenseMatrix((3.0, 1.0, 2.0), (-2.0, 1.0, 3.0))
val n = breeze.linalg.mean(m, breeze.linalg.Axis._1)

This is still there in head, so I will create a pull request for it in GitHub to fix. In the meantime I can just use my own implicit conversion.

score 1 · Answer 2 · edited May 23 '17 at 12:12

1

This is a perfect example (referring to this) of the problem with exceptions thrown in constructors! What is masquerading as a dependency problem is really an ArrayIndexOutOfBoundsException that was encountered in a constructor that was called during on-demand class loading. It's often very perplexing. (This is my "favorite" (most infamous) consequence of throwing exceptions in constructors.)

At least that's what it appears to be to me. I can't be sure, really.

edited May 23 '17 at 12:12

Community

1
1

answered Feb 15 '13 at 16:00

Randall Schulz

26,420
4
61
81

do you think your answer is valid even if question was based on Scala 2.9.2 and Breeze 0.1 ? – xhudik Feb 15 '13 at 16:33
@xhudik: I know nothing of Breeze / Scalala (I've heard of the old name) but I don't think my answer is in any way specific to the Scala version or the specific 3rd-party library (native or otherwise) in question. – Randall Schulz Feb 15 '13 at 16:48
@Kirk: Class-loading is usually triggered by a constructor call, right? – Randall Schulz Feb 15 '13 at 19:45
Hmm, weird. This edit appears to be confused with another. I'll revert it – Kirk Feb 15 '13 at 19:57

Why is it illegal to call mean on this 3x2 matrix of doubles in Scala Breeze?

2 Answers2