Java : a method to do multiple calculations on arrays quickly

Question

Sorry if this isn't the right place for these questions, but I'm in need of some base help.

I have a class called Differential and that has a member variable mValues (List>). What I want to do is iterate through all the values and compare them with each other. This means for five values I am doing 10 comparisons. However this is going to be used for at least 20,000 lists. There are three calculations that I want to do and I was wondering how I should approach this.

My current idea comes from this multi-threading example

I was thinking that I would use a completionService to iterate through the values in multi-threading. Then each thread would deal with the multiple calculations and return them. Does this seem appropriate? Is there a way of doing this somewhat quickly that I'm not realizing?

If you are allowed to use Java 8 then it might be worth having a look on the new lambda expressions — mschenk74, Jul 23 '14 at 20:33
Do you **have** to perform all the comparisons? Meaning, can the final point you want to reach be derived by performing a sub-set of comparisons? — noMAD, Jul 23 '14 at 20:36
To be able to use parallelity with lambdas and stream operations the first step is to formulate the problem in a way where the single steps can be parallelized. Then each step has to be implemented as some map or reduce task. But to be able to do that one should have at least some basic experience with lambdas and streams in java — mschenk74, Jul 23 '14 at 20:37
I don't believe so. I think I've optimized the comparisons as much as possible and it still needs all the data at once. If I have 5 lists it needs to compare 1-2,1-3,1-4,1-5,2-3,2-4,2-5,3-4,3-5,4-5. I do know the basics of lamnda and streams. — Sam, Jul 23 '14 at 20:38
So basically you are saying you cannot go below O(n^2). I can give you a clojure example of what mschenk is talking about. You do, `(map (fn [] your function) list)` where your `fn` (which is comparison here) is performed over every element in the list. — noMAD, Jul 23 '14 at 20:41
Could you give me some kind of example of this method? I'd love to try it out. — Sam, Jul 23 '14 at 20:44
Your question is really broad -- it's basically "how does concurrency work?" -- and you don't tell us how or why you are comparing values in the list. I think a concrete example (in code) of what you are doing is in order. — markspace, Jul 23 '14 at 20:50
Well, here is a simple example that will add `1` to each element in the list. `(map (fn [ele] (+ 1 ele)) [1 2 3 4 5])`. This will print `(2 3 4 5 6)`. I have never used java 8 so I can't tell how but am sure you can. Also, this won't necessarily decrease computation time, for that you would need to parallelize it like how hadoop does. — noMAD, Jul 23 '14 at 20:51
There are 20,000 lists of varying length, and for each item in each list you have to do 3 calculations? What is the average list length? Unless it's on the order of thousands, I just don't see where you'll need multiple threads. 60,000 * 1,000 is only 60 million--a fairly small number unless those calculations are very complex. — Jim Mischel, Jul 24 '14 at 18:53

score 1 · Answer 1 · edited May 23 '17 at 11:57

Before you make the multi-thread version, I would suggest making the single thread version work and show us the code. Then you can split the work of a for loop (sequence) easily. Lots of details omitted below.

mysequence = ...//array or list
//2 threads
c1=new Callable(){
Object call(){
  for(i=0...mysequence.size()/2)
     //do work
   }
return result;
}
c2=new Callable(){
Object call(){
  for(i=mysequence.size()/2+1 ... mysequence.size()-1)
     //do work
   }
return result;
}
ExecutorService exec=Executors.xxxThreadPool(2);
fut1= exec.submit(c1)
fut2 =exec.submit(c2)

fut1.get()
fut2.get()

Let me find a link to another post of mine. See my answer here for a better example. Thread won't naturally exit at end of run()

Java : a method to do multiple calculations on arrays quickly

1 Answers1

Linked