3
\$\begingroup\$

I have been trying to implement parallel merge sort in Scala. My current implementation is about as fast as .sorted, but I have no idea what would be the right aproach to parallelize it.

Input file with 1.2M integers:

  • 1.333580 seconds (my implementation)
  • 1.439293 seconds (.sorted)

How should I parallelize this?

object Mergesort extends App { //===================================================================================================================== // UTILITY implicit object comp extends Ordering[Any] { def compare(a: Any, b: Any) = { (a, b) match { case (a: Int, b: Int) => a compare b case (a: String, b: String) => a compare b case _ => 0 } } } //===================================================================================================================== // MERGESORT val THRESHOLD = 30 def inssort[A](a: Array[A], left: Int, right: Int): Array[A] = { for (i <- (left+1) until right) { var j = i val item = a(j) while (j > left && comp.lt(item,a(j-1))) { a(j) = a(j-1) j -= 1 } a(j) = item } a } def mergesort_merge[A](a: Array[A], temp: Array[A], left: Int, right: Int, mid: Int) : Array[A] = { var i = left var j = right while (i < mid) { temp(i) = a(i); i+=1; } while (j > mid) { temp(i) = a(j-1); i+=1; j-=1; } i = left j = right-1 var k = left while (k < right) { if (comp.lt(temp(i), temp(j))) { a(k) = temp(i); i+=1; k+=1; } else { a(k) = temp(j); j-=1; k+=1; } } a } def mergesort_split[A](a: Array[A], temp: Array[A], left: Int, right: Int): Array[A] = { if (right-left == 1) a if ((right-left) > THRESHOLD) { val mid = (left+right)/2 mergesort_split(a, temp, left, mid) mergesort_split(a, temp, mid, right) mergesort_merge(a, temp, left, right, mid) } else inssort(a, left, right) } def mergesort[A: ClassTag](a: Array[A]): Array[A] = { val temp = new Array[A](a.size) mergesort_split(a, temp, 0, a.size) } } 
\$\endgroup\$
3
  • \$\begingroup\$Disclaimer: I don't know Scala. You seem to be creating an awful lot of Lists and Futures. Whenever you create a Future it has to go onto the thread pool and wait until it can be processed, that's a lot of overhead. Ideally you would split the whole task into Runtime.availableProcessors futures to save on thread scheduling overhead. Scala List is an immutable linked list and has terrible random access times (presumably used when splitting).\$\endgroup\$
    – Johnbot
    CommentedOct 22, 2015 at 13:08
  • \$\begingroup\$Ok, I rewrote the piece of code and now it's almost as fast as sorted. Now parallelizing is the only thing missing\$\endgroup\$
    – warbaque
    CommentedOct 25, 2015 at 0:23
  • \$\begingroup\$I implemented alternative mergesort using courses.cs.washington.edu/courses/cse373/13wi/lectures/03-13/… as a base. Single threaded performance is slower than mine, but it's easier to parallelize. bpaste.net/show/f93cf489ca08 I still don't know what would be the correct way to use scala futures. Currently I just fork,wait and merge\$\endgroup\$
    – warbaque
    CommentedOct 25, 2015 at 2:53

0

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.