When processed in parallel, one process is heavy.

When running in parallel, one processing becomes heavier.

$scala
Welcome to Scala version 2.11.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_25).
Type in expressions to have them validated.
Type—help for more information.

// a function that measures processing time 
scala>deferred(f:=>Unit)={vals=new java.util.Date;f;println("time:"+(new java.util.Date().getTime-s.getTime)+"ms")}
elapsed: (f:=>Unit) Unit

// something heavy to dealings
scala>def=(1 to 1000000).map(_+scala.util.Random.nextInt(10000))
f:scala.collection.imutable.IndexedSeq [Int]

// Perform 30 heavy operations normally
scala>(1 to 30).foreach {_=>elapped(f)}
time —166 ms
time —84 ms
time —36 ms
time —35 ms
time —64 ms
time —55 ms
time —52 ms
time —55 ms
time —54 ms
time —54 ms
time —58 ms
time —54 ms
time —52 ms
time —53 ms
time —51 ms
time —56 ms
time —57 ms
time —52 ms
time —63 ms
time —56 ms
time —54 ms
time —70 ms
time —72 ms
time —55 ms
time —119 ms
time —106 ms
time —99 ms
time —86 ms
time —93 ms
time —61 ms

// Now run in parallel with par
scala>(1 to 30).par.foreach {_=>elapped(f)}
time —201 ms
time —212 ms
time —215 ms
time —221 ms
time —176 ms
time —217 ms
time —226 ms
time —251 ms
time —242 ms
time —230 ms
time: 235ms
time —268 ms
time —238 ms
time: 235ms
time: 235ms
time —383 ms
time —372 ms
time —382 ms
time —396 ms
time —414 ms
time —407 ms
time —337 ms
time —324 ms
time —354 ms
time —390 ms
time —407 ms
time —452 ms
time —159 ms
time —147 ms
time —37 ms

As for the example in par, it was similar in Future.

The number of cores is four, so the parallelism is four, but if you increase the parallelism further, it becomes heavier in proportion.

The processing time will be 10 times longer, so 10 seconds will be 100 seconds...

"I sometimes wonder if it's a natural phenomenon like ""PC suddenly gets heavy when you run various apps"", but I don't really understand..."

Is there any workaround?

Please let me know if you know more.I look forward to your kind cooperation.

scala

2022-09-29 21:37

2 Answers

scala.util.Random uses java.util.Random and is not suitable for simultaneous access from multiple threads.
(1 to 1000000).map(...) reserves a lot of memory.

That's why the performance doesn't seem to come out.Rewriting to ThreadLocalRandom and foreach may have overhead, but it is better than the example in the question.

scala>deff5=(1 to 1000000).foreach(_=>java.util.concurrent.ThreadLocalRandom.current().nextInt(10000))

f5 —Units

scala>(1 to 30).foreach {_=>elapped(f5)}
time —21 ms
time —31 ms
time —19 ms
time —22 ms
time —17 ms
time —17 ms
time —20 ms
time —19 ms
time —24 ms
time —21 ms
time —17 ms
time —21 ms
time —17 ms
time —20 ms
time —17 ms
time —17 ms
time —29 ms
time —25 ms
time —25 ms
time —17 ms
time —17 ms
time —30 ms
time —36 ms
time —19 ms
time —17 ms
time —18 ms
time —19 ms
time —17 ms
time —20 ms
time —16 ms

scala>(1 to 30).par.foreach {_=>elapped(f5)}
time —53 ms
time —53 ms
time —54 ms
time —56 ms
time —62 ms
time —63 ms
time —69 ms
time —74 ms
time —51 ms
time —43 ms
time —51 ms
time —50 ms
time —59 ms
time —43 ms
time —55 ms
time —50 ms
time —37 ms
time —40 ms
time —43 ms
time —42 ms
time —37 ms
time —46 ms
time —41 ms
time —35 ms
time —32 ms
time —34 ms
time —37 ms
time —32 ms
time —30 ms
time —30 ms

2022-09-29 21:37

A purely computationally intensive task will make the expected move.

Adjust the

//weight value to match the processing performance.I think around 30 would be good.
def (weight: Int): Unit = {
  deffib(n:Int):Int=n match{
    case0 = > 1
    case1 = > 1
    case_=>fib(n-2)+fib(n-1)
  }
  fib(weight)
}

// And I'm going to make it a display function that also shows the Thread ID.
defelapsed(f:=>Unit) = {
    value = new java.util.Date;
    f;
    println("thread:"+Thread.currentThread().getId()+"\ttime:\t"+(new java.util.Date().getTime-s.getTime)+"ms")
}

// The outer elapped shows the total at the end
scala>elapped (1 to 30).foreach {_=>elapped(f(30))})
thread : 1 time : 5ms
thread : 1 time : 4ms
thread : 1 time : 5ms
thread : 1 time : 5ms
thread : 1 time : 5ms
thread : 1 time : 5ms
thread : 1 time : 5ms
thread : 1 time : 5ms
thread : 1 time : 4ms
thread : 1 time : 5ms
thread : 1 time : 5ms
thread : 1 time : 5ms
thread : 1 time : 5ms
thread : 1 time : 5ms
thread : 1 time : 5ms
thread : 1 time : 5ms
thread : 1 time : 5ms
thread: 1 time: 6 ms
thread: 1 time: 6 ms
thread: 1 time: 7 ms
thread : 1 time : 5ms
thread : 1 time : 5ms
thread : 1 time : 5ms
thread : 1 time : 5ms
thread : 1 time : 5ms
thread : 1 time : 5ms
thread : 1 time : 5ms
thread : 1 time : 4ms
thread : 1 time : 5ms
thread : 1 time : 5ms
thread: 1 time: 152 ms

scala>elapped (1 to 30).par.foreach {_=>elapped(f(30))})
thread:56 time:5ms
thread:58 time:5ms
thread:49 time:5ms
thread:56 time:5ms
thread:58 time:4ms
thread:49 time:5ms
thread:54 time:13 ms
thread:56 time:5ms
thread:58 time:5ms
thread:49 time:4ms
thread:54 time:5ms
thread:56 time:4ms
thread:58 time:5ms
thread:49 time:5ms
thread:54 time:4ms
thread:56 time:5ms
thread:49 time:5ms
thread:58 time:5ms
thread:55 time:5ms
thread:56 time:5ms
thread:49 time:5ms
thread:58 time:5ms
thread:55 time:5ms
thread:56 time:5ms
thread:55 time:5ms
thread:57 time:4ms
thread:56 time:4ms
thread:59 time:4ms
thread:58 time:9 ms
thread:49 time:9 ms
thread:1 time:42 ms

(Don't worry about the large thread ID in the sample output because it's only a larger ID after a few runs.Parallelism 8)

Originally, the delay seems to be due to the behavior of scala.util.Random.If you need to use Random for real-world applications, you need to investigate the cause as you implement it.
If you just want to see the behavior of parallelism instead, it's safer not to use things that you don't know about.

2022-09-29 21:37

If you have any answers or tips

Popular Tags

python x 4647

android x 1593

java x 1494

javascript x 1427

c x 927

c++ x 878

ruby-on-rails x 696

php x 692

python3 x 685

html x 656