Practical Clojure - Parallel Programming

简介:

Agents

Agent是一种异步数据更新的机制. 
但同时也是一种并发机制, 因为agent是基于thread pool实现的, 通过send和send-off分别发送到不同的thread pool中. 
其中send对应的thread pool中的线程个数基本等于cup核数, 所以多条send指令其实是自动以cup核数的并发度进行并发处理的.

巧妙利用这个机制就可以实现并发

Concurrency Functions

pmap

pmap是map的进化版本, 但是它对于每个集合中的元素都是提交给一个线程去执行function

pmap is partially lazy in that the entire result set is not realized unless required, but the parallel computation does run ahead of the consumption to some degree.

例子, 先定义测试函数, 在真正执行函数前先sleep 1s

(defn make-heavy [f]
  (fn [& args]
    (Thread/sleep 1000)
      (apply f args)))

通过下面时间的简单对比, 就明白差别了

user=> (time (doall (map (make-heavy inc) [1 2 3 4 5])))
"Elapsed time: 5002.96291 msecs"
(2 3 4 5 6)

user=> (time (doall (pmap (make-heavy inc) [1 2 3 4 5])))
"Elapsed time: 1031.941815 msecs"
(2 3 4 5 6)

pvalues

pvalues takes any number of expressions and returns a lazy sequence of the values of each expression, evaluated in parallel. 
user=> (pvalues (+ 5 5) (- 5 3) (* 2 4)) 
(10 2 8) 

pcalls

pcalls takes any number of no-argument functions and returns a lazy sequence of their return values, executing them in parallel. 
user=> (pcalls #(+ 5 2) #(* 2 5)) 
(7 10)

 

Overhead and Performance

并发那么好, 是不是什么情况都需要使用并发了? 
肯定不是, 并发本身也是需要overhead的, 如果线程并发带来的时间节省太少, 就得不偿失. 
看下面比较极端的例子, 用并发反而慢了那么多, 就是因为时间都花费在并发的overhead上面了

user=> (time (dorun (map inc (range 1 1000))))
"Elapsed time: 9.150946 msecs"
user=> (time (dorun (pmap inc (range 1 1000))))
"Elapsed time: 182.349073 msecs"


Futures and Promises

Futures and promises are two slightly more low-level threading constructs, inspired by the similar features available in the Java 6 concurrency API. They are simple to understand, simple to use, and provide a very direct way to spawn threads using native Clojure syntax.

Futures

A Clojure future represents a computation, running in a single thread.

user=> (def my-future (future (* 100 100)))
#'user/my-future
user=> @my-future  ;如果thread没有执行完,会阻塞
10000

 

可以看到macro future是封装future-call函数, 你也可以直接调用这个函数, 不过比较麻烦些

(defmacro future
  [& body] `(future-call (fn [] ~@body)))

future-cancel 
It is possible to attempt to cancel a future that hasn’t yet finished executing.

future-cancelled? 
future-cancelled? takes a single future as an argument and returns true if it has been cancelled.

future-done? 
future-done? takes a single future as an argument and returns true if the future’s execution is complete, otherwise false.

future? 
future? takes a single value as an argument and returns true if it is a future, otherwise false.

 

Promises

A promise is a value that may not yet exist. If a promise is dereferenced before its value is set, the dereferencing thread blocks until a value is delivered to the promise.

可以用于多线程间同步协调, 谨慎使用, 容易导致死锁

user=> (def mypromise (promise))
#'user/mypromise
user=> @mypromise  ;导致主线程阻塞

user=> (def mypromise (promise))
#'user/mypromise
user=> (deliver mypromise 5)
#<AFn$IDeref$db53459f@c0f1ec: 5>
user=> @mypromise               
5

Java-based Threading

If none of Clojure’s other concurrency tools meet your needs for any reason, there’s always the option of 
falling back to Java’s native threading capabilities.

user=> (.start (Thread. #(println "hello")))
nil
hello

本文章摘自博客园,原文发布日期:2013-02-28
目录
相关文章
|
7月前
|
算法 安全 编译器
【C++20 新特性Concepts 概念】C++20 Concepts: Unleashing the Power of Template Programming
【C++20 新特性Concepts 概念】C++20 Concepts: Unleashing the Power of Template Programming
309 0
|
2月前
|
Go C# Python
Python Tricks :Lambdas Are Single-Expression Functions 原创
Python Tricks :Lambdas Are Single-Expression Functions 原创
15 0
|
SQL 编译器 API
Efficiently Compiling Efficient Query Plans for Modern Hardware 论文解读
这应该是SQL查询编译的一篇经典文章了,作者是著名的Thomas Neumann,主要讲解了TUM的HyPer数据库中对于CodeGen的应用。 在morsel-driven那篇paper 中,介绍了HyPer的整个执行框架,会以task为单位处理一个morsel的数据,而执行的处理逻辑(一个pipeline job)就被编译为一个函数。这篇paper则具体讲如何实现动态编译。
453 0
Efficiently Compiling Efficient Query Plans for Modern Hardware 论文解读
|
JavaScript Java Go
Rise of Kotlin: The Programming Language for the Next Generation
Rise of Kotlin: The Programming Language for the Next Generation https://hackernoon.
1566 0
|
Java Go Kotlin
For an experienced software engineer, what would be a preferred new programming language to learn?
For an experienced software engineer, what would be a preferred new programming language to learn? Chanaka Fernando, knows Sin...
1118 0

热门文章

最新文章