Clojure Cassandra驾驶员性能

我正在6节点Cassandra集群上测试Alia在Clojure中的性能。即使使用多线程，我也只能每秒写入400次。使用Firebrand Cassandra驱动程序和Java中的手动线程处理，我们能够以96个线程每秒5000次写入

我在使用这里的代理时做错了什么吗？在运行此功能的机器上，CPU的使用率只有约25%，看起来非常低

更新：根据Alia的建议，利用准备好的语句而不是原始语句，以同步的单线程方式实现了高达2500/秒的增益。我仍然需要使用Clojure进行多线程测试，并单独使用Alia/底层Java驱动程序中内置的异步函数，看看哪个更快

更新2：我现在看到了与下面的mpenet类似的结果，它额外利用了驱动程序内置的异步功能

(ns alia-perf-test.core
  (:gen-class)
  (:require [qbits.alia :as alia]
            [qbits.hayt :as hayt]))

(defn exec-query [session query]
  (alia/execute session (hayt/->raw query)))

(defmacro time-query
  [expr]
  `(let [start# (. System (nanoTime))
         ret# ~expr]
     (/ (double (- (. System (nanoTime)) start#)) 1000000.0)))

(defn write-entity
  [total-time session entity]
  (let [query (hayt/->raw (hayt/insert :entities (hayt/values entity) (hayt/using :timestamp 1234)))
        query-time (time-query (alia/execute session query))]
      (+ total-time query-time)))

(defn generate-entity []
  {:id (str (java.util.UUID/randomUUID)) :num 0})

(defn write-something
  [write-agent session]
  (send-off write-agent write-entity
        session 
        (generate-entity)))

(defn -main [& args]
  (let [cluster (alia/cluster ["server1"
                               "server2" 
                               "server3" 
                               "server4" 
                               "server5" 
                               "server6"]
                              :pooling-options {:core-connections-per-host [:local 16 :remote 16]
                                                :max-connections-per-host  [:local 1000 :remote 1000]
                                                :max-simultaneous-requests-per-connection [:local 32 :remote 32]
                                                :min-simultaneous-requests-per-connection [:local 16 :remote 16]})
        session (alia/connect cluster)]
    (alia/set-consistency! :any)
    (exec-query session (hayt/create-keyspace :aliaperftest
                                              (hayt/with {:replication
                                                          {:class "NetworkTopologyStrategy"
                                                           :dc1 3 :dc2 3}})))
    (exec-query session (hayt/use-keyspace :aliaperftest))
    (exec-query session (hayt/create-table :entities
                                           (hayt/column-definitions {:id :varchar
                                                                     :num :int
                                                                     :primary-key [:id]})))
    (let [num-entities 10000
          write-agent (agent 0)]
      (dotimes [n num-entities]
        (write-something write-agent session))
      (await write-agent)
      (println "Wrote" num-entities "entities in" @write-agent "ms -"
           (* (/ num-entities @write-agent) 1000.0) "ops/sec"))

    (exec-query session (hayt/drop-table :entities))
    (exec-query session (hayt/drop-keyspace :aliaperftest))
    (alia/shutdown session)
    (alia/shutdown cluster)
    (shutdown-agents)))

---

更新：我已经让它在异步模式下（使用alia的成功处理程序，使用单个节点）在8秒内完成100k个请求，使用atom接收结果并等待所有响应到达

根据您的设置和批处理，通过使用自定义执行器，也可能从中获得更高的性能，我没有走那么远。

---

请参阅：

另一个更新：从那时起，alia（和java驱动程序）使用了1.0+，在这个过程中性能显著提高（内存使用也是如此）。