Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/clojure/3.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/1/cocoa/3.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Clojure错误-超出GC开销限制_Clojure_Garbage Collection_Large Files_Fasta - Fatal编程技术网

Clojure错误-超出GC开销限制

Clojure错误-超出GC开销限制,clojure,garbage-collection,large-files,fasta,Clojure,Garbage Collection,Large Files,Fasta,我试图随机抽取一个大的FASTQ文件,并将其写入standard out。我不断收到“超出GC开销限制”错误,我不确定我做错了什么。我曾尝试在莱宁根增加Xmx,但没有效果。这是我的密码: (ns fastq-sample.core (:gen-class) (:use clojure.java.io)) (def n-read-pair-lines 8) (defn sample? [sample-rate] (> sample-rate (rand))) ; ; Age

我试图随机抽取一个大的FASTQ文件,并将其写入standard out。我不断收到“超出GC开销限制”错误,我不确定我做错了什么。我曾尝试在莱宁根增加Xmx,但没有效果。这是我的密码:

(ns fastq-sample.core
  (:gen-class)
  (:use clojure.java.io))

(def n-read-pair-lines 8)

(defn sample? [sample-rate]
  (> sample-rate (rand)))

;
; Agent for writing the reads asynchronously
;

(def wtr (agent (writer *out*)))

(defn write-out [r]
  (letfn [(write [out msg] (.write out msg) out)]
    (send wtr write r)))

(defn write-close []
  (send wtr #(.close %))
  (await wtr))

;
; Main
;

(defn reads [file]
  (->>
    (input-stream file)
    (java.util.zip.GZIPInputStream.)
    (reader)
    (line-seq)))

(defn -main [fastq-file sample-rate-str]
  (let [sample-rate (Float. sample-rate-str)
        in-reads    (partition n-read-pair-lines (reads fastq-file))]
    (doseq [x (filter (fn [_] (sample? sample-rate)) in-reads)]
      (write-out (clojure.string/join "\n" x)))
    (write-close)
    (shutdown-agents)))

当我试图将一个无限序列合并成一个简单的数据结构(如地图或向量)时,我经常会遇到同样的症状。这通常意味着内存紧张,垃圾收集器无法满足对新对象的需求。wtr代理很可能太大,无法存储内存。也许您不希望通过更改来将打印结果存储在atom中

(write [out msg] (.write out msg) out)


我认为这只会在第一次将输出作为代理的状态写入时起作用,并将输出更改为write to out函数的结果。我是基于此编写代码的-
(write [out msg] (.write out msg))