Parallel processing 如何并行化此代码?

Parallel processing 如何并行化此代码?,parallel-processing,rust,Parallel Processing,Rust,我最初是用ruby写的,后来发现MRI根本不支持并行执行。所以我用rust重新写了一遍,但在不同作品的所有权上遇到了麻烦 extern crate rand; extern crate csv; extern crate num_cpus; use std::fs::File; use csv::Reader; use rand::{thread_rng, sample}; use std::thread; use std::str::from_utf8; use std::io::{self

我最初是用ruby写的,后来发现MRI根本不支持并行执行。所以我用rust重新写了一遍,但在不同作品的所有权上遇到了麻烦

extern crate rand;
extern crate csv;
extern crate num_cpus;

use std::fs::File;
use csv::Reader;
use rand::{thread_rng, sample};
use std::thread;
use std::str::from_utf8;
use std::io::{self, Write};
use csv::index::{Indexed, create_index};

fn met_n_in_common(n:usize,csv:&mut Reader<File>)->usize{
    csv.byte_records().map(|r| if(from_utf8(r.unwrap().get(n).unwrap()).unwrap() == "TRUE"){1}else{0}).fold(0usize, |sum, i| sum + i)
}

fn mets_in_common(csv:&mut Reader<File>,current_set_length:usize)->usize {
    (0..csv.headers().unwrap().len()).map(|i| if(i == 0){0}else{met_n_in_common(i,csv)} ).filter(|&e| e==current_set_length ).count()
}

fn main() {

    let csv_s = || csv::Reader::from_file("/Users/camdennarzt/Documents/All 7000 series-Table 1-1-1-3.csv").unwrap();
    let mut csv = csv_s();
    let mut index_data = io::Cursor::new(Vec::new());
    create_index(csv_s(), index_data.by_ref()).unwrap();
    let mut index = Indexed::open(csv_s(), index_data).unwrap();

    let mut tried_indices = Vec::new();
    let mut threads : Vec<_> = (0..num_cpus::get()).map(|i|{
        thread::spawn(move || {
            let mut best_set : Vec<Vec<String>> = Vec::new();
            let mut best_count = 0;
            let mut rng = thread_rng();
            let mut indices = Vec::new();
            let limit = 2usize.pow(10)/num_cpus::get();
            for _ in (0..limit) {
                while {
                    let count = *sample(&mut rng, 13..83, 1).first().unwrap();
                    indices = sample(&mut rng, 1..83, count);
                    tried_indices.contains(&indices)
                }{}
                tried_indices.push(indices.to_owned());

                let current_set:Vec<_> = indices.iter().map(|&i|{
                    index.seek(i).unwrap();
                    index.records().next().unwrap().unwrap()
                }).collect();
                let current_count = mets_in_common(&mut csv,current_set.len());
                if (current_count > best_count){
                    best_count = current_count;
                    best_set = current_set;
                }
            }
            (best_count,best_set.iter().map(|r| *r.first().unwrap()).collect::<Vec<String>>())
        })
    }).collect();
}
extern板条箱兰德;
外部板条箱;
外部机箱数量(CPU);
使用std::fs::File;
使用csv::Reader;
使用rand::{thread\rng,sample};
使用std::线程;
使用std::str::from_utf8;
使用std::io::{self,Write};
使用csv::index::{index,create_index};
fn符合通用标准(n:usize、csv:&mut Reader)->usize{
csv.byte_records().map(| r|if(从_utf8(r.unwrap().get(n.unwrap()).unwrap()=“TRUE”){1}否则{0})。折叠(0usize,| sum,i | sum+i)
}
fn mets_in_common(csv:&mut读取器,当前设置_长度:usize)->usize{
(0..csv.headers().unwrap().len()).map(|i | if(i==0){0}else{met_n_in_common(i,csv)}).filter(|&e | e==当前设置长度).count()
}
fn main(){
让csv|s=|csv::Reader::from|u文件(“/Users/camdennarzt/Documents/All 7000 series Table 1-1-1-3.csv”).unwrap();
让mut csv=csv_s();
让mut index_data=io::Cursor::new(Vec::new());
创建索引(csv_s(),索引_data.by_ref()).unwrap();
让mut index=index::open(csv_s(),index_data).unwrap();
让mut\u index=Vec::new();
让mut线程:Vec=(0..num_CPU::get()).map(| i|{
线程::生成(移动| |{
设mut best_set:Vec=Vec::new();
让mut best_count=0;
设mut rng=thread_rng();
让mut index=Vec::new();
让limit=2usize.pow(10)/num_CPU::get();
对于uuin(0..limit){
当{
let count=*sample(&mut rng,13..83,1).first().unwrap();
指数=样本(&mut rng,1..83,计数);
已尝试索引。包含(&索引)
}{}
已尝试将索引推送(index.to_owned());
让当前设置:Vec=index.iter().map(|&i)|{
index.seek(i.unwrap();
index.records().next().unwrap().unwrap()
}).收集();
让current_count=mets_in_common(&mut csv,current_set.len());
如果(当前计数>最佳计数){
最佳计数=当前计数;
最佳_集=当前_集;
}
}
(最佳计数,最佳集合.iter().map(| r |*r.first().unwrap()).collect::())
})
}).收集();
}
具体来说,当我编译这个(rust 1.2 stable)时,我得到:

-*-模式:编译;默认目录:“~/Developer/Rust/optimal\u subset\u finder/src/”-*-
编译于8月19日星期三14:55:10开始
货物建造
编译最佳_子集_查找器v0.1.0(file:///Users/camdennarzt/Developer/Rust/optimal_subset_finder)
main.rs:31:23:56:10错误:无法移出'FnMut'闭包中捕获的外部变量
main.rs:31线程::繁殖(移动| |{
main.rs:32让mut-best_-set:Vec=Vec::new();
main.rs:33让mut最佳计数=0;
main.rs:34让mut rng=thread_rng();
rs:35让mut index=Vec::new();
main.rs:36 let limit=2usize.pow(10)/num_CPU::get();
...
注:在闭合膨胀的膨胀中
main.rs:31:23:56:10注:扩建场地
注:在闭合膨胀的膨胀中
主体:rs:30:57:57:6注:扩建场地
main.rs:31:23:56:10错误:无法移出'FnMut'闭包中捕获的外部变量
main.rs:31线程::繁殖(移动| |{
main.rs:32让mut-best_-set:Vec=Vec::new();
main.rs:33让mut最佳计数=0;
main.rs:34让mut rng=thread_rng();
rs:35让mut index=Vec::new();
main.rs:36 let limit=2usize.pow(10)/num_CPU::get();
...
注:在闭合膨胀的膨胀中
main.rs:31:23:56:10注:扩建场地
注:在闭合膨胀的膨胀中
主体:rs:30:57:57:6注:扩建场地
main.rs:31:23:56:10错误:无法移出'FnMut'闭包中捕获的外部变量
main.rs:31线程::繁殖(移动| |{
main.rs:32让mut-best_-set:Vec=Vec::new();
main.rs:33让mut最佳计数=0;
main.rs:34让mut rng=thread_rng();
rs:35让mut index=Vec::new();
main.rs:36 let limit=2usize.pow(10)/num_CPU::get();
...
注:在闭合膨胀的膨胀中
main.rs:31:23:56:10注:扩建场地
注:在闭合膨胀的膨胀中
主体:rs:30:57:57:6注:扩建场地
main.rs:55:49:55:68错误:无法移出借用内容
main.rs:55(最佳计数,最佳集合.iter().map(|r |*r.first().unwrap())。收集:())
^~~~~~~~~~~~~~~~~~~
注:在闭合膨胀的膨胀中
main.rs:55:45:55:68注:扩建场地
注:在闭合膨胀的膨胀中
main.rs:31:23:56:10注:扩建场地
注:在闭合膨胀的膨胀中
主体:rs:30:57:57:6注:扩建场地
错误:由于之前的4个错误而中止
无法编译“最优子集查找器”。
要了解更多信息,请使用--verbose再次运行该命令。
编译在8月19日星期三14:55:10异常退出,代码为101

如果我完全注释掉线程,那么它将编译并运行。但是我无法解析文档以使线程正常工作。而且,当需要知道如何解决此问题时,这些错误也没有特别大的帮助。

您的代码中有几个问题

首先,您显示的实际错误是由于您试图从
&Vec
中获取
字符串
。如果不进行克隆,这是不可能的,因为这需要将
字符串
&Vec
中移出,但您无法移出引用的数据。因为
最佳设置
在它是一个内部线程,在您从它返回数据后立即被删除,您可以安全地使用它
-*- mode: compilation; default-directory: "~/Developer/Rust/optimal_subset_finder/src/" -*-
Compilation started at Wed Aug 19 14:55:10

cargo build
   Compiling optimal_subset_finder v0.1.0 (file:///Users/camdennarzt/Developer/Rust/optimal_subset_finder)
main.rs:31:23: 56:10 error: cannot move out of captured outer variable in an `FnMut` closure
main.rs:31         thread::spawn(move || {
main.rs:32             let mut best_set : Vec<Vec<String>> = Vec::new();
main.rs:33             let mut best_count = 0;
main.rs:34             let mut rng = thread_rng();
main.rs:35             let mut indices = Vec::new();
main.rs:36             let limit = 2usize.pow(10)/num_cpus::get();
           ...
note: in expansion of closure expansion
main.rs:31:23: 56:10 note: expansion site
note: in expansion of closure expansion
main.rs:30:57: 57:6 note: expansion site
main.rs:31:23: 56:10 error: cannot move out of captured outer variable in an `FnMut` closure
main.rs:31         thread::spawn(move || {
main.rs:32             let mut best_set : Vec<Vec<String>> = Vec::new();
main.rs:33             let mut best_count = 0;
main.rs:34             let mut rng = thread_rng();
main.rs:35             let mut indices = Vec::new();
main.rs:36             let limit = 2usize.pow(10)/num_cpus::get();
           ...
note: in expansion of closure expansion
main.rs:31:23: 56:10 note: expansion site
note: in expansion of closure expansion
main.rs:30:57: 57:6 note: expansion site
main.rs:31:23: 56:10 error: cannot move out of captured outer variable in an `FnMut` closure
main.rs:31         thread::spawn(move || {
main.rs:32             let mut best_set : Vec<Vec<String>> = Vec::new();
main.rs:33             let mut best_count = 0;
main.rs:34             let mut rng = thread_rng();
main.rs:35             let mut indices = Vec::new();
main.rs:36             let limit = 2usize.pow(10)/num_cpus::get();
           ...
note: in expansion of closure expansion
main.rs:31:23: 56:10 note: expansion site
note: in expansion of closure expansion
main.rs:30:57: 57:6 note: expansion site
main.rs:55:49: 55:68 error: cannot move out of borrowed content
main.rs:55             (best_count,best_set.iter().map(|r| *r.first().unwrap()).collect::<Vec<String>>())
                                                           ^~~~~~~~~~~~~~~~~~~
note: in expansion of closure expansion
main.rs:55:45: 55:68 note: expansion site
note: in expansion of closure expansion
main.rs:31:23: 56:10 note: expansion site
note: in expansion of closure expansion
main.rs:30:57: 57:6 note: expansion site
error: aborting due to 4 previous errors
Could not compile `optimal_subset_finder`.

To learn more, run the command again with --verbose.

Compilation exited abnormally with code 101 at Wed Aug 19 14:55:10
best_set.into_iter()
    .flat_map(|r| r.take(1))
    .collect::<Vec<String>>()
let index = Arc::new(Mutex::new(Indexed::open(csv_s(), index_data).unwrap()));

...
let index = index.clone();
thread::spawn(move || {
    ...

    let current_set:Vec<_> = {
        let index = index.lock();
        indices.iter().map(|&i| {
            index.seek(i).unwrap();
            index.records().next().unwrap().unwrap()
        }).collect()
    };
    ...
});
let mut csv = csv_s();
thread::spawn(move || {
    ...
});