Ruby on rails 使用并行Gem创建对象_Ruby On Rails_Multithreading_Concurrency_Parallel Processing_Gem

Ruby on rails 使用并行Gem创建对象

ruby-on-rails multithreading concurrency parallel-processing

Ruby on rails 使用并行Gem创建对象,ruby-on-rails,multithreading,concurrency,parallel-processing,gem,Ruby On Rails,Multithreading,Concurrency,Parallel Processing,Gem,我试图在我的项目中引入一些并发性我现在正在玩，在并行进程中创建对象时遇到了一个问题 Parallel.map(["a"], :in_processes => 8) do |i| ActiveRecord::Base.connection_pool.with_connection do User.create end end 结果： /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8

我试图在我的项目中引入一些并发性

我现在正在玩，在并行进程中创建对象时遇到了一个问题

Parallel.map(["a"], :in_processes => 8) do |i|
  ActiveRecord::Base.connection_pool.with_connection do
    User.create
  end
end

结果：

/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:279:in `dump': no _dump_data is defined for class Binding (TypeError)
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:279:in `process_incoming_jobs'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:257:in `block in worker'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:250:in `fork'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:250:in `worker'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:238:in `block in create_workers'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:237:in `each'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:237:in `create_workers'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:201:in `work_in_processes'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:106:in `map'
    from (irb):15:in `irb_binding'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/workspace.rb:80:in `eval'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/workspace.rb:80:in `evaluate'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/context.rb:254:in `evaluate'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:159:in `block (2 levels) in eval_input'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:273:in `signal_status'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:156:in `block in eval_input'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/ruby-lex.rb:243:in `block (2 levels) in each_top_level_statement'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/ruby-lex.rb:229:in `loop'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/ruby-lex.rb:229:in `block in each_top_level_statement'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/ruby-lex.rb:228:in `catch'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/ruby-lex.rb:228:in `each_top_level_statement'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:155:in `eval_input'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:70:in `block in start'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:69:in `catch'
    from /Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:69:in `start'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/railties-3.2.13/lib/rails/commands/console.rb:47:in `start'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/railties-3.2.13/lib/rails/commands/console.rb:8:in `start'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/railties-3.2.13/lib/rails/commands.rb:41:in `<top (required)>'
    from script/rails:6:in `require'
    from script/rails:6:in `<main>'
Parallel::DeadWorker: Parallel::DeadWorker
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:51:in `rescue in work'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:48:in `work'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:216:in `block (4 levels) in work_in_processes'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:352:in `with_instrumentation'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:215:in `block (3 levels) in work_in_processes'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:210:in `loop'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:210:in `block (2 levels) in work_in_processes'
    from /Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:65:in `block (2 levels) in in_threads'

/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:279:在“dump”中：没有为类绑定定义“dump”数据（TypeError）
来自/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:279:in'process\u incoming\u jobs'
来自/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:257:in'block in worker'
来自/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:250:in'fork'
来自/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:250:in'worker'
来自/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:238:in`block in create_workers'
来自/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:237:in'each'
来自/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:237:in'create_workers'
来自/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:201:in“工作中的过程”
来自/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:106:in'map'
from（irb）：15:in'irb_binding'
from/Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/workspace.rb:80:in'eval'
来自/Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/workspace.rb:80:in“evaluate”
from/Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/context.rb:254:in“evaluate”
from/Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:159:in“eval_输入中的块（2级）”中
from/Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:273:处于“信号状态”
from/Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:156:in'block in eval_input'
from/Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/ruby-lex.rb:243:in‘每个顶级语句中的块（2个级别）’
from/Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/ruby-lex.rb:229:in'loop'
from/Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/ruby-lex.rb:229:in“每个顶级语句中的块”
from/Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/ruby-lex.rb:228:in“catch”
from/Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb/ruby-lex.rb:228:“每个顶级声明”
from/Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:155:in'eval_input'
from/Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:70:in'block in start'
from/Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:69:in'catch'
from/Users/tyler/.rvm/rubies/ruby-1.9.3-p392/lib/ruby/1.9.1/irb.rb:69:in'start'
来自/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/railties-3.2.13/lib/rails/commands/console.rb:47:in'start'
来自/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/railties-3.2.13/lib/rails/commands/console.rb:8:in'start'
来自/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/railties-3.2.13/lib/rails/commands.rb:41:in`'
来自脚本/rails:6:in'require'
来自脚本/rails:6:in`'
Parallel:：DeadWorker:Parallel:：DeadWorker
来自/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:51：“工作中的救援”
来自/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:48:in'work'
来自/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:216:in`block（4个级别）in work_in_进程'
来自/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:352:in“with_instrumentation”
来自/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:215:in `工作中的块（3个级别）\进程中的块'
来自/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:210:in'loop'
来自/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:210:in`block（2个级别）in work\u in\u进程'
来自/Users/tyler/.rvm/gems/ruby-1.9.3-p392@arcsite_mysql/gems/parallel-0.8.1/lib/parallel.rb:65:in'block（2层）in_threads'

平行块的任何变化似乎都会导致相同的错误。删除对

create

的调用将允许程序按预期运行。

看起来您返回的内容是不可转储的，因此要么返回nil，要么使用Parallel。或者，with_连接实际上不会返回用户对象，而是返回一些奇怪的内容。。。您还可以尝试返回user.attributes，而不是完整的用户对象

根据上述Mario的帖子，PostgreSQL不允许对多个线程使用同一连接。我想知道MySQL是否是相同的，加上MySQL可能会在访问它时锁定它。下面是在PostgreSQL上并行工作的解决方案：ActiveRecord:：Base.connection.reconnect