在Julia中使用分布式和共享DARRAY
使用Julia 1.2。我正在处理一个相当大的问题,涉及电力市场上的众多消费者,希望使用分布式计算,以便解决问题的时间更快。但我不确定布局应该是什么,我不希望工作人员一个接一个地写入共享数组。而且我的代码运行速度仍然非常慢 有人能检查一下下面的布局是否正确吗在Julia中使用分布式和共享DARRAY,julia,Julia,使用Julia 1.2。我正在处理一个相当大的问题,涉及电力市场上的众多消费者,希望使用分布式计算,以便解决问题的时间更快。但我不确定布局应该是什么,我不希望工作人员一个接一个地写入共享数组。而且我的代码运行速度仍然非常慢 有人能检查一下下面的布局是否正确吗 using Distributed n_workers = workers() if n_workers[end]<3 addprocs(3) end @everywhere using JuMP, Gurobi
using Distributed
n_workers = workers()
if n_workers[end]<3
addprocs(3)
end
@everywhere using JuMP, Gurobi, DataFrames, CSV, DelimitedFiles, SharedArrays
###I initialize many shared arrays into which I want my results to go into
Results=SharedArray{Float64,2}(length(vec4), 38)
Injection_IndPros=SharedArray{Float64,3}(length(vec4)*length(day_steps), length(time_steps), tot_consumers)
Injection_Prosumers=SharedArray{Float64,2}(length(vec4)*length(day_steps), length(time_steps))
Prosumer_caps=SharedArray{Float64,2}(length(vec4)*4, tot_households)
Year_behav=SharedArray{Float64,2}(length(vec4)*length(time_steps),6)
@sync @distributed for iter4=1:length(vec4)
###Call in a function which returns results
####Then I carry out some calculations on these results (here I also use normal matrices, I do not use @everywhere here)
####Then I put my calculated results into the shared arrays for e.g.
Results[iter4,:]=[w_max_with[1]/8, w_max_inj[1]/8, obj, Net_injection, Invest_PV, Totcap_PV, Invest_PVINV, Totcap_PVINV, Totgen_PV, Maxgen_PV, Curtailment_PV, Invest_BAT, Totcap_BAT, Invest_BATINV, Totcap_BATINV, Invest_WIND, Totcap_WIND, Totgen_WIND, Invest_Conv_Base, cap_conv_opt[1], Invest_Conv_Mid, cap_conv_opt[2], Invest_Conv_Peak, cap_conv_opt[3],Totgen_CONVBASE, Totgen_CONVMID, Totgen_CONVPEAK, Totgen_CONV, Totgen_ALL, Genshare_PV, Genshare_WIND, Genshare_CONVBASE, Genshare_CONVMID, Genshare_CONVPEAK, Yearavg_elec_price, Max_inj_pos, Avg_INVtoPV_ratio, Avg_INVtoBAT_ratio]
end #here ends the distributed for loop
使用分布式
n_工人=工人()
如果n_workers[end]在提问时,您应该提供一个最小的工作示例,而不仅仅是复制粘贴一些生产代码
下面是一个典型的SharedArrays
使用模式,与您可能尝试的操作类似:
using SharedArrays, Distributed
addprocs(3)
@everywhere using SharedArrays, Distributed
sx = SharedArray(Array(reshape(1:30,5,6)))
@everywhere function myjob(v)
sum(v)
end
@sync @distributed for i in 1:5
s = myjob(sx[i,1:end-1]) + 1000*myid()
sx[i,end] = s
end