Parallel processing 布拉斯诉。Julia SharedArray对象的并行更新
我对在科学计算项目中使用JuliaParallel processing 布拉斯诉。Julia SharedArray对象的并行更新,parallel-processing,julia,blas,Parallel Processing,Julia,Blas,我对在科学计算项目中使用JuliaSharedArrays感兴趣。我当前的实现对所有矩阵向量操作都要求BLAS,但我认为也许SharedArray会在多核机器上提供一些加速。我的想法是简单地逐个索引更新输出向量索引,将索引更新到工作进程 以前关于SharedArrays和关于共享内存对象的讨论没有就此问题提供明确的指导。从直觉上看,这似乎足够简单,但在测试之后,我有点困惑为什么这种方法工作得如此糟糕(请参阅下面的代码)。对于初学者来说,@parallel For似乎分配了大量内存。如果我在循环前
SharedArray
s感兴趣。我当前的实现对所有矩阵向量操作都要求BLAS,但我认为也许SharedArray
会在多核机器上提供一些加速。我的想法是简单地逐个索引更新输出向量索引,将索引更新到工作进程
以前关于SharedArray
s和关于共享内存对象的讨论没有就此问题提供明确的指导。从直觉上看,这似乎足够简单,但在测试之后,我有点困惑为什么这种方法工作得如此糟糕(请参阅下面的代码)。对于初学者来说,@parallel For
似乎分配了大量内存。如果我在循环前面加上@sync
,如果以后需要整个输出向量,这似乎是一件明智的事情,那么并行循环的速度会大大降低(尽管没有@sync
,循环速度会非常快)
我是否错误地解释了SharedArray
对象的正确用法?或者也许我没有有效地分配计算
### test for speed gain w/ SharedArray vs. Array ###
# problem dimensions
n = 10000; p = 25000
# set BLAS threads; 64 seems reasonable in testing
blas_set_num_threads(64)
# make normal Arrays
x = randn(n,p)
y = ones(p)
z = zeros(n)
# make SharedArrays
X = convert(SharedArray{Float64,2}, x)
Y = convert(SharedArray{Float64,1}, y)
Z = convert(SharedArray{Float64,1}, z)
# run BLAS.gemv! on Arrays twice, time second case
BLAS.gemv!('N', 1.0, x, y, 0.0, z)
@time BLAS.gemv!('N', 1.0, x, y, 0.0, z)
# does BLAS work equally well for SharedArrays?
# check timing result and ensure same answer
BLAS.gemv!('N', 1.0, X, Y, 0.0, Z)
@time BLAS.gemv!('N', 1.0, X, Y, 0.0, Z)
println("$(isequal(z,Z))") # should be true
# SharedArrays can be updated in parallel
# code a loop to farm updates to worker nodes
# use transposed X to place rows of X in columnar format
# should (hopefully) help with performance issues from stride
Xt = X'
@parallel for i = 1:n
Z[i] = dot(Y, Xt[:,i])
end
# now time the synchronized copy of this
@time @sync @parallel for i = 1:n
Z[i] = dot(Y, Xt[:,i])
end
# still get same result?
println("$(isequal(z,Z))") # should be true
4个工作节点+1个主节点的测试输出:
elapsed time: 0.109010169 seconds (80 bytes allocated)
elapsed time: 0.110858551 seconds (80 bytes allocated)
true
elapsed time: 1.726231048 seconds (119936 bytes allocated)
true
您遇到了几个问题,其中最重要的是
Xt[:,i]
创建一个新数组(分配内存)。下面是一个让您更接近您想要的演示:
n = 10000; p = 25000
# make normal Arrays
x = randn(n,p)
y = ones(p)
z = zeros(n)
# make SharedArrays
X = convert(SharedArray, x)
Y = convert(SharedArray, y)
Z = convert(SharedArray, z)
Xt = X'
@everywhere function dotcol(a, B, j)
length(a) == size(B,1) || throw(DimensionMismatch("a and B must have the same number of rows"))
s = 0.0
@inbounds @simd for i = 1:length(a)
s += a[i]*B[i,j]
end
s
end
function run1!(Z, Y, Xt)
for j = 1:size(Xt, 2)
Z[j] = dotcol(Y, Xt, j)
end
Z
end
function runp!(Z, Y, Xt)
@sync @parallel for j = 1:size(Xt, 2)
Z[j] = dotcol(Y, Xt, j)
end
Z
end
run1!(Z, Y, Xt)
runp!(Z, Y, Xt)
@time run1!(Z, Y, Xt)
zc = copy(sdata(Z))
fill!(Z, -1)
@time runp!(Z, Y, Xt)
@show sdata(Z) == zc
结果(开始时julia-p8
):
为了进行比较,在同一台机器上运行时:
julia> blas_set_num_threads(8)
julia> @time A_mul_B!(Z, X, Y);
elapsed time: 0.067611858 seconds (80 bytes allocated)
因此,原始Julia实现至少与BLAS具有竞争力。ooh,我没有想到
Xt[:,I]
的内存分配。您的示例代码很容易理解。非常感谢。为什么当使用B[i,j]
而不是B[j,i]
(无需转置矩阵X
)时dotcol
会更快呢?@Taiki这是一个跨步内存访问的问题吗?Julia以列主格式存储数组。有关详细信息,请参阅。
julia> blas_set_num_threads(8)
julia> @time A_mul_B!(Z, X, Y);
elapsed time: 0.067611858 seconds (80 bytes allocated)