python中缺少数据的二维卷积
我知道有scipy.signal.convolve2d函数来处理2d numpy数组的二维卷积,还有numpy.ma模块来处理丢失的数据,但这两种方法似乎互不兼容(这意味着即使在numpy中屏蔽2d数组,也不会影响convolve2d中的处理)。有没有办法只使用numpy和scipy包来处理卷积中缺少的值 例如:python中缺少数据的二维卷积,python,numpy,scipy,convolution,Python,Numpy,Scipy,Convolution,我知道有scipy.signal.convolve2d函数来处理2d numpy数组的二维卷积,还有numpy.ma模块来处理丢失的数据,但这两种方法似乎互不兼容(这意味着即使在numpy中屏蔽2d数组,也不会影响convolve2d中的处理)。有没有办法只使用numpy和scipy包来处理卷积中缺少的值 例如: 1 - 3 4 5 1 2 - 4 5 Array = 1 2 3 - 5 - 2 3 4 5
1 - 3 4 5
1 2 - 4 5
Array = 1 2 3 - 5
- 2 3 4 5
1 2 3 4 -
Kernel = 1 0
0 -1
卷积的期望结果(数组、内核、边界='wrap'):
感谢Aguy的建议,这是帮助计算卷积后结果的一个非常好的方法。现在让我们假设我们可以从Array.mask获得数组的掩码,这将给我们一个
False True False False False
False False True False False
Array.mask == False False False True False
True False False False False
False False False False True
如何使用此掩码将卷积后的结果转换为掩码数组?我不认为用0s替换是正确的方法,而是将共变值推向0。这些遗漏应按字面上的“遗漏”处理。因为它们代表丢失的信息片段,没有理由仅仅假设它们可能是0,而且它们根本不应该参与任何计算 我尝试将缺失值设置为
numpy.nan
,然后进行卷积,结果表明,内核和任何缺失之间的任何重叠都会在结果中产生nan
,即使重叠与内核中的0重叠,结果中的缺失孔也会增大。根据您的应用程序,这可能是期望的结果
但在某些情况下,你不想仅仅因为一个单一的缺失就放弃这么多信息(可能我发现了一个漏洞。使用虚数代替nan(如果是nan,则将其更改为1i)运行卷积运算,设置虚值高于阈值的地方为nan。无论何时低于阈值,只需取实值。下面是一段代码片段:
frames_complex = np.zeros_like(frames_, dtype=np.complex64)
frames_complex[np.isnan(frames_)] = np.array((1j))
frames_complex[np.bitwise_not(np.isnan(frames_))] =
frames_[np.bitwise_not(np.isnan(frames_))]
convolution = signal.convolve(frames_complex, gaussian_window, 'valid')
convolution[np.imag(convolution)>0.2] = np.nan
convolution = convolution.astype(np.float32)
卷积应该如何处理丢失的值?我觉得
结果[0,1]
应该是0
,而不是-
…您想用0替换屏蔽值,然后卷积,然后对结果重新应用屏蔽。请查看此repo()在那里,我创建了一个用Fortran编译的python模块,这将大大加快速度。我更新了自己的解决方案,在这里,我有两个问题要问@Jason的解决方案:你能发布你是如何创建/使用fxn的代码吗?你为什么不使用convolve2d,我如何使用它的属性,比如不同的模式,例如“有效”,即不使用ep相同形状的图像?制作滚动窗口功能的高超技巧。
from astropy.convolution import convolve
inarray=numpy.where(inarray.mask,numpy.nan,inarray) # masking still doesn't work, has to set to numpy.nan
result=convolve(inarray,kernel)
def convolve2d(slab,kernel,max_missing=0.5,verbose=True):
'''2D convolution with missings ignored
<slab>: 2d array. Input array to convolve. Can have numpy.nan or masked values.
<kernel>: 2d array, convolution kernel, must have sizes as odd numbers.
<max_missing>: float in (0,1), max percentage of missing in each convolution
window is tolerated before a missing is placed in the result.
Return <result>: 2d array, convolution result. Missings are represented as
numpy.nans if they are in <slab>, or masked if they are masked
in <slab>.
'''
from scipy.ndimage import convolve as sciconvolve
assert numpy.ndim(slab)==2, "<slab> needs to be 2D."
assert numpy.ndim(kernel)==2, "<kernel> needs to be 2D."
assert kernel.shape[0]%2==1 and kernel.shape[1]%2==1, "<kernel> shape needs to be an odd number."
assert max_missing > 0 and max_missing < 1, "<max_missing> needs to be a float in (0,1)."
#--------------Get mask for missings--------------
if not hasattr(slab,'mask') and numpy.any(numpy.isnan(slab))==False:
has_missing=False
slab2=slab.copy()
elif not hasattr(slab,'mask') and numpy.any(numpy.isnan(slab)):
has_missing=True
slabmask=numpy.where(numpy.isnan(slab),1,0)
slab2=slab.copy()
missing_as='nan'
elif (slab.mask.size==1 and slab.mask==False) or numpy.any(slab.mask)==False:
has_missing=False
slab2=slab.copy()
elif not (slab.mask.size==1 and slab.mask==False) and numpy.any(slab.mask):
has_missing=True
slabmask=numpy.where(slab.mask,1,0)
slab2=numpy.where(slabmask==1,numpy.nan,slab)
missing_as='mask'
else:
has_missing=False
slab2=slab.copy()
#--------------------No missing--------------------
if not has_missing:
result=sciconvolve(slab2,kernel,mode='constant',cval=0.)
else:
H,W=slab.shape
hh=int((kernel.shape[0]-1)/2) # half height
hw=int((kernel.shape[1]-1)/2) # half width
min_valid=(1-max_missing)*kernel.shape[0]*kernel.shape[1]
# dont forget to flip the kernel
kernel_flip=kernel[::-1,::-1]
result=sciconvolve(slab2,kernel,mode='constant',cval=0.)
slab2=numpy.where(slabmask==1,0,slab2)
#------------------Get nan holes------------------
miss_idx=zip(*numpy.where(slabmask==1))
if missing_as=='mask':
mask=numpy.zeros([H,W])
for yii,xii in miss_idx:
#-------Recompute at each new nan in result-------
hole_ys=range(max(0,yii-hh),min(H,yii+hh+1))
hole_xs=range(max(0,xii-hw),min(W,xii+hw+1))
for hi in hole_ys:
for hj in hole_xs:
hi1=max(0,hi-hh)
hi2=min(H,hi+hh+1)
hj1=max(0,hj-hw)
hj2=min(W,hj+hw+1)
slab_window=slab2[hi1:hi2,hj1:hj2]
mask_window=slabmask[hi1:hi2,hj1:hj2]
kernel_ij=kernel_flip[max(0,hh-hi):min(hh*2+1,hh+H-hi),
max(0,hw-hj):min(hw*2+1,hw+W-hj)]
kernel_ij=numpy.where(mask_window==1,0,kernel_ij)
#----Fill with missing if not enough valid data----
ksum=numpy.sum(kernel_ij)
if ksum<min_valid:
if missing_as=='nan':
result[hi,hj]=numpy.nan
elif missing_as=='mask':
result[hi,hj]=0.
mask[hi,hj]=True
else:
result[hi,hj]=numpy.sum(slab_window*kernel_ij)
if missing_as=='mask':
result=numpy.ma.array(result)
result.mask=mask
return result
frames_complex = np.zeros_like(frames_, dtype=np.complex64)
frames_complex[np.isnan(frames_)] = np.array((1j))
frames_complex[np.bitwise_not(np.isnan(frames_))] =
frames_[np.bitwise_not(np.isnan(frames_))]
convolution = signal.convolve(frames_complex, gaussian_window, 'valid')
convolution[np.imag(convolution)>0.2] = np.nan
convolution = convolution.astype(np.float32)