Python 将360度电影映射到球体/实现vtkMPEGReader类

底部编辑的版本

我有一个360摄像头的电影片段，我想把它映射到一个球体上。场景保存为MP4文件中的等矩形投影

我发现了如何将图像映射到球体上的解释，并且能够使用mayavi.animate decorator包装示例中的代码，从MP4文件中读取连续帧，并按照预期将纹理映射到球体上（使用从mayavi示例部分剪下的代码，该部分将numpy数组转换为tvtk.ImageData对象）

然而，我遇到了一个问题，使我怀疑我可能是用这个完全错误的奇怪的圆形瑕疵：

当我将链接示例与电影中的JPG快照一起使用时，投影看起来就像它应该的那样（上面的部分看起来好像是错误的，但这是视频片段的一部分）：

代码如下：

'''
Altered version of Andras Deak's
on https://stackoverflow.com/questions/53074908/map-an-image-onto-a-sphere-and-plot-3d-trajectories

'''

import imageio
from mayavi import mlab
from tvtk.api import tvtk # python wrappers for the C++ vtk ecosystem
from tvtk.tools import visual
from tvtk.common import configure_input_data, is_old_pipeline

def image_from_array(ary):
    """ Create a VTK image object that references the data in ary.
        The array is either 2D or 3D with.  The last dimension
        is always the number of channels.  It is only tested
        with 3 (RGB) or 4 (RGBA) channel images.
        Note: This works no matter what the ary type is (accept
        probably complex...).  uint8 gives results that make since
        to me.  Int32 and Float types give colors that I am not
        so sure about.  Need to look into this...
    """

    sz = ary.shape
    dims = len(sz)
    # create the vtk image data
    img = tvtk.ImageData()

    if dims == 2:
        # 1D array of pixels.
        img.whole_extent = (0, sz[0]-1, 0, 0, 0, 0)
        img.dimensions = sz[0], 1, 1
        img.point_data.scalars = ary

    elif dims == 3:
        # 2D array of pixels.
        if is_old_pipeline():
            img.whole_extent = (0, sz[0]-1, 0, sz[1]-1, 0, 0)
        else:
            img.extent = (0, sz[0]-1, 0, sz[1]-1, 0, 0)
        img.dimensions = sz[0], sz[1], 1

        # create a 2d view of the array
        ary_2d = ary[:]
        ary_2d.shape = sz[0]*sz[1],sz[2]
        img.point_data.scalars = ary_2d

    else:
        raise ValueError("ary must be 3 dimensional.")

    return img



# create a figure window (and scene)
fig = mlab.figure(size=(600, 600))
visual.set_viewer(fig)

# load video
vid = imageio.get_reader('movie.mp4', 'ffmpeg')

# use a TexturedSphereSource, a.k.a. getting our hands dirty
R = 1
Nrad = 180

# create the sphere source with a given radius and angular resolution
sphere = tvtk.TexturedSphereSource(radius=R, theta_resolution=Nrad, phi_resolution=Nrad)

# assemble rest of the pipeline, assign texture
sphere_mapper = tvtk.PolyDataMapper(input_connection=sphere.output_port)
sphere_actor = tvtk.Actor(mapper=sphere_mapper)

@mlab.show
@mlab.animate(delay=50)
def auto_sphere():

    for i in range(1,600):

        image = vid.get_data(i)
        img = image_from_array(image)
        texture = tvtk.Texture(interpolate=1)
        configure_input_data(texture, img)

        sphere_actor.texture = texture
        fig.scene.add_actor(sphere_actor)

        yield


auto_sphere()

我对这个话题完全陌生。如何正确地做到这一点

---

编辑：

所以我想我设法找出了真正的问题。但我还不知道如何解决它问题似乎在于读取MP4文件的方式。在这个修改版本中（使用MP4中各个帧的jpeg文件），vtk管道和生成的渲染电影看起来是正确的：

from mayavi import mlab
from tvtk.api import tvtk # python wrappers for the C++ vtk ecosystem

# create a figure
fig = mlab.figure(size=(600, 600))

# sphere source
R = 1
Nrad = 180
sphere = tvtk.TexturedSphereSource(radius=R, theta_resolution=Nrad, phi_resolution=Nrad)
# sphere mapper
sphere_mapper = tvtk.PolyDataMapper(input_connection=sphere.output_port)
# actor
sphere_actor = tvtk.Actor(mapper=sphere_mapper)
# image reader
image = tvtk.JPEGReader()
image.file_name = 'testdata/frame001.jpg'
# texture
texture = tvtk.Texture(input_connection=image.output_port, interpolate=1)
sphere_actor.texture = texture

# add actor to scene
fig.scene.add_actor(sphere_actor)

@mlab.show
@mlab.animate(delay=50)
def auto_sphere():

    for i in range(2,101):
        num = str(i)

        filepath = 'testdata/frame%s.jpg' % num.zfill(3)

        image.file_name = filepath

        # render current texture
        fig.scene.render()

        yield

auto_sphere()

我想我现在的新问题是： 我可以在python中实现一个定制的vtkImageReader2类或类似的东西，允许我在mp4的连续帧中进行读取吗？如果是这样的话，那么如何才能做到这一点呢？ 不幸的是，我找不到任何关于如何做到这一点的教程。

你可以看到你可以看到