Python 3.x 使用DataRicks将txt文件从Azure文件复制到Blob存储
我想从Azure文件中读取一个文件(通过使用ShareClient成功),并将此文件导出到Azure blob存储 首先,我使用以下代码在Databricks中装载容器:Python 3.x 使用DataRicks将txt文件从Azure文件复制到Blob存储,python-3.x,azure-storage-blobs,databricks,dbutils,storage-file-share,Python 3.x,Azure Storage Blobs,Databricks,Dbutils,Storage File Share,我想从Azure文件中读取一个文件(通过使用ShareClient成功),并将此文件导出到Azure blob存储 首先,我使用以下代码在Databricks中装载容器: def mount(container, account_name): """Mount a container in blob storage""" mount_list = [_.mountPoint for _ in dbutils.fs.mou
def mount(container, account_name):
"""Mount a container in blob storage"""
mount_list = [_.mountPoint for _ in dbutils.fs.mounts()]
mount_point = f"/mnt/{container}/"
if mount_point not in mount_list:
dbutils.fs.mount(
source = f"wasbs://{container}@{account_name}.blob.core.windows.net/",
mount_point = mount_point,
extra_configs = {f"fs.azure.account.key.saweaaedwh.blob.core.windows.net":dbutils.secrets.get(scope = "KEY-WE-AAE-DWH", key = f"key-{account_name}")})
print(f"Container {container} is successfully mounted")
else:
print("Container is already mounted")
with open(f"/dbfs/mnt/datascience/spc/failed2/test.txt", "wb") as outfile:
download_stream = file_client.download_file()
outfile.write(download_stream.readall())
当我想上传带有以下代码的文件时:
def mount(container, account_name):
"""Mount a container in blob storage"""
mount_list = [_.mountPoint for _ in dbutils.fs.mounts()]
mount_point = f"/mnt/{container}/"
if mount_point not in mount_list:
dbutils.fs.mount(
source = f"wasbs://{container}@{account_name}.blob.core.windows.net/",
mount_point = mount_point,
extra_configs = {f"fs.azure.account.key.saweaaedwh.blob.core.windows.net":dbutils.secrets.get(scope = "KEY-WE-AAE-DWH", key = f"key-{account_name}")})
print(f"Container {container} is successfully mounted")
else:
print("Container is already mounted")
with open(f"/dbfs/mnt/datascience/spc/failed2/test.txt", "wb") as outfile:
download_stream = file_client.download_file()
outfile.write(download_stream.readall())
出现以下错误消息:
FileNotFoundError: [Errno 2] No such file or directory: '/dbfs/mnt/datascience/spc/failed2/test.txt'
为了创建此目录,我使用了以下代码:
dbutils.fs.mkdirs('/mnt/datascience/spc/failed2/test.txt')
问题是,这也会创建“空”目录。如果目录至少包含一个文件,您知道如何创建目录吗
您能详细描述一下您的问题吗?路径
/dbfs/mnt/datascience/spc/failed2
不存在,您希望在该路径中创建一个文件。对吗?我在第一个代码片段中安装了带有Databricks的Blob存储。之后,我想上传一个txt文件到blob存储。其中spc/failed2目录尚不存在。所以我使用dbutils.fs.mkdirs来创建这样一个目录。然后创建了空文件夹和普通文件夹(重复),看起来很乱。(请参见创建内容的图片)Azure Blob服务基于平面存储方案,而不是分层方案。它没有目录
。它只是在blob名称中使用字符或字符串分隔符来创建虚拟层次结构。因此,当我们使用dbutils.fs.mkdirs创建东西时,Blob服务器只会认为我们想要创建Blob。有关更多详细信息,请参阅。