在Azure函数中安装雪花连接器python[pandas]

我正试图利用pythonsowflake连接器，通过VS代码扩展，将pandas额外功能部署到azure函数中。在本地运行函数是可行的&部署本身也是可行的

部署期间安装的依赖项的My requirements.txt如下所示：

azure-functions
xlrd
numpy
pandas
azure-storage-blob
snowflake-connector-python

代码中的导入如下所示：

import numpy as np
import pandas as pd
import azure.storage.blob
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient
import snowflake.connector
from snowflake.connector import DictCursor
#from snowflake.connector.pandas_tools import write_pandas

import logging
import snowflake.connector
import pandas as pd

class SnowflakeReader(object):
    def __init__(self, accnt: str, warehouse: str, dbuser: str, pw: str, db: str, dbschema: str):
    self.accnt = accnt
    self.warehouse = warehouse
    self.db = db
    self.dbschema = dbschema
    self.dbuser = dbuser
    self.pw = pw


    def form_connection(self):
        try:
            ctx = snowflake.connector.connect(
                user = self.dbuser,
                password = self.pw,
                account = self.accnt,
                warehouse = self.warehouse,
                database = self.db,
                schema = self.dbschema
            )
        except Exception as e:
            logging.error('Failed to connect to Snowflake: ' + str(e))
        return ctx

    def read_all(self, query: str, conn) -> pd.DataFrame:
        self.query = query
        self.conn = conn

        try:
            cs = conn.cursor()
            sql_cmd = self.query
            cs.execute(sql_cmd)
            result = cs.fetchall()
            result_df = pd.DataFrame(result)
        except Exception as e:
            logging.error('Failed to read from Snowflake: ' + str(e))

        return result_df

有了这些要求，一切正常。

问题是，只要我更改

requirements.txt

以包含雪花连接器的额外组件（代码中需要这些组件）：

我在尝试执行该函数时收到以下错误消息：

Result: Failure Exception: KeyError: 'snowflake-connector-python' Stack: File "/azure-functions-host/workers/python/3.8/LINUX/X64/azure_functions_worker/dispatcher.py", line 262, in _handle__function_load_request func = loader.load_function( File "/azure-functions-host/workers/python/3.8/LINUX/X64/azure_functions_worker/utils/wrappers.py", line 32, in call return func(*args, **kwargs) File "/azure-functions-host/workers/python/3.8/LINUX/X64/azure_functions_worker/loader.py", line 76, in load_function mod = importlib.import_module(fullmodname) File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 783, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/site/wwwroot/HttpTrigger1/__init__.py", line 23, in <module> import snowflake.connector File "/home/site/wwwroot/.python_packages/lib/site-packages/snowflake/connector/__init__.py", line 17, in <module> from .connection import SnowflakeConnection File "/home/site/wwwroot/.python_packages/lib/site-packages/snowflake/connector/connection.py", line 47, in <module> from .cursor import SnowflakeCursor, LOG_MAX_QUERY_LENGTH File "/home/site/wwwroot/.python_packages/lib/site-packages/snowflake/connector/cursor.py", line 48, in <module> from .arrow_result import ArrowResult File "arrow_result.pyx", line 16, in init snowflake.connector.arrow_result File "/home/site/wwwroot/.python_packages/lib/site-packages/snowflake/connector/options.py", line 35, in <module> _pandas_extras = pkg_resources.working_set.by_key['snowflake-connector-python']._dep_map['pandas']

Result:Failure Exception:KeyError:'snowflake connector python'Stack:File“/azure functions host/workers/python/3.8/LINUX/X64/azure_functions\u worker/dispatcher.py”，第262行，在"句柄"函数"加载"请求func=loader.load"函数（文件“/azure functions host/workers/python/3.8/LINUX/X64/azure_functions\u worker/utils/wrappers.py”，第32行，在调用返回函数（*args，**kwargs）文件“/azure functions host/workers/python/3.8/LINUX/X64/azure_functions_worker/loader.py”中，第76行，在加载函数mod=importlib.import_module（fullmodname）文件“/usr/local/lib/python3.8/importlib/u init_u.py”中，第127行，在导入_模块返回引导过程中。_gcd_导入（名称[级别：，包，级别）文件“”，第1014行，在“gcd\U导入文件”中，第991行，在“查找和加载文件”中，第961行，在“查找和加载未锁定文件”中，第219行，在“调用已删除文件”中，第1014行，在“gcd\U导入文件”中，第991行，在“查找和加载文件”中，第975行，在“查找和加载未锁定文件”中，第671行，在“加载未锁定文件”中，第783行，在exec_模块文件“”中，第219行，在带框架的调用文件“/home/site/wwwroot/HttpTrigger1/\uuu init_uuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuu“/home/site/wwwroot/.python\u packages/lib/site packages/snowflake/connector/connection.py”，第47行，from.cursor导入雪花光标，LOG\u MAX\u查询长度文件“/home/site/wwwroot/.python\u packages/lib/site packages/snowflake/connector/cursor.py”，第48行，from.arrow\u结果导入箭头结果文件“arrow\u result.pyx”，第16行，在init snowflake.connector.arrow_结果文件“/home/site/wwwroot/.python_packages/lib/site packages/snowflake/connector/options.py”，第35行，在_pandas_extras=pkg_resources.working_set.by_key['snowflake-connector-python']中

---

关于如何使函数正确运行，有什么帮助或想法吗？谢谢！

这似乎是pandas工具中的一个错误，我也遇到了同样的问题。根据您计划使用pandas工具的内容，您可能可以绕过它

当我查询Snowflake时，我的函数依赖于fetch_pandas_all（）函数，但我可以通过只安装基本的Snowflake connector python包并直接安装pandas来解决这个问题，然后将代码更改为使用以下代码：

import numpy as np
import pandas as pd
import azure.storage.blob
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient
import snowflake.connector
from snowflake.connector import DictCursor
#from snowflake.connector.pandas_tools import write_pandas

import logging
import snowflake.connector
import pandas as pd

class SnowflakeReader(object):
    def __init__(self, accnt: str, warehouse: str, dbuser: str, pw: str, db: str, dbschema: str):
    self.accnt = accnt
    self.warehouse = warehouse
    self.db = db
    self.dbschema = dbschema
    self.dbuser = dbuser
    self.pw = pw


    def form_connection(self):
        try:
            ctx = snowflake.connector.connect(
                user = self.dbuser,
                password = self.pw,
                account = self.accnt,
                warehouse = self.warehouse,
                database = self.db,
                schema = self.dbschema
            )
        except Exception as e:
            logging.error('Failed to connect to Snowflake: ' + str(e))
        return ctx

    def read_all(self, query: str, conn) -> pd.DataFrame:
        self.query = query
        self.conn = conn

        try:
            cs = conn.cursor()
            sql_cmd = self.query
            cs.execute(sql_cmd)
            result = cs.fetchall()
            result_df = pd.DataFrame(result)
        except Exception as e:
            logging.error('Failed to read from Snowflake: ' + str(e))

        return result_df

---

看起来您正在尝试导入write_pandas，这让我觉得您希望将数据帧直接写入Snowflake中的表。如果是这种情况，您可以使用旧的基于SQLAlchemy的方法构建类似的解决方案，或者只是将数据帧转换为dict并使用execute（）函数传递给insert语句。

你说得对，我尝试使用

write_pandas（）

函数将数据帧直接写入Snowflake中的表。最后，我完全按照你的建议重新编写了代码，使用了较旧的SQLAlchemy方法，效果很好。