Python和Snowflake使用SQL Alchemy引擎将新数据追加到现有表中,返回的是当前会话,但没有当前数据库
我需要在snowflake上的现有表中添加一些新数据。我正在使用Python和Snowflake使用SQL Alchemy引擎将新数据追加到现有表中,返回的是当前会话,但没有当前数据库,python,sqlalchemy,snowflake-cloud-data-platform,pandas-to-sql,Python,Sqlalchemy,Snowflake Cloud Data Platform,Pandas To Sql,我需要在snowflake上的现有表中添加一些新数据。我正在使用sqlalchemy作为引擎,同时使用pandas数据帧来创建sql()。以下是导入和脚本: import pandas as pd import os import snowflake.connector as snowCtx import getpass import json import numpy as np from datetime import date, datetime import time from sqla
sqlalchemy
作为引擎,同时使用pandas数据帧来创建sql()
。以下是导入和脚本:
import pandas as pd
import os
import snowflake.connector as snowCtx
import getpass
import json
import numpy as np
from datetime import date, datetime
import time
from sqlalchemy import create_engine
from sqlalchemy.dialects import registry
import snowflake.sqlalchemy
from snowflake.connector.pandas_tools import pd_writer
from sqlalchemy.ext.declarative import declarative_base
registry.register('snowflake', 'snowflake.sqlalchemy', 'dialect')
columns_df = pd.DataFrame(data.columns.to_list(), columns={'survey_column_name'})
columns_df['survey_id'] = nextval
columns_df = columns_df[['survey_id', 'survey_column_name']]
columns_df.to_sql('SURVEY_METADATA_COLUMN_NAMES',
index = False,
index_label = None,
con = engine,
schema = 'PUBLIC',
if_exists = 'append',
chunksize = 300,
method = pd_writer)
我得到的错误如下:
user = input('Your Snowflake username: ')
password = getpass.getpass('Your Snowflake Password: ')
account = 'MY_ACCOUNT'
conn = snowCtx.connect(
user=user,
password=password,
account=account,
database='MY_DB',
schema='PUBLIC',
warehouse='COMPUTE_WH',
role='SYSADMIN'
)
engine = create_engine(
'snowflake://{user}:{password}@{account}/'.format(
user=user,
password=password,
account=account,
database='MY_DB',
schema = 'PUBLIC',
warehouse='COMPUTE_WH',
role='SYSADMIN',
cache_column_metadata=True
)
)
编程错误:(雪花。连接器。错误。编程错误)090105
(22000):无法执行创建
桌子。此会话没有当前数据库。调用“使用”
“数据库”,或使用限定名称。[SQL:
创建表“PUBLIC”。“调查\元数据\列\名称”(
调查id整数
调查(列名称文本)
]
连接如下:
user = input('Your Snowflake username: ')
password = getpass.getpass('Your Snowflake Password: ')
account = 'MY_ACCOUNT'
conn = snowCtx.connect(
user=user,
password=password,
account=account,
database='MY_DB',
schema='PUBLIC',
warehouse='COMPUTE_WH',
role='SYSADMIN'
)
engine = create_engine(
'snowflake://{user}:{password}@{account}/'.format(
user=user,
password=password,
account=account,
database='MY_DB',
schema = 'PUBLIC',
warehouse='COMPUTE_WH',
role='SYSADMIN',
cache_column_metadata=True
)
)
我改为使用
write\u pandas()
:
success, nchunks, nrows, _ = write_pandas(conn,
columns_df,
'SURVEY_METADATA_COLUMN_NAMES',
chunk_size = 300,
schema = 'PUBLIC')
print(success, nchunks, nrows)
if(success):
print(filename+' columns uploaded')
else:
print(filename+' columns were not uploaded')
它需要pyarrow
库,因此我使用以下方法安装它:
pip install pyarrow
我删除了与sqlalchemy
相关的所有导入,并保留了以下内容:
import pandas as pd
import os
import snowflake.connector as snowCtx
import getpass
import json
import numpy as np
from datetime import date, datetime
import time
from snowflake.connector.pandas_tools import write_pandas