Teradata 如何使用完美任务运行sql命令

Teradata 如何使用完美任务运行sql命令,teradata,python-3.5,dask,prefect,Teradata,Python 3.5,Dask,Prefect,我正在尝试从teradata获取数据,然后使用完美任务写入和读取拼花地板文件 我的代码在单独执行sql时正常工作,如下所示 def fetch_data(host,db_name,user,password,query): 'logic' @task(name="Write dask dataframe into GPFS parquet file") def write_data(dask_dataframe,file_name): 'logic' @t

我正在尝试从teradata获取数据,然后使用完美任务写入和读取拼花地板文件 我的代码在单独执行sql时正常工作,如下所示

def fetch_data(host,db_name,user,password,query):
    'logic'

@task(name="Write dask dataframe into GPFS parquet file")
def write_data(dask_dataframe,file_name):
    'logic'

@task(name="Read data from GPFS parquet file into dask dataframe")
def read_data(file_name):
    'logic'

with Flow("Teradata Example") as flow:
    result = fetch_data(host,db_name,user,password,query)
    write_data(dask_dataframe=result,file_name=file_name)
    read_data(file_name=file_name)

flow.run()
但当同一个获取程序代码作为任务运行时,代码失败

@task(name="Fetch sql query data from teradta data source into dask dataframe")
def fetch_data(host,db_name,user,password,query):
    'logic'

@task(name="Write dask dataframe into GPFS parquet file")
def write_data(dask_dataframe,file_name):
    'logic'

@task(name="Read data from GPFS parquet file into dask dataframe")
def read_data(file_name):
    'logic'

with Flow("Teradata Example") as flow:
    result = fetch_data(host,db_name,user,password,query)
    write_data(dask_dataframe=result,file_name=file_name)
    read_data(file_name=file_name)

flow.run()
添加了teradata文件代码:

def get_partitions(num_partitions):
    list_range =[]
    initial_start=0
    for i in range(num_partitions):
        amp_range = 3240//num_partitions
        start = (i*amp_range+1)*initial_start
        end   = (i+1)*amp_range
        list_range.append((start,end))
        initial_start = 1
    return list_range

@delayed
def load(query,start,end,connString):
    return pd.read_sql(query.format(start, end),connString)

class TeradataFetch(Task):
    def __init__(
        args)

    @defaults_from_attrs("fetch", "fetch_count", "query", "commit", "charset")
    def run(
        self,
        query: str,
    ) -> Any:
        try:
            results = from_delayed([load(query,start, end,connString) for start,end in get_partitions(self.num_partitions)])
            logging.debug("Fetch Results: %s", results)
            return results

        except Exception as e:
            raise e



有人能在这里建议/帮助吗?

怎么会“失败”?哪一步失败了?错误消息将非常有用。我得到以下错误:[2020-11-13 16:20:05]INFO-perfict.FlowRunner |开始运行“Teradata示例”[2020-11-13 16:20:05]INFO-perfict.TaskRunner |任务“将sql查询数据从Teradata数据源提取到dask数据帧”:开始任务运行。。。[2020-11-13 16:20:05]错误-Perfict.TaskRunner |意外错误:ValueError('无法推断活动流上下文'),文件“teradata_dask_Flow.py”,第33行,在fetch_data fetch_results=teradata_fetch(query=query)文件“path\envs\perfict\lib\site packages\perfict-0.13.13-py3.6.egg\perfict\core\task.py”,第514行,在call*args中,mapped=mapped,upstream\u tasks=upstream\u tasks,flow=flow,**kwargs文件“path\envs\perfict\lib\site packages\perfict-0.13.13-py3.6.egg\perfict\core\task.py”,第566行,在bind raise ValueError中(“无法推断活动的流上下文”)ValueError:无法推断活动流上下文。您似乎没有显示所有代码-该错误表明您正试图在
获取数据
中使用名为
teradata\u fetch
的任务task@chriswhite它是teradata的一个连接器,这就是为什么我没有包括它,但现在我在上面添加了我文件的整个逻辑