Azure data factory Azure Data Factory V2数据集动态文件夹
在Azure数据工厂(V1)中,我能够创建一张幻灯片并将输出存储到特定文件夹(即{Year}/{Month}/{Day})。请参阅下面的代码 如何在Azure Data Factory V2中创建相同类型的切片?我发现必须创建参数。是的,我无法确定如何传递参数Azure data factory Azure Data Factory V2数据集动态文件夹,azure-data-factory,Azure Data Factory,在Azure数据工厂(V1)中,我能够创建一张幻灯片并将输出存储到特定文件夹(即{Year}/{Month}/{Day})。请参阅下面的代码 如何在Azure Data Factory V2中创建相同类型的切片?我发现必须创建参数。是的,我无法确定如何传递参数 "folderPath": "@{dataset().path}", "parameters": { "path": { "type": "String" 这里是原始的ADF V1代码 { "name": "EMS
"folderPath": "@{dataset().path}",
"parameters": {
"path": {
"type": "String"
这里是原始的ADF V1代码
{
"name": "EMS_EMSActivations_L1_Snapshot",
"properties": {
"published": false,
"type": "AzureDataLakeStore",
"linkedServiceName": "SalesIntelligence_ADLS_LS",
"typeProperties": {
"fileName": "EMS.FACT_EMSActivations_WTA.tsv",
"folderPath": "/Snapshots/EMS/FACT_EMSActivations_WTA/{Year}/{Month}/{Day}",
"format": {
"type": "TextFormat",
"rowDelimiter": "␀",
"columnDelimiter": "\t",
"nullValue": "#NULL#",
"quoteChar": "\""
},
"partitionedBy": [
{
"name": "Year",
"value": {
"type": "DateTime",
"date": "SliceStart",
"format": "yyyy"
}
},
{
"name": "Month",
"value": {
"type": "DateTime",
"date": "SliceStart",
"format": "MM"
}
},
{
"name": "Day",
"value": {
"type": "DateTime",
"date": "SliceStart",
"format": "dd"
}
},
{
"name": "Hour",
"value": {
"type": "DateTime",
"date": "SliceStart",
"format": "HH"
}
},
{
"name": "Minute",
"value": {
"type": "DateTime",
"date": "SliceStart",
"format": "mm"
}
}
]
},
"availability": {
"frequency": "Day",
"interval": 1
}
}
}
以下是将数据从SQL导入ADL时如何创建动态文件夹路径
{
"name": "EBC_BriefingActivitySummary_L1_Snapshot",
"properties": {
"linkedServiceName": {
"referenceName": "SIAzureDataLakeStore",
"type": "LinkedServiceReference"
},
"type": "AzureDataLakeStoreFile",
"typeProperties": {
"format": {
"type": "TextFormat",
"columnDelimiter": ",",
"rowDelimiter": "",
"nullValue": "\\N",
"treatEmptyAsNull": false,
"firstRowAsHeader": false
},
"fileName": {
"value": "EBC.rpt_BriefingActivitySummary.tsv",
"type": "Expression"
},
"folderPath": {
"value": "@concat('/Snapshots/EBC/rpt_BriefingActivitySummary/', formatDateTime(pipeline().parameters.scheduledRunTime, 'yyyy'), '/', formatDateTime(pipeline().parameters.scheduledRunTime, 'MM'), '/', formatDateTime(pipeline().parameters.scheduledRunTime, 'dd'), '/')",
"type": "Expression"
}
}
}
}
以下是将数据从SQL导入ADL时如何创建动态文件夹路径
{
"name": "EBC_BriefingActivitySummary_L1_Snapshot",
"properties": {
"linkedServiceName": {
"referenceName": "SIAzureDataLakeStore",
"type": "LinkedServiceReference"
},
"type": "AzureDataLakeStoreFile",
"typeProperties": {
"format": {
"type": "TextFormat",
"columnDelimiter": ",",
"rowDelimiter": "",
"nullValue": "\\N",
"treatEmptyAsNull": false,
"firstRowAsHeader": false
},
"fileName": {
"value": "EBC.rpt_BriefingActivitySummary.tsv",
"type": "Expression"
},
"folderPath": {
"value": "@concat('/Snapshots/EBC/rpt_BriefingActivitySummary/', formatDateTime(pipeline().parameters.scheduledRunTime, 'yyyy'), '/', formatDateTime(pipeline().parameters.scheduledRunTime, 'MM'), '/', formatDateTime(pipeline().parameters.scheduledRunTime, 'dd'), '/')",
"type": "Expression"
}
}
}
}
第1步:
在folderpath中使用WindowsStartTime/WindowEndTime
"folderPath": {
"value": "<<path>>/@{formatDateTime(pipeline().parameters.windowStart,'yyyy')}-@{formatDateTime(pipeline().parameters.windowStart,'MM')}-@{formatDateTime(pipeline().parameters.windowStart,'dd')}/@{formatDateTime(pipeline().parameters.windowStart,'HH')}/",
"type": "Expression"
}
步骤3:在TumblingWindow触发器中添加运行参数
(这在第2步中提到)
"parameters": {
"windowStart": {
"type": "Expression",
"value": "@trigger().outputs.windowStartTime"
},
"windowEnd": {
"type": "Expression",
"value": "@trigger().outputs.windowEndTime"
}
}
如需了解更多详细信息,请参阅
请参阅此链接
步骤1:
在folderpath中使用WindowsStartTime/WindowEndTime
"folderPath": {
"value": "<<path>>/@{formatDateTime(pipeline().parameters.windowStart,'yyyy')}-@{formatDateTime(pipeline().parameters.windowStart,'MM')}-@{formatDateTime(pipeline().parameters.windowStart,'dd')}/@{formatDateTime(pipeline().parameters.windowStart,'HH')}/",
"type": "Expression"
}
步骤3:在TumblingWindow触发器中添加运行参数
(这在第2步中提到)
"parameters": {
"windowStart": {
"type": "Expression",
"value": "@trigger().outputs.windowStartTime"
},
"windowEnd": {
"type": "Expression",
"value": "@trigger().outputs.windowEndTime"
}
}
如需了解更多详细信息,请参阅
请参阅此链接