Azure data factory 使用参数从逻辑应用触发Azure数据工厂管道

Azure data factory 使用参数从逻辑应用触发Azure数据工厂管道,azure-data-factory,azure-logic-apps,Azure Data Factory,Azure Logic Apps,让我先说一句,我是逻辑应用程序和数据工厂的noob。无论如何,我目前正在进行集成,其中一部分是我需要从逻辑应用程序触发数据工厂中的管道。我已经成功地做到了这一点,但有一部分我似乎还没有弄清楚,那就是如何将参数传递给我的管道。我尝试在“参数”和“触发器”部分下修改JSON,但到目前为止还没有得到任何可点击的内容。管道最终执行,但仅使用默认参数 有人在这方面取得过成功吗?非常感谢您的帮助。正如我在评论中所说,我使用azure功能创建了一个解决方案。Azure功能和逻辑应用程序配合良好。 在此链接上,

让我先说一句,我是逻辑应用程序和数据工厂的noob。无论如何,我目前正在进行集成,其中一部分是我需要从逻辑应用程序触发数据工厂中的管道。我已经成功地做到了这一点,但有一部分我似乎还没有弄清楚,那就是如何将参数传递给我的管道。我尝试在“参数”和“触发器”部分下修改JSON,但到目前为止还没有得到任何可点击的内容。管道最终执行,但仅使用默认参数


有人在这方面取得过成功吗?非常感谢您的帮助。

正如我在评论中所说,我使用azure功能创建了一个解决方案。Azure功能和逻辑应用程序配合良好。 在此链接上,您可以看到如何使用.net创建和管理管道

如果您已经有了ADF和管道,您只想运行它(使用管道),那么您可以

Dictionary<string, object> parameters = new Dictionary<string, object>
       {
           {"BoxSerialNumbers", req.BoxSerialNumbers},
           {"StartDate", req.StartDate },
           {"EndDate",req.EndDate },
           {"Recipient", req.Recipient }
       };//this is how you add initialaze parameters

        var client = Authenticate(); //Authentication with azure
        log.Info("Creating.");
        CreateRunResponse runResponse = client.Pipelines.CreateRun(resourceGroup, dataFactoryName, "pipeline1", parameters);//run pipeline, you can do this async (it's better)
        log.Info("Created.");
        var response = new HttpResponseMessage();
        if (client.PipelineRuns.Get(resourceGroup, dataFactoryName, runResponse.RunId).Status.Equals("InProgress"))
        {
            response = new HttpResponseMessage(HttpStatusCode.OK)
            {
                Content = new StringContent(runResponse.RunId, Encoding.UTF8)
            };
        }
        else
        {
            response = new HttpResponseMessage(HttpStatusCode.BadRequest)
            {
                Content = new StringContent("Pipeline didn't started", Encoding.UTF8)//just some validation for function
            };
        }
        return response;                                               


    public static DataFactoryManagementClient Authenticate()
    {
        var context = new AuthenticationContext("https://login.windows.net/" + tenantID);
        ClientCredential cc = new ClientCredential(applicationID, authenticationKey);
        AuthenticationResult result = context.AcquireTokenAsync("https://management.azure.com/", cc).Result;
        ServiceClientCredentials cred = new TokenCredentials(result.AccessToken);
        return new DataFactoryManagementClient(cred) { SubscriptionId = subscriptionID };
    }
字典参数=新字典
{
{“BoxSerialNumbers”,req.BoxSerialNumbers},
{“StartDate”,req.StartDate},
{“EndDate”,req.EndDate},
{“收件人”,请求收件人}
};//这就是添加initialaze参数的方式
var client=Authenticate()//azure身份验证
log.Info(“创建”);
CreateRunResponse runResponse=client.Pipelines.CreateRun(resourceGroup,dataFactoryName,“pipeline1”,参数)//运行管道,您可以异步执行此操作(更好)
log.Info(“创建”);
var response=新的HttpResponseMessage();
if(client.PipelineRuns.Get(resourceGroup、dataFactoryName、runResponse.RunId).Status.Equals(“InProgress”))
{
response=新的HttpResponseMessage(HttpStatusCode.OK)
{
Content=newstringcontent(runResponse.RunId,Encoding.UTF8)
};
}
其他的
{
响应=新的HttpResponseMessage(HttpStatusCode.BadRequest)
{
Content=newstringcontent(“管道没有启动”,Encoding.UTF8)//只是对函数进行一些验证
};
}
返回响应;
公共静态DataFactoryManagementClient身份验证()
{
var context=新的AuthenticationContext(“https://login.windows.net/“+租户);
ClientCredential cc=新的ClientCredential(applicationID,authenticationKey);
AuthenticationResult=context.AcquireTokenAsync(“https://management.azure.com/“,cc)。结果;
ServiceClientCredentials cred=新令牌凭据(result.AccessToken);
返回新的DataFactoryManagementClient(cred){SubscriptionId=SubscriptionId};
}

因此,在请求中,您可以通过逻辑应用程序传递参数,使用runId可以检查状态。然后在logic应用程序中,只需简单的HTTP请求即可调用此函数。希望这能帮助到别人。

我使用了DraganB的解决方案,但电话签名在

CreateRunResponse runResponse = client.Pipelines.CreateRun(resourceGroup, dataFactoryName, "pipeline1", parameters);
已经改变了。较小的编辑使这项工作完美:

CreateRunResponse runResponse = client.Pipelines.CreateRun(resourceGroup, dataFactoryName, "pipeline1", parameters: parameters);
这是为任何需要它的人提供的函数

[FunctionName("DatafactoryShim")]
    public async static Task<HttpResponseMessage> Run(
        [HttpTrigger(AuthorizationLevel.Function, "post")]
        HttpRequestMessage req,
        ExecutionContext context,
        TraceWriter log
    )
    {
        string messageBody = await req.Content.ReadAsStringAsync();

        BlobToDatalakeFactoryParameters postValues = JsonHelper.ToClass<BlobToDatalakeFactoryParameters>(messageBody);

        Dictionary<string, object> parameters = new Dictionary<string, object>
        {
            {"blobContainer", postValues.BlobContainer},
            {"blobFolder", postValues.BlobFolder },
            {"relativeDatalakeFolder", postValues.RelativeDatalakeFolder },
            {"modelType", postValues.ModelType }

        }; //this is how you add initialaze parameters

        var client = Authenticate(); //Authentication with azure

        string resourceGroup = ConfigurationManager.AppSettings["resourceGroup"];
        string dataFactoryName = ConfigurationManager.AppSettings["dataFactoryName"];
        string pipelineName = ConfigurationManager.AppSettings["pipelineName"];

        Console.WriteLine("Creating pipeline run...");
        CreateRunResponse runResponse = client.Pipelines.CreateRunWithHttpMessagesAsync(
            resourceGroup,
            dataFactoryName,
            pipelineName,
            parameters: parameters).Result.Body;
        Console.WriteLine("Pipeline run ID: " + runResponse.RunId);

        var response = new HttpResponseMessage();

        if (client.PipelineRuns.Get(ConfigurationManager.AppSettings["resourceGroup"],
            ConfigurationManager.AppSettings["dataFactoryName"], runResponse.RunId).Status.Equals("InProgress"))
        {
            response = new HttpResponseMessage(HttpStatusCode.OK)
            {
                Content = new StringContent(runResponse.RunId, Encoding.UTF8)
            };
        }
        else
        {
            response = new HttpResponseMessage(HttpStatusCode.BadRequest)
            {
                Content =
                    new StringContent("Pipeline didn't started", Encoding.UTF8) //just some validation for function
            };
        }

        return response;
    }
[FunctionName(“DatafactoryShim”)]
公共异步静态任务运行(
[HttpTrigger(AuthorizationLevel.Function,“post”)]
HttpRequestMessage请求,
ExecutionContext上下文,
TraceWriter日志
)
{
字符串messageBody=wait req.Content.ReadAsStringAsync();
BlobToDatalakeFactoryParameters postValues=JsonHelper.ToClass(messageBody);
字典参数=新字典
{
{“blobContainer”,postValues.blobContainer},
{“blobFolder”,postValues.blobFolder},
{“RelativeDataakeFolder”,postValues.RelativeDataakeFolder},
{“modelType”,postValues.modelType}
};//这就是添加initialaze参数的方法
var client=Authenticate();//使用azure进行身份验证
字符串resourceGroup=ConfigurationManager.AppSettings[“resourceGroup”];
字符串dataFactoryName=ConfigurationManager.AppSettings[“dataFactoryName”];
字符串pipelineName=ConfigurationManager.AppSettings[“pipelineName”];
WriteLine(“创建管道运行…”);
CreateRunResponse runResponse=client.Pipelines.CreateRunWithHttpMessageAsync(
资源组,
dataFactoryName,
管道名称,
参数:parameters)。Result.Body;
控制台.WriteLine(“管道运行ID:+runResponse.RunId”);
var response=新的HttpResponseMessage();
if(client.PipelineRuns.Get(ConfigurationManager.AppSettings[“resourceGroup”]),
ConfigurationManager.AppSettings[“dataFactoryName”],runResponse.RunId).Status.Equals(“InProgress”))
{
response=新的HttpResponseMessage(HttpStatusCode.OK)
{
Content=newstringcontent(runResponse.RunId,Encoding.UTF8)
};
}
其他的
{
响应=新的HttpResponseMessage(HttpStatusCode.BadRequest)
{
内容=
new StringContent(“管道未启动”,Encoding.UTF8)//只是对函数进行一些验证
};
}
返回响应;
}

您可以使用逻辑应用程序的“创建管道运行”操作的body属性将参数传递给管道。和往常一样,要小心,因为这个动作不仅在预览中,而且我在任何MS文档中都找不到这个解决方案。我只是根据其他类似动作的格式做了一个有根据的猜测

例如:

"Run_my_pipeline": {
  "inputs": {
    "host": {
      "connection": {
        "name": "@parameters('$connections')['azuredatafactory']['connectionId']"
      }
    },
    "method": "post",
    "body": {
      "param1": "myParamValue",
      "param2": "myParamValue"
    },
    "path": "...",
    "queries": {
      "x-ms-api-version": "2017-09-01-preview"
    },
    "authentication": "@parameters('$authentication')"
  }
}

是否使用REST API触发数据工厂管道运行。这对你的情况可能有用。logic app connector不支持参数?我可能需要处理这个问题,可能需要在logic app中使用HTTP操作?主要是想知道我是否真的能够在ADF连接器中使用参数,听起来我不会这么做