Java 通过Scala Spark读取动态Yaml文件
需要帮助,我有一个YAML文件。需要读取此yml文件并相应地编写scala spark代码。下面是我的yml文件 看完简介后。首先让我们假设'SubscriberProfilePanel'是我的表名,然后'highUtilizationScoreData'是我的列名,然后count是我编写了一个应用函数的条件。所以基本上我必须把这个动态Yaml文件作为字段的数量来读取,比如table,字段可以是或多或少的。修复的问题是“SubscriberProfilePanel”名称、复制或计算所有条件。那么,有人能帮助我如何在spark中通过java或scala读取动态yml文件吗Java 通过Scala Spark读取动态Yaml文件,java,scala,apache-spark,Java,Scala,Apache Spark,需要帮助,我有一个YAML文件。需要读取此yml文件并相应地编写scala spark代码。下面是我的yml文件 看完简介后。首先让我们假设'SubscriberProfilePanel'是我的表名,然后'highUtilizationScoreData'是我的列名,然后count是我编写了一个应用函数的条件。所以基本上我必须把这个动态Yaml文件作为字段的数量来读取,比如table,字段可以是或多或少的。修复的问题是“SubscriberProfilePanel”名称、复制或计算所有条件。那么
profile:
SubscriberProfilePanel:
activationDate:
copy:
table: SubscriebrAudirLog
field: activationDateCust
highUtilizationScoreData:
count:
table: FactADHPostpaidSubscriptionDataThresholdLimitSource
field: subscriber_key
filters:
operator: AND
1:
dateCompare:
date1:
table: FactADHPostpaidSubscriptionDataThresholdLimitSource
field: effective_date_ge
operator: <
date2:
table: FactADHPostpaidSubscriptionDataThresholdLimitSource
field: currentBillCycleStartDate
2:
dateCompare:
date1:
table: FactADHPostpaidSubscriptionDataThresholdLimitSource
field: effective_date_ge
operator: <
date2:
table: FactADHPostpaidSubscriptionDataThresholdLimitSource
field: threeBillCyclesStartDate
origin: destination
numCompare:
num1:
table: FactADHPostpaidSubscriptionDataThresholdLimitSource
field: threshold
operator: >=
num2:
table: FactADHPostpaidSubscriptionDataThresholdLimitSource
field: 80
AccountProfilePanel:
HomeProfilePanel:
PrepaidProfilePanel:
我得到的结果是
{profile={SubscriberProfilePanel={activationDate={copy={table=SubscriebrAudirLog, field=activationDateCust}}, highUtilizationScoreData={count={table=FactADHPostpaidSubscriptionDataThresholdLimitSource, field=subscriber_key, filters={operator=AND, 1={dateCompare={date1={table=FactADHPostpaidSubscriptionDataThresholdLimitSource, field=effective_date_ge}, operator=greater, date2={table=FactADHPostpaidSubscriptionDataThresholdLimitSource, field=currentBillCycleStartDate}}}, 2={dateCompare={date1={table=FactADHPostpaidSubscriptionDataThresholdLimitSource, field=effective_date_ge}, operator=greater, date2={table=FactADHPostpaidSubscriptionDataThresholdLimitSource, field=threeBillCyclesStartDate, origin=destination}}, numCompare={num1={table=FactADHPostpaidSubscriptionDataThresholdLimitSource, field=threshold}, operator=less_equal, num2={table=FactADHPostpaidSubscriptionDataThresholdLimitSource, field=80}}}}}}}, AccountProfilePanel=null, HomeProfilePanel=null
请在这里帮助我看我更新了我的问题,你能在这里帮助我吗看我更新了我的问题,你能在这里帮助我吗