Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/scala/16.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Java 通过Scala Spark读取动态Yaml文件_Java_Scala_Apache Spark - Fatal编程技术网

Java 通过Scala Spark读取动态Yaml文件

Java 通过Scala Spark读取动态Yaml文件,java,scala,apache-spark,Java,Scala,Apache Spark,需要帮助,我有一个YAML文件。需要读取此yml文件并相应地编写scala spark代码。下面是我的yml文件 看完简介后。首先让我们假设'SubscriberProfilePanel'是我的表名,然后'highUtilizationScoreData'是我的列名,然后count是我编写了一个应用函数的条件。所以基本上我必须把这个动态Yaml文件作为字段的数量来读取,比如table,字段可以是或多或少的。修复的问题是“SubscriberProfilePanel”名称、复制或计算所有条件。那么

需要帮助,我有一个YAML文件。需要读取此yml文件并相应地编写scala spark代码。下面是我的yml文件

看完简介后。首先让我们假设'SubscriberProfilePanel'是我的表名,然后'highUtilizationScoreData'是我的列名,然后count是我编写了一个应用函数的条件。所以基本上我必须把这个动态Yaml文件作为字段的数量来读取,比如table,字段可以是或多或少的。修复的问题是“SubscriberProfilePanel”名称、复制或计算所有条件。那么,有人能帮助我如何在spark中通过java或scala读取动态yml文件吗

profile: 
    SubscriberProfilePanel: 
        activationDate: 
            copy: 
                table: SubscriebrAudirLog
                field: activationDateCust
        highUtilizationScoreData: 
            count: 
                table: FactADHPostpaidSubscriptionDataThresholdLimitSource
                field: subscriber_key
                filters:
                    operator: AND
                    1: 
                        dateCompare: 
                            date1: 
                                table: FactADHPostpaidSubscriptionDataThresholdLimitSource
                                field: effective_date_ge
                            operator: <
                            date2: 
                                table: FactADHPostpaidSubscriptionDataThresholdLimitSource
                                field: currentBillCycleStartDate
                    2: 
                        dateCompare: 
                            date1: 
                                table: FactADHPostpaidSubscriptionDataThresholdLimitSource
                                field: effective_date_ge
                            operator: <
                            date2: 
                                table: FactADHPostpaidSubscriptionDataThresholdLimitSource
                                field: threeBillCyclesStartDate
                                origin: destination
                        numCompare: 
                            num1: 
                                table: FactADHPostpaidSubscriptionDataThresholdLimitSource
                                field: threshold
                            operator: >=
                            num2: 
                                table: FactADHPostpaidSubscriptionDataThresholdLimitSource
                                field: 80
    AccountProfilePanel: 
    HomeProfilePanel: 
    PrepaidProfilePanel: 
我得到的结果是


{profile={SubscriberProfilePanel={activationDate={copy={table=SubscriebrAudirLog, field=activationDateCust}}, highUtilizationScoreData={count={table=FactADHPostpaidSubscriptionDataThresholdLimitSource, field=subscriber_key, filters={operator=AND, 1={dateCompare={date1={table=FactADHPostpaidSubscriptionDataThresholdLimitSource, field=effective_date_ge}, operator=greater, date2={table=FactADHPostpaidSubscriptionDataThresholdLimitSource, field=currentBillCycleStartDate}}}, 2={dateCompare={date1={table=FactADHPostpaidSubscriptionDataThresholdLimitSource, field=effective_date_ge}, operator=greater, date2={table=FactADHPostpaidSubscriptionDataThresholdLimitSource, field=threeBillCyclesStartDate, origin=destination}}, numCompare={num1={table=FactADHPostpaidSubscriptionDataThresholdLimitSource, field=threshold}, operator=less_equal, num2={table=FactADHPostpaidSubscriptionDataThresholdLimitSource, field=80}}}}}}}, AccountProfilePanel=null, HomeProfilePanel=null

请在这里帮助我

看我更新了我的问题,你能在这里帮助我吗看我更新了我的问题,你能在这里帮助我吗