将JSON解析为U-SQL,然后转换为csv

将JSON解析为U-SQL,然后转换为csv,json,azure-data-factory,azure-data-lake,u-sql,Json,Azure Data Factory,Azure Data Lake,U Sql,我正在尝试将一些JSON格式的遥测数据转换为CSV格式,然后使用U-SQL将其写入文件 问题是一些JSON键值中有句点,因此当我执行SELECT操作时,U-SQL无法识别它们。当我检查输出文件时,我看到的只是“p1”的值。如何在脚本中表示JSON键名称的名称,以便识别它们。提前感谢您的帮助 代码: JSON: {“EventInfo.Source”:“JS_default_Source”,“EventInfo.Sequence”:“1”,“EventInfo.Name”:“daysofweek”

我正在尝试将一些JSON格式的遥测数据转换为CSV格式,然后使用U-SQL将其写入文件

问题是一些JSON键值中有句点,因此当我执行SELECT操作时,U-SQL无法识别它们。当我检查输出文件时,我看到的只是“p1”的值。如何在脚本中表示JSON键名称的名称,以便识别它们。提前感谢您的帮助

代码:

JSON:


{“EventInfo.Source”:“JS_default_Source”,“EventInfo.Sequence”:“1”,“EventInfo.Name”:“daysofweek”,“EventInfo.Time”:“2018-01-25T21:09:36.779Z”,“EventInfo.SdkVersion”:“ACT-Web-JS-2.6.0”,“AppInfo.Language”:“en”,“UserInfo.Language”:“en-US”,“UserInfo.TimeZone”:-08:00”,“DeviceInfo.BrowserName”:“Chrome”,“DeviceInfo.BrowserVersion”:“63.0.3239.132”,“DeviceInfo.OsName”:“Mac OS X”,“DeviceInfo.OsVersion”:“10”,“p1”:“V1”,“PipelineInfo.IngestionTime”:“2018-01-25T21:09:33.9930000Z”,“PipelineInfo.ClientCountry”:“CA”,“PipelineInfo.IngestionPath”:“FastPath”,“EventInfo.BaseType”:“自定义”,“EventInfo.IngestionTime”:“2018-01-25T21:09:33.9930000Z”}我用单引号和单方括号实现了这一点,例如

@columnized = SELECT 
            json["['EventInfo.Source']"] AS EventInfoSource,
...
完整代码:

@columnized = SELECT 
            json["['EventInfo.Source']"] AS EventInfoSource,
            json["['EventInfo.InitId']"] AS EventInfoInitId,
            json["['EventInfo.Sequence']"] AS EventInfoSequence,
            json["['EventInfo.Name']"] AS EventInfoName,
            json["['EventInfo.Time']"] AS EventInfoTime,
            json["['EventInfo.SdkVersion']"] AS EventInfoSdkVersion,
            json["['AppInfo.Language']"] AS AppInfoLanguage,
            json["['UserInfo.Language']"] AS UserInfoLanguage,
            json["['DeviceInfo.BrowserName']"] AS DeviceInfoBrowswerName,
            json["['DeviceInfo.BrowserVersion']"] AS BrowswerVersion,
            json["['DeviceInfo.OsName']"] AS DeviceInfoOsName,
            json["['DeviceInfo.OsVersion']"] AS DeviceInfoOsVersion,
            json["['DeviceInfo.Id']"] AS DeviceInfoId,
            json["p1"] AS p1,
            json["['PipelineInfo.AccountId']"] AS PipelineInfoAccountId, 
            json["['PipelineInfo.IngestionTime']"] AS PipelineInfoIngestionTime, 
            json["['PipelineInfo.ClientIp']"] AS PipelineInfoClientIp,
            json["['PipelineInfo.ClientCountry']"] AS PipelineInfoClientCountry,
            json["['PipelineInfo.IngestionPath']"] AS PipelineInfoIngestionPath,
            json["['AppInfo.Id']"] AS AppInfoId,
            json["['EventInfo.Id']"] AS EventInfoId,
            json["['EventInfo.BaseType']"] AS EventInfoBaseType,
            json["['EventINfo.IngestionTime']"] AS EventINfoIngestionTime
    FROM @jsonify;
我的结果:


您有JSON文件的示例吗?代码本身没有帮助。@MichaelRys感谢您的评论。我已经添加了原始JSON文件的一部分。由于数据机密性,无法发布整个内容。我的问题是:我是否需要特殊语法来避开JSON键值中的句点?我之所以问这个问题,是因为输出文件仅包含h作为“p1”的值“,这是唯一一个没有句点的键,所以我猜这就是问题所在。您必须引用包含空格的名称,如wBob显示。我知道您的问题已经解决。但您能否围绕这个问题分享实时需求。它帮助我了解U-SQL的真正用法。我还有JSON格式的遥测数据。在将其发送到azure data lake之前,我必须将其转换为文件吗?谢谢,这很有效!还有一个问题:我正试图保持输出文件与输入文件的结构相同。当输入文件字符串是动态的(如原问题中所述)时,您知道如何陈述输出文件字符串以使其与输入文件字符串匹配吗?感谢所有的帮助这是一项经常被要求的功能,并且在这里的反馈列表中,同时还显示了Powershell的解决方法:在data lake engine处理这些功能之前,是否必须转换为CSV?我不能直接在azure data lake中处理所有传入的json遥测吗?
@columnized = SELECT 
            json["['EventInfo.Source']"] AS EventInfoSource,
            json["['EventInfo.InitId']"] AS EventInfoInitId,
            json["['EventInfo.Sequence']"] AS EventInfoSequence,
            json["['EventInfo.Name']"] AS EventInfoName,
            json["['EventInfo.Time']"] AS EventInfoTime,
            json["['EventInfo.SdkVersion']"] AS EventInfoSdkVersion,
            json["['AppInfo.Language']"] AS AppInfoLanguage,
            json["['UserInfo.Language']"] AS UserInfoLanguage,
            json["['DeviceInfo.BrowserName']"] AS DeviceInfoBrowswerName,
            json["['DeviceInfo.BrowserVersion']"] AS BrowswerVersion,
            json["['DeviceInfo.OsName']"] AS DeviceInfoOsName,
            json["['DeviceInfo.OsVersion']"] AS DeviceInfoOsVersion,
            json["['DeviceInfo.Id']"] AS DeviceInfoId,
            json["p1"] AS p1,
            json["['PipelineInfo.AccountId']"] AS PipelineInfoAccountId, 
            json["['PipelineInfo.IngestionTime']"] AS PipelineInfoIngestionTime, 
            json["['PipelineInfo.ClientIp']"] AS PipelineInfoClientIp,
            json["['PipelineInfo.ClientCountry']"] AS PipelineInfoClientCountry,
            json["['PipelineInfo.IngestionPath']"] AS PipelineInfoIngestionPath,
            json["['AppInfo.Id']"] AS AppInfoId,
            json["['EventInfo.Id']"] AS EventInfoId,
            json["['EventInfo.BaseType']"] AS EventInfoBaseType,
            json["['EventINfo.IngestionTime']"] AS EventINfoIngestionTime
    FROM @jsonify;