Java 使用Hadoop/Hive的管道架构/技术建议_Java_Spring_Hadoop_Architecture_Hive

Java 使用Hadoop/Hive的管道架构/技术建议

java spring hadoop architecture hive

Java 使用Hadoop/Hive的管道架构/技术建议,java,spring,hadoop,architecture,hive,Java,Spring,Hadoop,Architecture,Hive,我的架构由两个阶段组成 1. ETL putting files on HDFS file system. 2. Hive running sql scripts on top of Hadoop and generating result set table. 3. The table is converted into XML 4. the XML is being uploaded to another location using http post. 我们发现自己在配置单元sql和b

我的架构由两个阶段组成

1. ETL putting files on HDFS file system.
2. Hive running sql scripts on top of Hadoop and generating result set table.
3. The table is converted into XML
4. the XML is being uploaded to another location using http post.

我们发现自己在配置单元sql和bash脚本上拥有逻辑。不确定这样做是否正确

我正在寻找一个pipleline框架来帮助我确定这个架构（Java/Spring或任何其他）

有什么建议吗？例子？我试过猪，但我们有并发症

谢谢，雷