如何使用Apache Camel聚合CSV行?

如何使用Apache Camel聚合CSV行?,csv,groovy,jms,apache-camel,Csv,Groovy,Jms,Apache Camel,我的CSV与此类似: County City Area Street county1 city1 area1 street1 county1 city1 area2 street2 county1 city1 area3 street7 county1 city2 area2 street2 county1 city2 area6 street1 county2 city1 area3 street3 county2 city1 area3 street2 ... 在CSV解析过程中,我

我的CSV与此类似:

County  City  Area  Street
county1 city1 area1 street1
county1 city1 area2 street2
county1 city1 area3 street7
county1 city2 area2 street2
county1 city2 area6 street1
county2 city1 area3 street3
county2 city1 area3 street2
...
在CSV解析过程中,我需要聚合相同的县/市,以创建如下最终结构:

county1/city1: [ [area1, street1], [area2, street2], [area3, street7] ]
county1/city2: [ [area2, street2], [area6, street1] ]
county2/city1: [ [area3, street3], [area3, street2] ]
基本上是按县/市分组

我用骆驼试过不同的东西,这是最新的:

class CsvAppender {
    CsvRow append(CsvRow existing, CsvRow next) {
        next.previous = existing
        next
    }
}

@CsvRecord(separator = "\\t")
class CsvRow {
    @DataField(pos = 1)
    private String county

    @DataField(pos = 2)
    private String city

    @DataField(pos = 3)
    private String area

    @DataField(pos = 4)
    private String street

    CsvRow previous

    boolean sameAggregateWithPrevious() {
        previous?.county == county && previous?.city == city
    }

    public String toString() {
        "${county} ${city} ${area} ${street}"
    }
}

class CsvRouteBuilder extends RouteBuilder {

    void configure() {
        CsvAppender appender = new CsvAppender()

        Closure predicate = { exchange ->
            def body = exchange.getIn().getBody(CsvRow.class)
            def currentAggregate = exchange.getIn().getHeader('CurrentAggregate')
            def nextAggregate = exchange.getIn().getHeader('NextAggregate')

            if (!currentAggregate) {
                currentAggregate = body.previous ? [ body.previous ] : []
                nextAggregate = []
            } else if (exchange.getIn().getHeader('AggregateComplete')) {
                currentAggregate = nextAggregate
                nextAggregate = []
            }

            def aggregateComplete = body.sameAggregateWithPrevious()
            if (aggregateComplete) {
                nextAggregate << body
            } else {
                currentAggregate << body
            }
            exchange.getIn().setHeaders(['CurrentAggregate': currentAggregate,
                                         'NextAggregate': nextAggregate,
                                         'AggregateComplete': aggregateComplete])
            aggregateComplete
        }

        from("file:/tmp/folder?noop=true")
            .split(body().tokenize('\n')).streaming()
            .unmarshal().bindy(BindyType.Csv, CsvRow.class)
                .aggregate(constant(true), AggregationStrategies.bean(appender, "append")).completionPredicate(predicate)
                .process({
                    it.getOut().setBody(it.getIn().getHeader('CurrentAggregate')) })
                .convertBodyTo(String.class)
            .to("jms:myCsvSplitter")
    }
}
CsvAppender类{
CsvRow追加(CsvRow现有,CsvRow下一个){
next.previous=现有
下一个
}
}
@CsvRecord(分隔符=“\\t”)
类CsvRow{
@数据字段(位置=1)
私人弦县
@数据字段(位置=2)
私人字符串城市
@数据字段(位置=3)
私人弦区
@数据字段(位置=4)
私家弦街
CsvRow先前
布尔值SameAggerGateWithPrevious(){
以前的?.county==县和以前的?.city==市
}
公共字符串toString(){
“${county}${city}${area}${street}”
}
}
类CSVRoutBuilder扩展了RouteBuilder{
void configure(){
CsvAppender appender=新的CsvAppender()
闭包谓词={exchange->
def body=exchange.getIn().getBody(CsvRow.class)
def currentAggregate=exchange.getIn().getHeader('currentAggregate')
def nextAggregate=exchange.getIn().getHeader('nextAggregate')
如果(!currentAggregate){
currentAggregate=body.previous?[body.previous]:[]
nextAggregate=[]
}else if(exchange.getIn().getHeader('AggregateComplete')){
currentAggregate=nextAggregate
nextAggregate=[]
}
def aggregateComplete=body.sameAggerGateWithPrevious()
如果(聚合完成){

nextAggregate我有一些粗略的代码,希望能够很好地帮助您。它是用Java编写的,而不是用Groovy编写的,因为我的Groovy不够好。不过它应该很容易翻译

首先,聚合器:

public class MyAgregationStrategy implements AggregationStrategy {
    @Override
    public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
        CsvRow newBody = (CsvRow)newExchange.getIn().getBody();
        Map<String, List<CsvRow>> map = null;
        if (oldExchange == null) {
            map = new HashMap<String, List<CsvRow>>();
            ArrayList list = new ArrayList<CsvRow>();
            list.add(newBody);
            map.put(newBody.getCounty(), list);
            newExchange.getIn().setBody(map);
            return newExchange;
        } else {
            map = oldExchange.getIn().getBody(Map.class);
            List list = map.get(newBody.getCounty());
            if ( list == null ) {
                list = new ArrayList<CsvRow>();
            }
            list.add(newBody);
            map.put(newBody.getCounty(), list);

            oldExchange.setProperty("CamelSplitComplete", newExchange.getProperty("CamelSplitComplete"));
            return oldExchange;
        }
    }
}
它使用CamelSplitComplete属性来检测何时完成拆分。在最后的processpr中,您可以对映射执行您喜欢的操作。或者,您可以更改聚合器策略,以根据需要聚合结果


希望这能有所帮助。

您使用的Camel版本是什么,因为它会影响您可以使用的聚合()您确实需要Apache Camel来解析此文件?@matthelliwell 2.13.2(最新版本)@Opal很好,将所有内容都放在“一”环境中会很好(Camel通常支持csv)谢谢!它并没有完全解决问题,但它从你的例子中学到了更多
public class MyRouteBuilder extends RouteBuilder {
    @Override
    public void configure() throws Exception {
        from("file:/c:/dev/test?noop=true")
        .split(body().tokenize("\n"))
        .log("Read line ${body}")
        .unmarshal()
        .bindy(BindyType.Csv, CsvRow.class)
            .aggregate(constant(true), new MyAgregationStrategy()).completionPredicate(simple("${property.CamelSplitComplete} == true"))
        .process(new Processor() {
            @Override
            public void process(Exchange exchange) throws Exception {
                Map results = (Map) exchange.getIn().getBody();
                System.out.println("Got results for " + results.size() + " counties");
            }
        });
    }
}