Hadoop 分位数分组

Hadoop 分位数分组,hadoop,apache-pig,conditional-operator,quantile,Hadoop,Apache Pig,Conditional Operator,Quantile,我正在寻找一些解决Pig分组问题的方法。目前我有一个数据集,看起来像 Group | Height | Weight A | 96.5 | 110.2 B | 88.2 | 122.5 A | 94.1 | 100.8 B | 84.1 | 115.6 我使用DataFu库中的StreamingQuantile方法来计算高度变量上的分位数(25、50…蚀刻)。目前它可以工作,但我还需要计算每个组的平均权重+它们的分位数;所以它看起来像这样 A |

我正在寻找一些解决Pig分组问题的方法。目前我有一个数据集,看起来像

Group | Height | Weight
  A   | 96.5   | 110.2
  B   | 88.2   | 122.5
  A   | 94.1   | 100.8
  B   | 84.1   | 115.6
我使用DataFu库中的StreamingQuantile方法来计算高度变量上的分位数(25、50…蚀刻)。目前它可以工作,但我还需要计算每个组的平均权重+它们的分位数;所以它看起来像这样

A | Quantile1 | 88.5 (height)  | 134.4 (avg weight)
A | Quantile2 | 125.3 (height) | 156.2 (avg weight)
etc.....
B | Quantile4 | 144.3 (height) | 134.2 (avg weight)
作为参考,这里是计算分位数的简单Pig

REGISTER /usr/lib/datafu-1.2.0.jar;
define Quantile datafu.pig.stats.StreamingQuantile('0.0','0.25','0.5','0.75','1.0');
A = load 'mydata';
Group_A = GROUP A BY $0;
Quant = FOREACH GROUP_A GENERATE group,Quantile(A.$1);

我是否也可以计算每个分位数和组的平均$2?

为了避免混淆,我将列命名

REGISTER /usr/lib/datafu-1.2.0.jar;
define Quantile datafu.pig.stats.StreamingQuantile('0.0','0.25','0.5','0.75','1.0');
A = load 'mydata' using PigStorage('|') as (grouping:chararray, height:double, weight:double);
Group_A = GROUP A BY grouping;
Quant = FOREACH GROUP_A GENERATE group, Quantile(A.height) as q;
joined = join Quant by group, A by grouping;
grouped = group joined by (grouping, q)
average = foreach grouped generate group, AVG(joined.weight);