Hadoop “是什么意思？”；无法打开别名“的迭代器”；猪？

我试图使用联合运算符，如下所示

uni_b = UNION A, B, C, D, E, F, G, H;

这里所有的关系A，B，C…H都有相同的模式

当我使用转储操作器时，直到85%它运行良好。。之后，它将显示以下错误

ERROR 1066: Unable to open iterator for alias uni_b

这是什么？问题在哪里？我应该如何调试

这是我的猪剧本

ip = load '/jee/jee_data.txt' USING PigStorage(',') as (id:Biginteger, fname:chararray , lname:chararray , board:chararray , eid:chararray , gender:chararray , math:double , phy:double , chem:double , jeem:double , jeep:double , jeec:double ,cat:chararray , dob:chararray);

todate_ip = foreach ip generate id, fname , lname , board , eid , gender   , math , phy , chem , jeem , jeep , jeec , cat , ToDate(dob,'dd/MM/yyyy') as dob;

jnbresult1 = foreach todate_ip generate id, fname , lname , board , eid , gender , math , phy , chem , jeem , jeep , jeec, ROUND_TO(AVG(TOBAG( math , phy , chem )),3) as bresult, ROUND_TO(SUM(TOBAG(jeem , jeep , jeec )),3) as jresult , cat , dob;

rankjnbres = rank jnbresult1 by jresult DESC , bresult DESC , jeem DESC, math DESC, jeep DESC, phy DESC, jeec DESC, chem DESC, gender ASC, dob ASC, fname ASC, lname ASC DENSE;

rankjnbres1 = rank jnbresult1 by bresult DESC , jeem DESC, math DESC, jeep DESC, phy DESC, jeec DESC, chem DESC, gender ASC, dob ASC, fname ASC, lname ASC DENSE;

allper = foreach rankjnbres generate id, rank_jnbresult1 , fname , lname , board , eid , gender , math , phy , chem , jeem , jeep , jeec, bresult, jresult , cat , dob , ROUND_TO(((double)((10000-rank_jnbresult1)/100.000)),3) as aper;

allper1 = foreach rankjnbres1 generate id, rank_jnbresult1 , fname , lname , board , eid , gender , math , phy , chem , jeem , jeep , jeec, bresult, jresult , cat , dob , ROUND_TO(((double)((10000-rank_jnbresult1)/100.000)),3) as a1per;

SPLIT allper into cbseB if board=='CBSE', anbB if board=='Andhra Pradesh', apB if board=='Arunachal Pradesh', bhB if board=='Bihar', gjB if board=='Gujarat' , jnkB if board=='Jammu and Kashmir', mpB if board=='Madhya Pradesh', mhB if board=='Maharashtra',  rjB if board=='Rajasthan' ,  ngB if board=='Nagaland' ,  tnB if board=='Tamil Nadu' , wbB if board=='West Bengal' ,  upB if board=='Uttar Pradesh';

rankcbseB = rank cbseB by jresult DESC , bresult DESC , jeem DESC, math DESC, jeep DESC, phy DESC, jeec DESC, chem DESC, gender ASC, dob ASC, fname ASC, lname ASC DENSE;

grp = group rankcbseB all;

maxno = foreach grp generate MAX(rankcbseB.rank_cbseB) as max1;

cbseper = foreach rankcbseB generate id, rank_cbseB , fname , lname , board , eid , gender , math , phy , chem , jeem , jeep , jeec, bresult, jresult , cat , dob , ROUND_TO(((double)((maxno.max1-rank_cbseB)*100.000/maxno.max1)),3) as per , aper;

rankBcbseB = rank cbseB by bresult DESC , jeem DESC, math DESC, jeep DESC, phy DESC, jeec DESC, chem DESC, gender ASC, dob ASC, fname ASC, lname ASC DENSE;

grp = group rankBcbseB all;

maxno = foreach grp generate MAX(rankBcbseB.rank_cbseB) as max1;

Bcbseper = foreach rankBcbseB generate id, rank_cbseB , fname , lname , board , eid , gender , math , phy , chem , jeem , jeep , jeec, bresult, jresult , cat , dob , ROUND_TO(((double)((maxno.max1-rank_cbseB)*100.000/maxno.max1)),3) as bper , aper;

rankanbB = rank anbB by jresult DESC , bresult DESC , jeem DESC, math DESC, jeep DESC, phy DESC, jeec DESC, chem DESC, gender ASC, dob ASC, fname ASC, lname ASC DENSE;

grp = group rankanbB all;

maxno = foreach grp generate MAX(rankanbB.rank_anbB) as max1;

anbper = foreach rankanbB generate id, rank_anbB , fname , lname , board , eid , gender , math , phy , chem , jeem , jeep , jeec, bresult,jresult , cat , dob , ROUND_TO(((double)((maxno.max1-rank_anbB)*100.000/maxno.max1)),3) as per , aper;

rankBanbB = rank anbB by bresult DESC , jeem DESC, math DESC, jeep DESC, phy DESC, jeec DESC, chem DESC, gender ASC, dob ASC, fname ASC, lname ASC DENSE;

grp = group rankBanbB all;

maxno = foreach grp generate MAX(rankBanbB.rank_anbB) as max1;

Banbper = foreach rankanbB generate id, rank_anbB , fname , lname , board , eid , gender , math , phy , chem , jeem , jeep , jeec, bresult, jresult , cat , dob , ROUND_TO(((double)((maxno.max1-rank_anbB)*100.000/maxno.max1)),3) as bper , aper;

joinall = join cbseper by (per) , Bcbseper by (bper) ;

joinall = foreach joinall generate Bcbseper::id as id,cbseper::jresult as b1;

A = cross Bcbseper , allper;

A1 = foreach A generate Bcbseper::id as id,Bcbseper::rank_cbseB as rank,Bcbseper::fname as fname,Bcbseper::lname as lname,Bcbseper::board as board,Bcbseper::eid as eid ,Bcbseper::gender as gender, Bcbseper::bresult as bresult,Bcbseper::jresult as jresult,Bcbseper::cat as cat,Bcbseper::dob as dob,Bcbseper::bper as bper,Bcbseper::aper as aper,allper::jresult as b2,allper::aper as a1per;

B = filter A1 by bper > a1per;

C = group B by id;

Dcbse = foreach C {
E = order B by a1per DESC;
F = limit E 1;
generate FLATTEN(F.id) , FLATTEN(F.b2);
};

joincbse = join joinall by id , Dcbse by id;

joincbse = foreach joincbse generate joinall::id as id , joinall::b1 as b1, Dcbse::null::b2 as b2;

joinall = join anbper by (per) , Banbper by (bper) ;

joinall = foreach joinall generate Banbper::id as id,anbper::jresult as b1;

A = cross Banbper , allper;

A1 = foreach A generate Banbper::id as id,Banbper::rank_anbB as rank,Banbper::fname as fname,Banbper::lname as lname,Banbper::board as board,Banbper::eid as eid ,Banbper::gender as gender, Banbper::bresult as bresult,Banbper::jresult as jresult,Banbper::cat as cat,Banbper::dob as dob,Banbper::bper as bper,Banbper::aper as aper,allper::jresult as b2,allper::aper as a1per;

B = filter A1 by bper > a1per;

C = group B by id;

Danb = foreach C {
E = order B by a1per DESC;
F = limit E 1;
generate FLATTEN(F.id) , FLATTEN(F.b2);
};

joinanb = join joinall by id , Danb by id;

joinanb = foreach joinanb generate joinall::id as id , joinall::b1 as b1, Danb::null::b2 as b2;

uni_b = UNION joincbse , joinanb ;

---

我找到了解决办法。我所做的是以下内容

首先，我存储了所有的关系A，B，C。。。。使用存储操作如下

STORE A into into '/opA/' using PigStorage(',');

ipA = load '/opA/part-r-00000' USING PigStorage (',') as (id:Biginteger, b1: double, b2: double);

uni_b = UNION ipA ,ipB ,ipC , ipD ,ipE ;

然后，我使用如下加载操作加载所有关系的输入

STORE A into into '/opA/' using PigStorage(',');

ipA = load '/opA/part-r-00000' USING PigStorage (',') as (id:Biginteger, b1: double, b2: double);

uni_b = UNION ipA ,ipB ,ipC , ipD ,ipE ;

最后，我使用union操作完成了union，如下所示

STORE A into into '/opA/' using PigStorage(',');

ipA = load '/opA/part-r-00000' USING PigStorage (',') as (id:Biginteger, b1: double, b2: double);

uni_b = UNION ipA ,ipB ,ipC , ipD ,ipE ;

---

我得到了答案，没有任何错误。

我得到了这个问题的解决方案。我所做的是以下内容

首先，我存储了所有的关系A，B，C。。。。使用存储操作如下

STORE A into into '/opA/' using PigStorage(',');

ipA = load '/opA/part-r-00000' USING PigStorage (',') as (id:Biginteger, b1: double, b2: double);

uni_b = UNION ipA ,ipB ,ipC , ipD ,ipE ;

然后，我使用如下加载操作加载所有关系的输入

STORE A into into '/opA/' using PigStorage(',');

ipA = load '/opA/part-r-00000' USING PigStorage (',') as (id:Biginteger, b1: double, b2: double);

uni_b = UNION ipA ,ipB ,ipC , ipD ,ipE ;

最后，我使用union操作完成了union，如下所示

STORE A into into '/opA/' using PigStorage(',');

ipA = load '/opA/part-r-00000' USING PigStorage (',') as (id:Biginteger, b1: double, b2: double);

uni_b = UNION ipA ,ipB ,ipC , ipD ,ipE ;

---

我得到的答案没有任何错误。

可能有多个原因导致该错误。请发布整个pigscript和示例数据/输出请将其简化为一个最小的工作示例。你能把A，B，A，B，C，然后A，B，C，D等结合起来吗？那么A，A，A，A…？请发布pig脚本，如果uni_b的错误是唯一返回的错误，我会感到惊讶。但是，为了验证问题是否发生在这里，请描述并将输入存储到uni_b（我建议使用模式存储pigstorage），然后在新的最小脚本中加载并合并它们。这应该清楚错误发生的位置。该错误可能有多种原因。请发布整个pigscript和示例数据/输出。请将其简化为一个最小的工作示例。你能把A，B，A，B，C，然后A，B，C，D等结合起来吗？那么A，A，A，A…？请发布pig脚本，如果uni_b的错误是唯一返回的错误，我会感到惊讶。但是，为了验证问题是否发生在这里，请描述并将输入存储到uni_b（我建议使用模式存储pigstorage），然后在新的最小脚本中加载并合并它们。这应该清楚错误发生的位置。