dst = distinct data;
DISTINCT只能对整个记录(整行)去重,不能在字段级别去重。
触发reduce阶段
data = load 'data';
distinct data;
dst = distinct data;
data = load 'data';
distinct data;
转载于:https://www.cnblogs.com/lishouguang/p/4559598.html