客网时事新闻,网址导航,心情日记,客家人Rss FeedHome信息导航Home Page AD.hadoop pig安装No commentsAugust 14th, 2013admin1.下载pig包：(pig-0.9.1) apache版本：http://pig.apache.org/2.解压文件：# tar -zxvf pig-{version}.tar.gz3.配置路径cat ~/.bashrcexport PIG_CLASSPATH=$HADOOP_HOME/conf/export PIG_INSTALL=/home/hadoop/pig-0.11.1export PATH=$PATH:$PIG_INSTALL/bin4.配置生效source ~/.bashrc5.启动pig#pig出现下面提示表示启动成功2013-08-15 15:19:47,294 [main] INFO  org.apache.pig.Main – Apache Pig version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:532013-08-15 15:19:47,295 [main] INFO  org.apache.pig.Main – Logging error messages to: /home/hadoop/pig-0.11.1/conf/pig_1376551187287.log2013-08-15 15:19:47,331 [main] INFO  org.apache.pig.impl.util.Utils – Default bootup file /home/hadoop/.pigbootup not found2013-08-15 15:19:47,653 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine – Connecting to hadoop file system at: hdfs://192.168.3.194:543102013-08-15 15:19:48,114 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine – Connecting to map-reduce job tracker at: 192.168.3.194:54311grunt>涂鸦hadoop pig:sql link 类sql的数据检索No commentsAugust 14th, 2013admin同时我将数据放在了两个文件,存放在/tmp/data_file_1和/tmp/data_file_21.从文件导入数据1)Mysql (Mysql需要先创建表).CREATE TABLE TMP_TABLE(USER VARCHAR(32),AGE INT,IS_MALE BOOLEAN);CREATE TABLE TMP_TABLE_2(AGE INT,OPTIONS VARCHAR(50));   — 用于JoinLOAD DATA LOCAL INFILE ‘/tmp/data_file_1′  INTO TABLE TMP_TABLE ;LOAD DATA LOCAL INFILE ‘/tmp/data_file_2′  INTO TABLE TMP_TABLE_2;2)Pigtmp_table = LOAD ‘/tmp/data_file_1′ USING PigStorage(‘\t’) AS (user:chararray, age:int,is_male:int);tmp_table_2= LOAD ‘/tmp/data_file_2′ USING PigStorage(‘\t’) AS (age:int,options:chararray);2.查询整张表1)MysqlSELECT * FROM TMP_TABLE;2)PigDUMP tmp_table;3. 查询前50行1)MysqlSELECT * FROM TMP_TABLE LIMIT 50;2)Pigtmp_table_limit = LIMIT tmp_table 50;DUMP tmp_table_limit; 4.查询某些列1)MysqlSELECT USER FROM TMP_TABLE;2)Pigtmp_table_user = FOREACH tmp_table GENERATE user;DUMP tmp_table_user;5. 给列取别名1)MysqlSELECT USER AS USER_NAME,AGE AS USER_AGE FROM TMP_TABLE;2)Pigtmp_table_column_alias = FOREACH tmp_table GENERATE user AS user_name,age AS user_age;DUMP tmp_table_column_alias; 6.排序1)MysqlSELECT * FROM TMP_TABLE ORDER BY AGE;2)Pigtmp_table_order = ORDER tmp_table BY age ASC;DUMP tmp_table_order;7.条件查询1）MysqlSELECT * FROM TMP_TABLE WHERE AGE>20;2) Pigtmp_table_where = FILTER tmp_table by age > 20;DUMP tmp_table_where;8.内连接Inner Join1)MysqlSELECT * FROM TMP_TABLE A JOIN TMP_TABLE_2 B ON A.AGE=B.AGE;2)Pigtmp_table_inner_join = JOIN tmp_table BY age,tmp_table_2 BY age;DUMP tmp_table_inner_join;9.左连接Left  Join1)MysqlSELECT * FROM TMP_TABLE A LEFT JOIN TMP_TABLE_2 B ON A.AGE=B.AGE;2)Pigtmp_table_left_join = JOIN tmp_table BY age LEFT OUTER,tmp_table_2 BY age;DUMP tmp_table_left_join;10.右连接Right Join1)MysqlSELECT * FROM TMP_TABLE A RIGHT JOIN TMP_TABLE_2 B ON A.AGE=B.AGE;2)Pigtmp_table_right_j