开发者学堂课程【Hadoop 分布式计算框架 MapReduce:WordCount 案例Mapper】学习笔记,与课程紧密联系,让用户快速学习知识。
课程地址:https://developer.aliyun.com/learning/course/94/detail/1482
WordCount 案例 Mapper
简介:
1、环境准备
2、编写程序
1、环境准备:
(1)(2)在 pom.xm1 文件中添加如下依赖
<dependencies>
<dependency>
<groupId>junit</ groupId>
<artifactId>junit</ artifactId>
<version>RELEASE</ version>
</ dependency>
<dependency>
<groupId>org.apache. logging. log4j</groupId>
<artifactId>log4j-core</ artifactId>
<version>2.8.2</version>
</ dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</ artifactId>
<version>2.7.2</ version>
</ dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</ artifactId>
<version>2.7.2</ version>
</ dependency>v
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-hdfs</ artifactId>
<version>2.7.2</ version>
</ dependencv>
</dependencies>
<aroupId>orq.apache.hadoop</ aroupId>
<veision>2.7.2</ version>
</ dependency>
</dependencies>v
(2)在项目的 src/main/resources 目录下,新建一个文件,命名为“log4j.properties”,在
文件中填入。
log4j.rootLogger=INFO,stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.
log4j.PatternLayout.log4j.appender.stdout.layout. ConversionPattern=%d sp [%c]-%m%n”
log4j.appender.logfile=org.apache
log4j.FileAppender.log4j.appender. logfile.File=target /spring .log-
log4j.appender.logfile.layout=org.apache
log4j.PatternLayout
log4j.appender.logfile.layout.ConversionPattern=%dp[%c]
-%m%n·
2、编写程序
(1)编写 Mapper 类
package com . atguigu.mapreduce;
import java.io.IOException;
irmport org.apache . hadoop.io.Intwritable;
import org.apache.hadoop.io.LongWritable;
irmport org.apache .hadoop.io.Tezt;-
import org.apache.hadoop.mapreduce.Mapper;
public class WordcountMapper extends Mapper<Longwritable,Text,Text,Intwritable>(
Text k = new Text(;
Intwritable v - new Intwritable ( 1);
override