自定义 MR1 之 Mapper | 学习笔记

开发者学堂课程【分布式数据库 HBase 快速入门：自定义 MR1 之 Mapper 】学习笔记，与课程紧密联系，让用户快速学习知识。

课程地址：https://developer.aliyun.com/learning/course/101/detail/1755

自定义 MR1 之 Mapper

内容介绍

一、自定义 MR1 之 Mapper

一、自定义 MR1 之 Mapper

自定义 HBase-MapReduce1

目标：将 fruit 表中的一部分数据，通过 MR 迁入到 fruit_mr 表中。

分步实现:

1．构建 ReadFruitMapper 类，用于读取 fruit 表中的数据

package com.atguigu . hbase_mr;  

import java.io.IOException;  

import org.apache.hadoop.hbase.cell;  

import org. apache.hadoop. hbase.cellutil;  

import org.apache.hadoop.hbase.client. Put;  

import org.apache.hadoop.hbase.client. Result;  

Import org.apache.hadoop.hbase.io. ImmutableByteswritable;

import org.apache.hadoop.hbase.mapreduce.TableMapper;

import org.apache.hadoop. hbase.util.Bytes;  

Public              class              ReadFruitMapper           extends

TableMapper {

@override

protected void map(ImmutableByteswritable key,Result value,Context context)  

throws IOException,InterruptedException {

Tota1 time spent by a17 maps in occupied slots (ms)=5222Tota1 time spent by a17 reduces in occupied slots (ms)=0Total time spent by al7 map tasks (ms)=5222

Tota1 vcore-mi11iseconds taken by a17 map tasks=5222

Total megabyte-mil7iseconds taken by a17 map tasks=5347328

Map-Reduce Framework

Map input records=3Map output records=3Input split bytes=96spi77ed Records=oFailed shuffles=oMerged Map outputs=o

Gc time elapsed (ms)=272CPu time spent (ms)=2510

Physical memory (bytes) snapshot=222109696virtua7 memory (bytes) snapshot=3001012224Tota1 committed heap usage' (bytes)=152043520

ImportTsv

Bad Lines=o

File Input Format counters

Bytes Read=54

File output Format counters

Bytes written=o

[atguigu@hadoop102 hbase]$

/**

*Extends the base Mapper class to add the required input Key  

*and value classes.

*

*@param  The type of the key.

*@param  The type of the value.

*@see org.apache. hadoop. mapreduce.Mapper

*/

@InterfaceAudience.Public

@InterfaceStability.Stable

public abstract class TableMapper

extends Mapper{

}

import org.apache.hadoop. hbase. mapreduce.TableMapper;

import org.apache. hadoop.hbase.uti1.Bytes;

import java.io.IOException;

public class FruitMapper extends TableMapper

@Override

protected void map(ImmutableBytesWritable key，Result value，Context context) throws

IOException,InterruptedException {

//构建Put对象

Put put = new Put(key.get()) ;

//遍历数据

Cell[] cells = value.rawCells();

cell[] cells = value.rawCells();

for (Cell cell : cells){

if ("name".equals(Bytes.toString(CellUtil.cloneQualifier(cell)))){

  Putadd(cell);

}

}