通过mapreduce程序统计旅游订单（wordcount升级版）
本文将结合一个实际的MapReduce程序案例，探讨如何通过分析旅游产品的预订数据来揭示消费者的偏好。
程序概览
首先，让我们来看一下这个MapReduce程序的核心代码。这个程序的目的是处理一个包含旅游产品预订信息的文本文件，并统计每个产品特性的出现次数。Map阶段的代码如下：
public static class Map extends Mapper<LongWritable, Text, Text, IntWritable> { private Text word = new Text(); public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException { if (key.get() > 0) { // 跳过表头 String line = value.toString(); String[] fields = line.split("\t"); if (fields.length > 1 && !fields[1].isEmpty()) { String[] arrstr = Arrays.copyOfRange(fields, 8, fields.length - 1); for(String str:arrstr){ if(StringUtils.isNotBlank(str)){ word.set(str); context.write(word, new IntWritable(1)); } } } } } }
Reduce阶段的代码如下：
public static class Reduce extends Reducer<Text, IntWritable, Text, IntWritable> { public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException { int sum = 0; for (IntWritable val : values) { sum += val.get(); } context.write(key, new IntWritable(sum)); } }
全部代码
package org.example; import java.io.IOException; import java.util.Arrays; import org.apache.commons.lang.StringUtils; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.Path; import org.apache.hadoop.io.*; import org.apache.hadoop.mapreduce.*; public class KeyWord{ public static class Map extends Mapper<LongWritable, Text, Text, IntWritable> { // private final static IntWritable one = new IntWritable(1); private Text word = new Text(); public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException { if (key.get() > 0) { // 跳过表头 String line = value.toString(); String[] fields = line.split("\t"); if (fields.length > 1 && !fields[1].isEmpty()) { String[] arrstr = Arrays.copyOfRange(fields, 8, fields.length - 1); for(String str:arrstr){ if(StringUtils.isNotBlank(str)){ word.set(str); context.write(word, new IntWritable(1)); } } // int a; // if(StringUtils.isNotBlank(fields[4])){ // a = Integer.parseInt(fields[4]); // }else{ // a=0; // } } } } } public static class Reduce extends Reducer<Text, IntWritable, Text, IntWritable> { public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException { int sum = 0; for (IntWritable val : values) { sum += val.get(); } context.write(key, new IntWritable(sum)); } } public void keyWorsds() throws Exception { Configuration conf = new Configuration(); Job job = Job.getInstance(conf, "Word Count on Second Field"); job.setJarByClass(KeyWord.class); job.setMapperClass(Map.class); job.setReducerClass(Reduce.class); job.setOutputKeyClass(Text.class); job.setOutputValueClass(IntWritable.class); job.setMapOutputKeyClass(Text.class); job.setMapOutputValueClass(IntWritable.class); job.setInputFormatClass(org.apache.hadoop.mapreduce.lib.input.TextInputFormat.class); job.setOutputFormatClass(org.apache.hadoop.mapreduce.lib.output.TextOutputFormat.class); org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(job, new Path("/Users/shareit/ds_task_am/wordcount/src/main/resources/mapreduce数据(1).txt")); org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.setOutputPath(job, new Path("/Users/shareit/ds_task_am/wordcount/producttotalhuman")); job.waitForCompletion(true); } }
结论
通过MapReduce程序对旅游产品预订数据的分析，我们能够洞察到消费者的偏好和行为模式。这些信息对于旅游企业来说是宝贵的，可以帮助他们更好地定位市场，设计符合消费者需求的产品，并最终提高客户满意度和市场份额。随着数据分析技术的不断进步，旅游行业将能够更加精准地满足消费者的需求，推动行业的持续发展。
