常用进程
1、将用户信息导出为excel表格(导出数据....)
2、将Excel表中的信息录入到网站数据库(习题上传....)
开发中经常会设计到excel的处理,如导出Excel,导入Excel到数据库中!
操作Excel目前比较流行的就是 Apache POI 和 阿里巴巴的 easyExcel !
Apache POI
Apache POI 官网:Apache POI - the Java API for Microsoft Documents
阿里巴巴 EasyExcel
easyExcel 官网地址:GitHub - alibaba/easyexcel: 快速、简洁、解决大文件内存溢出的java处理Excel工具
官方文档:EasyExcel · 语雀
EasyExcel 是阿里巴巴开源的一个excel处理框架,以使用简单、节省内存著称。
POI-Excel
- maven依赖
<properties> <maven.compiler.source>8</maven.compiler.source> <maven.compiler.target>8</maven.compiler.target> <poiVersion>3.17</poiVersion> <ooxmlVersion>3.17</ooxmlVersion> <jobatimeVersion>2.10.6</jobatimeVersion> <junitVersion>4.13.1</junitVersion> </properties> <dependencies> <!--apach的poi支持 03版本 xls--> <dependency> <groupId>org.apache.poi</groupId> <artifactId>poi</artifactId> <version>${poiVersion}</version> </dependency> <!--apach的poi支持 07版本 xlsx--> <dependency> <groupId>org.apache.poi</groupId> <artifactId>poi-ooxml</artifactId> <version>${ooxmlVersion}</version> </dependency> <!--日期格式化工具--> <dependency> <groupId>joda-time</groupId> <artifactId>joda-time</artifactId> <version>${jobatimeVersion}</version> </dependency> <!--单元测试--> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>${junitVersion}</version> <scope>test</scope> </dependency> </dependencies>
03 | 07 版本,就是对象不同,方法一样的!
需要注意:2003 版本和 2007 版本存在兼容性的问题!03最多只有 65535 行!
- 需要理清概念:
1、工作簿:
2、工作表:
3、行:
4、列: - 03版本xls
/** * @author starrysky * @title: PoiExecl * @projectName Execl_Poi_EasyExecl * @description: poi * @date 2021/2/408:54 */ public class PoiExcel { public static void main(String[] args) { //xls版本03 行数65536 Workbook book = new HSSFWorkbook(); Sheet sheet = book.createSheet("kubernetes学习计划表"); Row row1 = sheet.createRow(0); Cell cell1 = row1.createCell(0); Cell cell2 = row1.createCell(1); cell1.setCellValue("视频名称"); cell2.setCellValue("视频时长"); Row row2 = sheet.createRow(1); Cell cell3 = row2.createCell(0); Cell cell4 = row2.createCell(1); cell3.setCellValue("Poi及EasyExcel01"); cell4.setCellValue(new DateTime().toString("yyyy-MM-dd HH:mm:ss")); FileOutputStream excel01 =null; try { excel01 = new FileOutputStream("excel01.xls"); book.write(excel01); System.out.println("写入完毕"); } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } finally { if (excel01!=null){ try { excel01.close(); } catch (IOException e) { e.printStackTrace(); } } } } }
- 07版本xlsx
/** * @author starrysky * @title: PoiExcel02 * @projectName Excel_Poi_EasyExcel * @description: TODO * @date 2021/2/409:26 */ public class PoiExcel02 { public static void main(String[] args) { //xlsx版 07 行数无限 Workbook workbook = new XSSFWorkbook(); //一个分表 Sheet sheet = workbook.createSheet("学生在线统计表"); // 第一行 Row row1 = sheet.createRow(0); Cell cell1 = row1.createCell(0); Cell cell2 = row1.createCell(1); // 第二行 Row row2 = sheet.createRow(1); Cell cell3 = row2.createCell(0); Cell cell4 = row2.createCell(1); cell1.setCellValue("姓名"); cell2.setCellValue("学号"); cell3.setCellValue("张楠"); cell4.setCellValue("201920201590088"); FileOutputStream fileOutputStream =null; try { fileOutputStream = new FileOutputStream("excel-"+ UUID.randomUUID() +".xlsx"); workbook.write(fileOutputStream); System.out.println("文档生成完毕"); } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } finally { if (fileOutputStream!=null){ try { fileOutputStream.close(); } catch (IOException e) { e.printStackTrace(); } } } } }
大文件写HSSF
缺点:最多只能处理65536行,否则会抛出异常
java.lang.IllegalArgumentException: Invalid row number (65536) outside allowable range (0..65535)
优点:过程中写入缓存,不操作磁盘,最后一次性写入磁盘,速度快
/** * @author starrysky * @title: PoiBigTableV03 * @projectName Excel_Poi_EasyExcel * @description: HSSFWorkbook特点:最大行数65536超出会有异常抛出,写入时会先写到内存,再一次性写到文件,速度快 * @date 2021/2/410:02 */ public class PoiBigTableV03 { public static void main(String[] args) { long startTime = System.currentTimeMillis(); Workbook workbook = new HSSFWorkbook(); Sheet sheet = workbook.createSheet("数字矩阵表"); for (int rowNumber = 0; rowNumber < 65539; rowNumber++) { Row row = sheet.createRow(rowNumber); for (int colNumber = 0; colNumber < 6; colNumber++) { Cell cell = row.createCell(colNumber); cell.setCellValue(UUID.randomUUID().toString().substring(0,5)); } } FileOutputStream xlsBigTablesV03 = null; try { xlsBigTablesV03 = new FileOutputStream("XlsBigTablesV03_oom.xls"); workbook.write(xlsBigTablesV03); System.out.println("写入完毕"); } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } finally { if (xlsBigTablesV03!=null){ try { xlsBigTablesV03.close(); } catch (IOException e) { e.printStackTrace(); } } } long endTime = System.currentTimeMillis(); System.out.println("耗时总时间"+(double)(endTime-startTime)/1000); } }
大文件写XSSF
缺点:写数据时速度非常慢,非常耗内存,也会发生内存溢出,如100万条
优点:可以写较大的数据量,如20万条
/** * @author starrysky * @title: PoiBigTableV07 * @projectName Excel_Poi_EasyExcel * @description: XSSFWorkbook特点:写数据时速度非常慢,非常耗内存,也会发生内存溢出,如100万条,可以写很大数据 * @date 2021/2/410:12 */ public class PoiBigTableV07 { public static void main(String[] args) { long startTime = System.currentTimeMillis(); Workbook workbook = new XSSFWorkbook(); Sheet sheet = workbook.createSheet("数字矩阵表"); for (int rowNumber = 0; rowNumber < 65536; rowNumber++) { Row row = sheet.createRow(rowNumber); for (int colNumber = 0; colNumber < 6; colNumber++) { Cell cell = row.createCell(colNumber); cell.setCellValue(UUID.randomUUID().toString().substring(0,5)); } } FileOutputStream xlsBigTablesV03 = null; try { xlsBigTablesV03 = new FileOutputStream("XlsBigTablesV07.xlsx"); workbook.write(xlsBigTablesV03); System.out.println("写入完毕"); } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } finally { if (xlsBigTablesV03!=null){ try { xlsBigTablesV03.close(); } catch (IOException e) { e.printStackTrace(); } } } long endTime = System.currentTimeMillis(); System.out.println("耗时总时间"+(double)(endTime-startTime)/1000); } }
大文件写SXSSF
优点:可以写非常大的数据量,如100万条甚至更多条,写数据速度快,占用更少的内存
注意:
过程中会产生临时文件,需要清理临时文件
默认由100条记录被保存在内存中,如果超过这数量,则最前面的数据被写入临时文件
如果想自定义内存中数据的数量,可以使用new SXSSFWorkbook ( 数量 )
/** * @author starrysky * @title: PoiBigTableV07First * @projectName Excel_Poi_EasyExcel * @description: TODO * @date 2021/2/410:19 */ public class PoiBigTableV07First { public static void main(String[] args) { long startTime = System.currentTimeMillis(); SXSSFWorkbook workbook = new SXSSFWorkbook(); Sheet sheet = workbook.createSheet("数字矩阵表"); for (int rowNumber = 0; rowNumber < 65536; rowNumber++) { Row row = sheet.createRow(rowNumber); for (int colNumber = 0; colNumber < 6; colNumber++) { Cell cell = row.createCell(colNumber); cell.setCellValue(UUID.randomUUID().toString().substring(0,5)); } } FileOutputStream xlsBigTablesV03 = null; try { xlsBigTablesV03 = new FileOutputStream("XlsBigTablesV07First.xlsx"); workbook.write(xlsBigTablesV03); System.out.println("写入完毕"); } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } finally { if (xlsBigTablesV03!=null){ try { xlsBigTablesV03.close(); workbook.close(); //关闭清理临时文件 workbook.dispose(); } catch (IOException e) { e.printStackTrace(); } } } long endTime = System.currentTimeMillis(); System.out.println("耗时总时间"+(double)(endTime-startTime)/1000); } }
SXSSFWorkbook-来至官方的解释:实现“BigGridDemo”策略的流式XSSFWorkbook版本。这允许写入非常大的文件而不会耗尽内存,因为任何时候只有可配置的行部分被保存在内存中。
请注意,仍然可能会消耗大量内存,这些内存基于您正在使用的功能,例如合并区域,注释......仍然只存储在内存中,因此如果广泛使用,可能需要大量内存。
再使用 POI的时候!内存问题 Jprofile!
2、POI-Excel读
03|07
03版本
/** * @author starrysky * @title: PoiReadV03 * @projectName Excel_Poi_EasyExcel * @description: TODO * @date 2021/2/411:01 */ public class PoiReadV03 { public static void main(String[] args) { FileInputStream stream =null; HSSFWorkbook workbook =null; try { stream = new FileInputStream("excel01.xls"); if (stream!=null){ workbook = new HSSFWorkbook(stream); Sheet sheetAt = workbook.getSheetAt(0); //从sheet中获取行数 for (int rowNumber = 0; rowNumber < sheetAt.getPhysicalNumberOfRows(); rowNumber++) { StringBuilder stringBuilder = new StringBuilder(); Row row = sheetAt.getRow(rowNumber); if (rowNumber!=0){ //某一行中获取列数 for (int colNumber = 0; colNumber < row.getPhysicalNumberOfCells(); colNumber++) { stringBuilder.append("| "); stringBuilder.append(row.getCell(colNumber).getStringCellValue()); stringBuilder.append(" |"); } }else { for (int colNumber = 0; colNumber < row.getPhysicalNumberOfCells(); colNumber++) { stringBuilder.append("| "); stringBuilder.append(row.getCell(colNumber).getStringCellValue()); stringBuilder.append(" |"); } } System.out.println(stringBuilder.toString()); } } } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } finally { if (stream!=null){ try { stream.close(); if (workbook!=null){ workbook.close(); } } catch (IOException e) { e.printStackTrace(); } } } } }
07版本
/** * @author starrysky * @title: PoiReadV07 * @projectName Excel_Poi_EasyExcel * @description: TODO * @date 2021/2/410:30 */ public class PoiReadV07 { public static void main(String[] args) { FileInputStream stream =null; XSSFWorkbook workbook =null; try { stream = new FileInputStream("XlsBigTablesV07First.xlsx"); if (stream!=null){ workbook = new XSSFWorkbook(stream); Sheet sheetAt = workbook.getSheetAt(0); for (int rowNumber = 0; rowNumber < 65536; rowNumber++) { StringBuilder stringBuilder = new StringBuilder(); Row row = sheetAt.getRow(rowNumber); if (rowNumber!=0){ for (int colNumber = 0; colNumber < 6; colNumber++) { stringBuilder.append("| "); stringBuilder.append(row.getCell(colNumber).getNumericCellValue()); stringBuilder.append(" |"); } }else { for (int colNumber = 0; colNumber < 6; colNumber++) { stringBuilder.append("| "); stringBuilder.append(row.getCell(colNumber).getStringCellValue()); stringBuilder.append(" |"); } } System.out.println(stringBuilder.toString()); } } } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } finally { if (stream!=null){ try { stream.close(); if (workbook!=null){ workbook.close(); } } catch (IOException e) { e.printStackTrace(); } } } } }
注意获取值的类型即可
读取不同的数据类型(最麻烦的就是这里了!)
/** * @author starrysky * @title: PoiReadV03 * @projectName Excel_Poi_EasyExcel * @description: 读取不同的数据类型, cell.getCellType()视频中的过时了变成枚举getCellTypeEnum() * @date 2021/2/411:01 */ public class PoiReadV03 { public static void main(String[] args) { FileInputStream stream = null; XSSFWorkbook workbook = null; try { stream = new FileInputStream("XlsBigTablesV07First.xlsx"); if (stream != null) { workbook = new XSSFWorkbook(stream); Sheet sheetAt = workbook.getSheetAt(0); //从sheet中获取行数 for (int rowNumber = 0; rowNumber < sheetAt.getPhysicalNumberOfRows(); rowNumber++) { StringBuilder stringBuilder = new StringBuilder(); Row row = sheetAt.getRow(rowNumber); //某一行中获取列数 for (int colNumber = 0; colNumber < row.getPhysicalNumberOfCells(); colNumber++) { Cell cell = row.getCell(colNumber); if (cell != null) { CellType cellType = cell.getCellTypeEnum(); switch (cellType) { /** * 字符串 */ case STRING: stringBuilder.append(" | "); stringBuilder.append(cell.getStringCellValue()); stringBuilder.append(" | "); break; /** * 布尔 */ case BOOLEAN: stringBuilder.append(" | "); stringBuilder.append(cell.getBooleanCellValue()); stringBuilder.append(" | "); break; /** * 空 */ case BLANK: break; /** * 数字,分时间和普通数字 */ case NUMERIC: /** * 日期 */ if (HSSFDateUtil.isCellDateFormatted(cell)) { Date date = cell.getDateCellValue(); stringBuilder.append(" | "); stringBuilder.append(new DateTime(date).toString("yyyy-MM-dd")); stringBuilder.append(" | "); } else { /** * 不是日期格式,防止数字过长! */ cell.setCellType(CellType.STRING); stringBuilder.append(" | "); stringBuilder.append(cell); stringBuilder.append(" | "); } break; /** * 类型错误 */ case ERROR: break; default: break; } } } System.out.println(stringBuilder); } } } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } finally { if (stream != null) { try { stream.close(); if (workbook != null) { workbook.close(); } } catch (IOException e) { e.printStackTrace(); } } } } }
- ⚠️ Tips:XSSFWorkbook=》xlsx、HSSFWorkbook=》xls、SXSSFWorkbook用于写的xlsx、xls
注意,类型转换问题;
计算公式
/** * @author starrysky * @title: PoiPublicity * @projectName Excel_Poi_EasyExcel * @description: TODO * @date 2021/2/411:58 */ public class PoiPublicity { public static void main(String[] args) { FileInputStream stream = null; XSSFWorkbook workbook = null; try { stream = new FileInputStream("excel.xlsx"); workbook = new XSSFWorkbook(stream); Sheet sheetAt = workbook.getSheetAt(0); Row row = sheetAt.getRow(4); Cell cell = row.getCell(1); FormulaEvaluator evaluator = new XSSFFormulaEvaluator(workbook); /** * 获取单元格的属性是否为公式,枚举! */ CellType cellTypeEnum = cell.getCellTypeEnum(); switch (cellTypeEnum) { case FORMULA: String cellFormula = cell.getCellFormula(); System.out.println(cellFormula); /** * 计算 */ CellValue evaluate = evaluator.evaluate(cell); String value = evaluate.formatAsString(); System.out.println(value); default: break; } } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } finally { if (stream != null) { try { if (stream != null & workbook != null) { stream.close(); workbook.close(); } } catch (IOException e) { e.printStackTrace(); } } } } }
EasyExcel操作
导入依赖,还需要fastjson
<dependency> <groupId>com.alibaba</groupId> <artifactId>easyexcel</artifactId> <version>2.2.0-beta2</version> </dependency>
1、写入测试
1.1、创建对象
@Data public class DemoData { @ExcelProperty("字符串标题") private String string; @ExcelProperty("日期标题") private Date date; @ExcelProperty("数字标题") private Double doubleData; /** * 忽略这个字段 */ @ExcelIgnore private String ignore; }
1.2、拿到实体类里的值
String PATH =="D:\\Project\\IdeaProject\\file\\"; private List<DemoData> data() { List<DemoData> list = new ArrayList<DemoData>(); for (int i = 0; i < 10; i++) { DemoData data = new DemoData(); data.setString("字符串" + i); data.setDate(new Date()); data.setDoubleData(0.56); list.add(data); } return list; }
1.3、将list写入Excel
// 根据list 写入excel @Test public void simpleWrite() { // 写法1 String fileName = PATH + "EasyTest.xlsx"; // 这里 需要指定写用哪个class去写,然后写到第一个sheet,名字为模板 然后文件流会自动关闭 // write (fileName, 格式类) // sheet (表明) // doWrite (数据) EasyExcel.write(fileName, DemoData.class).sheet("模板").doWrite(data()); }
2、读取测试
2.1、对象
2.2、监听器
// 有个很重要的点 DemoDataListener 不能被spring管理,要每次读取excel都要new,然后里面用到spring可以构造方法传进去 public class DemoDataListener extends AnalysisEventListener<DemoData> { private static final Logger LOGGER = LoggerFactory.getLogger(DemoDataListener.class); private static final int BATCH_COUNT = 5; List<DemoData> list = new ArrayList<DemoData>(); private DemoDAO demoDAO; public DemoDataListener() { // 这里是demo,所以随便new一个。实际使用如果到了spring,请使用下面的有参构造函数 demoDAO = new DemoDAO(); } public DemoDataListener(DemoDAO demoDAO) { this.demoDAO = demoDAO; } // 读取数据会执行 invoke 方法 // DemoData 类型 // AnalysisContext 分析上问 @Override public void invoke(DemoData data, AnalysisContext context) { System.out.println(JSON.toJSONString(data)); list.add(data); // 达到BATCH_COUNT了,需要去存储一次数据库,防止数据几万条数据在内存,容易OOM if (list.size() >= BATCH_COUNT) { saveData(); // 持久化逻辑! // 存储完成清理 list list.clear(); } } /** * 所有数据解析完成了 都会来调用 * * @param context */ @Override public void doAfterAllAnalysed(AnalysisContext context) { // 这里也要保存数据,确保最后遗留的数据也存储到数据库 saveData(); LOGGER.info("所有数据解析完成!"); } /** * 加上存储数据库 */ private void saveData() { LOGGER.info("{}条数据,开始存储数据库!", list.size()); demoDAO.save(list); LOGGER.info("存储数据库成功!"); } }
2.3、持久层
/** * 假设这个是你的DAO存储。当然还要这个类让spring管理,当然你不用需要存储,也不需要这个类。 **/ public class DemoDAO { public void save(List<DemoData> list) { // 持久化操作! // 如果是mybatis,尽量别直接调用多次insert,自己写一个mapper里面新增一个方法batchInsert,所有数据一次性插入 } }
2.4、测试代码
@Test public void simpleRead() { // 有个很重要的点 DemoDataListener 不能被spring管理,要每次读取excel都要new,然后里面用到spring可以构造方法传进去 // 写法1: String fileName = PATH + "EasyTest.xlsx"; // 这里 需要指定读用哪个class去读,然后读取第一个sheet 文件流会自动关闭 // 重点注意读取的逻辑 DemoDataListener EasyExcel.read(fileName, DemoData.class, new DemoDataListener()).sheet().doRead(); }
⚠️ Tips:pojo的类上不允许使用lombok的链式调用的注解,否则读取excel表的时候将会读取出空的json字符串
固定套路:
1、写入,固定类格式进行写入
2、读取,根据监听器设置的规则进行读取!