SQL*Loader 笔记 (一) 热身练习

本文涉及的产品
RDS MySQL Serverless 基础系列,0.5-2RCU 50GB
RDS MySQL Serverless 高可用系列,价值2615元额度,1个月
云数据库 RDS PostgreSQL,高可用系列 2核4GB
简介:

SQL*Loader是加载数据的主流方法
传统路径:SQLLDR会利用SQL插入的方式加载数据
直径路径:直接格式化数据块

从一个平面文件读取数据,并将其直接写至格式化的数据库块,而绕开整个SQL引擎,同时还能避免redo、undo的生成。要在一个没有任何数据的数据库中充分加载数据,最快的方法就是采用并行直接路径加载

外部表:允许访问操作系统文件,就好像它们是数据库表一样

数据卸载技术

平面文件卸载(flat file unload)
数据泵卸载(data pump unload)

实验1

文件路径
[oracle@node1 mysqlldr]$ pwd
/home/oracle/mysqlldr
[oracle@node1 mysqlldr]$ ll
total 16
-rw-r--r-- 1 oracle oinstall   76 Jul 22 09:53 afiedt.buf
-rw-r--r-- 1 oracle oinstall  173 Jul 22 09:56 demo1.ctl
-rw-r--r-- 1 oracle oinstall 1611 Jul 22 09:57 demo1.log
-rw-r--r-- 1 oracle oinstall  111 Jul 22 09:47 dept.sql

配置文件demo1.ctl
[oracle@node1 mysqlldr]$ cat demo1.ctl
LOAD DATA
INFILE *
INTO TABLE DEPT
FIELDS TERMINATED BY ','
(DEPTNO,DNAME,LOC)
BEGINDATA
10,Sales,Virginia
20,Accounting,Virginia
30,Consulting,Virginia
40,Finance,Virginia
[oracle@node1 mysqlldr]$

创建表脚本
[oracle@node1 mysqlldr]$ cat dept.sql
create table dept
(deptno number(2) constraint dept_pk primary key,
dname varchar2(14),
loc   varchar2(13)
)
/

执行命令
sqlldr userid=alex/alex control=/home/oracle/mysqlldr/demo1.ctl

实验2

配置文件中第四条记录的最后一个字段有很多的字符
[oracle@node1 mysqlldr]$ cat demo2.ctl
LOAD DATA
INFILE *
INTO TABLE DEPT
FIELDS TERMINATED BY ','
(DEPTNO,DNAME,LOC)
BEGINDATA
10,Sales,Virginia
20,Accounting,Virginia
30,Consulting,Virginia
40,Finance,Virginia text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text moretext more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text moretext more text more text more text more text more text more text more text moretext more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text moretext more text more text more text more

清空之前插入的数据
ALEX@devdb1 >truncate table dept;


执行命令
[oracle@node1 mysqlldr]$ sqlldr userid=alex/alex control=/home/oracle/mysqlldr/demo2.ctl


在demo2.log中出现了exceeds maximum length 最大长度不够
[oracle@node1 mysqlldr]$ cat demo2.log

   Column Name                  Position   Len  Term Encl Datatype
------------------------------ ---------- ----- ---- ---- ---------------------
DEPTNO                              FIRST     *   ,       CHARACTER
DNAME                                NEXT     *   ,       CHARACTER
LOC                                  NEXT     *   ,       CHARACTER

Record 4: Rejected - Error on table DEPT, column LOC.
Field in data file exceeds maximum length

Table DEPT:
  3 Rows successfully loaded.
  1 Row not loaded due to data errors.
  0 Rows not loaded because all WHEN clauses were failed.
  0 Rows not loaded because all fields were null.

数据只成功加载3条,最后一条没有成功
ALEX@devdb1 >select * from dept;

DEPTNO DNAME                                      LOC
---------- ------------------------------------------ ---------------------------------------
        10 Sales                                      Virginia
        20 Accounting                                 Virginia
        30 Consulting                                 Virginia

demo2.bad文件中记录了加载失败的数据
[oracle@node1 mysqlldr]$ cat demo2.bad
40,Finance,Virginia text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text moretext more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text moretext more text more text more text more text more text more text more text moretext more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text moretext more text more text more text more


修改控制文件demo2.ctl,将char默认的255扩展到1000
[oracle@node1 mysqlldr]$ cat demo2.ctl
LOAD DATA
INFILE *
INTO TABLE DEPT
FIELDS TERMINATED BY ','
(DEPTNO,DNAME ,LOC char(1000))
BEGINDATA
10,Sales,Virginia
20,Accounting,Virginia
30,Consulting,Virginia
40,Finance,Virginia text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text moretext more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text moretext more text more text more text more text more text more text more text moretext more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text more text moretext more text more text more text more

更改表dept的字段loc为varchar2(1000)
ALEX@devdb1 >alter table dept modify loc varchar2(1000);

Table altered.

ALEX@devdb1 >desc dept;
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 DEPTNO                                    NOT NULL NUMBER(2)
 DNAME                                              VARCHAR2(14)
 LOC                                                VARCHAR2(1000)

清空表后重新加载

truncate table dept;

$ sqlldr userid=alex/alex control=/home/oracle/mysqlldr/demo2.ctl

验证结果插入成功

select * from dept;

    DEPTNO DNAME
---------- ------------------------------------------
LOC
--------------------------------------------------------------------------------
        10 Sales
Virginia

        20 Accounting
Virginia

        30 Consulting
Virginia


    DEPTNO DNAME
---------- ------------------------------------------
LOC
--------------------------------------------------------------------------------
        40 Finance
Virginia text more text more text more text more text more text more text more t
ext more text more text more text more text more text more text more text more t
ext moretext more text more text more text more text more text more text more te
xt more text more text more text more text more text more text more text more te
xt more text more text more text more text moretext more text more text more tex
t more text more text more text more text moretext more text more text more text
 more text m
ore text more text more text more text more text more text more text
 more text more text more text more text more text more text more text more text

    DEPTNO DNAME
---------- ------------------------------------------
LOC
--------------------------------------------------------------------------------
 more text more text more text more text more text more text more text more text
 moretext more text more text more text more

实验3

配置文件中加入选项“ ”区域中代表一个完整的字段,可忽略其中的,号。

如果字段中含有”,需要用”“来替代
[oracle@node1 mysqlldr]$ cat demo3.ctl
LOAD DATA
INFILE *
INTO TABLE DEPT
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
(DEPTNO,DNAME,LOC)
BEGINDATA
10,Sales,"Virginia,USA"
20,Accounting,"Va,""USA"""
30,Consulting,Virginia
40,Finance,Virginia


ALEX@devdb1 >select * from dept;

    DEPTNO DNAME      LOC
---------- ---------- --------------------
        10 Sales      Virginia,USA   <===字段中包含分隔符 , 
        20 Accounting Va,”USA”    <===字段中包含” 
        30 Consulting Virginia
        40 Finance    Virginia

实验4 使用FILLER参数来过滤掉不需要的列

[oracle@node1 mysqlldr]$ cp demo3.ctl demo4.ctl
[oracle@node1 mysqlldr]$ vi demo4.ctl

LOAD DATA
INFILE *
INTO TABLE DEPT
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
(DEPTNO,DNAME,LOC,dummy1 filler)
BEGINDATA
10,Sales,"Virginia,USA",haha
20,Accounting,"Va,""USA""",haha
30,Consulting,Virginia,haha
40,Finance,Virginia,haha

truncate table dept;

sqlldr userid=alex/alex control=/home/oracle/mysqlldr/demo4.ctl
 
[oracle@node1 mysqlldr]$ cat demo4.log


   Column Name                  Position   Len  Term Encl Datatype
------------------------------ ---------- ----- ---- ---- ---------------------
DEPTNO                              FIRST     *   ,  O(") CHARACTER           
DNAME                                NEXT     *   ,  O(") CHARACTER           
LOC                                  NEXT     *   ,  O(") CHARACTER           
DUMMY1                               NEXT     *   ,  O(") CHARACTER           
  (FILLER FIELD)


[oracle@node1 mysqlldr]$ sqlplus alex/alex

ALEX@devdb1 >select * from dept;

    DEPTNO DNAME           LOC
---------- --------------- ---------------
        10 Sales           Virginia,USA
        20 Accounting      Va,"USA"
        30 Consulting      Virginia
        40 Finance         Virginia

实验5 日期类型的加载

ALEX@devdb1 >alter table dept add last_updated date;

Table altered.

ALEX@devdb1 >desc dept;
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 DEPTNO                                    NOT NULL NUMBER(2)
 DNAME                                              VARCHAR2(14)
 LOC                                                VARCHAR2(1000)
 LAST_UPDATED                                       DATE


[oracle@node1 mysqlldr]$ cat demo5.ctl
LOAD DATA
INFILE *
INTO TABLE DEPT
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
(DEPTNO,DNAME,LOC,LAST_UPDATED date 'yyyy-mm-dd')
BEGINDATA
10,Sales,Virginia,2014-05-01
20,Accounting,"Va,""USA""",2014-06-03
30,Consulting,Virginia,2014-07-22
40,Finance,Virginia,2014-07-23

[oracle@node1 mysqlldr]$ sqlldr userid=alex/alex control=/home/oracle/mysqlldr/demo5.ctl

col dname for a15
col loc for a15

select * from dept;

    DEPTNO DNAME           LOC             LAST_UPDATED
---------- --------------- --------------- -------------------
        10 Sales           Virginia        2014/05/01 00:00:00
        20 Accounting      Va,"USA"        2014/06/03 00:00:00
        30 Consulting      Virginia        2014/07/22 00:00:00
        40 Finance         Virginia        2014/07/23 00:00:00

实验6 函数加载

[oracle@node1 mysqlldr]$ cat demo6.ctl
LOAD DATA
INFILE *
INTO TABLE DEPT
REPLACE
FIELDS TERMINATED BY ','
(DEPTNO,
DNAME "upper(:dname)",
LOC "upper(:loc),
LAST_UPDATED date 'yyyy-mm-dd')
BEGINDATA
10,Sales,Virginia,1998-09-01
20,Accounting,Virginia,2002-08-09
30,Consulting,Virginia,2008-08-08
40,Finance,Virginia,2014-06-03

[oracle@node1 mysqlldr]$ sqlldr userid=alex/alex control=demo6.ctl
ALEX@devdb1 >select * from dept;

    DEPTNO DNAME           LOC             LAST_UPDATED
---------- --------------- --------------- -------------------
        10 SALES           VIRGINIA        1998/09/01 00:00:00
        20 ACCOUNTING      VIRGINIA        2002/08/09 00:00:00
        30 CONSULTING      VIRGINIA        2008/08/08 00:00:00
        40 FINANCE         VIRGINIA        2014/06/03 00:00:00

本文转自ICT时空 dbasdk博客,原文链接: SQL*Loader 笔记 (一) 热身练习,如需转载请自行联系原博主。

相关实践学习
每个IT人都想学的“Web应用上云经典架构”实战
本实验从Web应用上云这个最基本的、最普遍的需求出发,帮助IT从业者们通过“阿里云Web应用上云解决方案”,了解一个企业级Web应用上云的常见架构,了解如何构建一个高可用、可扩展的企业级应用架构。
MySQL数据库入门学习
本课程通过最流行的开源数据库MySQL带你了解数据库的世界。 &nbsp; 相关的阿里云产品:云数据库RDS MySQL 版 阿里云关系型数据库RDS(Relational Database Service)是一种稳定可靠、可弹性伸缩的在线数据库服务,提供容灾、备份、恢复、迁移等方面的全套解决方案,彻底解决数据库运维的烦恼。 了解产品详情:&nbsp;https://www.aliyun.com/product/rds/mysql&nbsp;
相关文章
|
SQL 关系型数据库 MySQL
【MySQL实战笔记】02.一条SQL更新语句是如何执行的-1
【4月更文挑战第4天】SQL更新语句执行涉及查询和日志模块,主要为`redo log`和`binlog`。`redo log`先写日志再写磁盘,保证`crash-safe`;`binlog`记录逻辑日志,支持所有引擎,且追加写入。执行过程分为执行器查找数据、更新内存和`redo log`(prepare状态)、写入`binlog`、提交事务(`redo log`转commit)。两阶段提交确保日志逻辑一致,支持数据库恢复至任意时间点。
142 0
|
9月前
|
SQL 缓存 Java
框架源码私享笔记(02)Mybatis核心框架原理 | 一条SQL透析核心组件功能特性
本文详细解构了MyBatis的工作机制,包括解析配置、创建连接、执行SQL、结果封装和关闭连接等步骤。文章还介绍了MyBatis的五大核心功能特性:支持动态SQL、缓存机制(一级和二级缓存)、插件扩展、延迟加载和SQL注解,帮助读者深入了解其高效灵活的设计理念。
|
12月前
|
SQL 存储 关系型数据库
SQL自学笔记(3):SQL里的DCL,DQL都代表什么?
本文介绍了SQL的基础语言类型(DDL、DML、DCL、DQL),并详细说明了如何创建用户和表格,最后推荐了几款适合初学者的免费SQL实践平台。
649 3
SQL自学笔记(3):SQL里的DCL,DQL都代表什么?
|
12月前
|
SQL 数据挖掘 数据库
SQL自学笔记(2):如何用SQL做简单的检索
本文深入介绍了SQL的基本语法,包括数据查询、过滤、排序、分组及表连接等操作,并通过实际案例展示了SQL在用户研究中的应用,如用户行为分析、用户细分、用户留存分析及满意度调查数据分析。
202 0
SQL自学笔记(2):如何用SQL做简单的检索
|
12月前
|
SQL 数据挖掘 关系型数据库
SQL自学笔记(1):什么是SQL?有什么用?
本文为用户研究新手介绍SQL(结构化查询语言),解释了SQL的基本概念、入门方法及在用户研究中的应用通过实际案例说明,如用户行为分析、用户细分和满意度调查数据分析,展示了SQL在用户研究中的重要作用。
3867 0
SQL自学笔记(1):什么是SQL?有什么用?
|
SQL 关系型数据库 MySQL
【MySQL系列笔记】常用SQL
常用SQL分为三种类型,分别为DDL,DML和DQL;这三种类型的SQL语句分别用于管理数据库结构、操作数据、以及查询数据,是数据库操作中最常用的语句类型。 在后面学习的多表联查中,SQL是分析业务后业务后能否实现的基础,以及后面如何书写动态SQL,以及完成级联查询的关键。
346 6
|
SQL 存储 关系型数据库
【MySQL系列笔记】SQL优化
SQL优化是通过调整数据库查询、索引、表结构和配置参数等方式,提高SQL查询性能和效率的过程。它旨在减少查询执行时间、减少系统资源消耗,从而提升数据库系统整体性能。优化方法包括索引优化、查询重写、表分区、适当选择和调整数据库引擎等。
394 3
|
SQL 存储 数据库
技术好文:SQL查询语句基本练习
技术好文:SQL查询语句基本练习
|
SQL 存储 关系型数据库
技术笔记:MYSQL常用基本SQL语句总结
技术笔记:MYSQL常用基本SQL语句总结
121 0
|
SQL 存储 Oracle
《SQL必知必会》个人笔记
《SQL必知必会》个人笔记
157 1