EDB Postgres MTK 使用小记

简介: EDB Postgres Sever Migration Toolkit 是PPAS自带的一款数据迁移工具。使用MTK可以方便的把目标数据库(ORACLE/MySQL/Sybase/MS SQL Sever)的数据对象和数据内容迁移到 PPAS or PostgreSQL 。 ###安装 MTK 软件 安装方式: 图像化 or command-line 测试案例为 text mode

EDB Postgres Sever Migration Toolkit 是PPAS自带的一款数据迁移工具。使用MTK可以方便的把目标数据库(ORACLE/MySQL/Sybase/MS SQL Sever)的数据对象和数据内容迁移到 PPAS or PostgreSQL 。

安装 MTK 软件

安装方式: 图像化 or command-line
测试案例为 text mode

[root@Opython mtk]# ./edb-migrationtoolkit-51.0.0-1-linux-x64.run 
Language Selection

Please select the installation language
[1] English - English
[2] Japanese - 日本語
[3] Simplified Chinese - 简体中文
[4] Traditional Chinese - 繁体中文
[5] Korean - 한국어
Please choose an option [1] : 
----------------------------------------------------------------------------
Welcome to the EDB Postgres Migration Toolkit Setup Wizard.

----------------------------------------------------------------------------
Please read the following License Agreement. You must accept the terms of this 
agreement before continuing with the installation.

Press [Enter] to continue:
Limited Use Software License Agreement
Version 2.9

IMPORTANT - READ CAREFULLY

...............................................

Please specify the directory where Migration Toolkit will be installed.

Installation Directory [/opt/edb/mtk]: 

----------------------------------------------------------------------------
Setup is now ready to begin installing Migration Toolkit on your computer.

Do you want to continue? [Y/n]: Y

----------------------------------------------------------------------------
Please wait while Setup installs Migration Toolkit on your computer.

 Installing EDB Postgres Migration Toolkit
 0% ______________ 50% ______________ 100%
 #########################################

----------------------------------------------------------------------------
EnterpriseDB is the leading provider of value-added products and services for 
the Postgres community.

Please visit our website at www.enterprisedb.com

安装 ORACLE client oracle-instantclient11.2-basic-11.2.0.4.0-1.x86_64.rpm

下载地址:http://www.oracle.com/technetwork/cn/database/features/instant-client/index-097480.html

[root@Opython mtk]# rpm -ivh oracle-instantclient11.2-basic-11.2.0.4.0-1.x86_64.rpm 
Preparing...                ########################################### [100%]
   1:oracle-instantclient11.########################################### [100%]

[root@Opython mtk]# cp /usr/lib/oracle/11.2/client64/lib/ojdbc6.jar  /usr/lib/jvm/jre-1.7.0/lib/ext/

配置 toolkit.properties 配置文件(位置在 MTK 主目录 /etc 下)

SRC_DB_URL=jdbc:oracle:thin:@1.2.3.4:1528:wprod11g   #Ensure that the oracle instance client is avalible
SRC_DB_USER=system
SRC_DB_PASSWORD=oracle

TARGET_DB_URL=jdbc:edb://5.6.7.8:5444/cww
TARGET_DB_USER=pgmg
TARGET_DB_PASSWORD=oracle

迁移测试

/opt/edb/mtk/bin/runMTK.sh -dropSchema true SCOTT
Running EnterpriseDB Migration Toolkit (Build 51.0.0) ...
Source database connectivity info...
conn =jdbc:oracle:thin:@
user =system
password=******
Target database connectivity info...
conn =jdbc:edb://
user =pgmg
password=******
Connecting with source Oracle database server...
Connected to Oracle, version 'Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options'
Connecting with target EDB Postgres database server...
Connected to EnterpriseDB, version '9.3.1.3'
Importing redwood schema SCOTT...
Dropping Schema: scott 
Creating Schema...scott 
Creating Tables...
Creating Table: BONUS
Creating Table: DEPT
Creating Table: EMP
Creating Table: SALGRADE
Created 4 tables.
Loading Table Data in 8 MB batches...
Loading Table: BONUS ...
[BONUS] Table Data Load Summary: Total Time(s): 0.089 Total Rows: 0
Loading Table: DEPT ...
[DEPT] Migrated 4 rows.
[DEPT] Table Data Load Summary: Total Time(s): 0.051 Total Rows: 4
Loading Table: EMP ...
[EMP] Migrated 14 rows.
[EMP] Table Data Load Summary: Total Time(s): 0.066 Total Rows: 14
Loading Table: SALGRADE ...
[SALGRADE] Migrated 5 rows.
[SALGRADE] Table Data Load Summary: Total Time(s): 0.049 Total Rows: 5
Data Load Summary: Total Time (sec): 0.563 Total Rows: 23 Total Size(MB): 0.0
Creating Constraint: PK_DEPT
Creating Constraint: PK_EMP
Creating Constraint: FK_DEPTNO

Schema SCOTT imported successfully.

MTK-14003: One or more users couldn't be found in the source Oracle database.
With -users mode, the user name should be in uppercase unless it is case-sensitive.

One or more schema objects could not be imported during the migration process. Please review the migration output for more details.

Migration logs have been saved to /root/.enterprisedb/migration-toolkit/logs

******************** Migration Summary ********************
Tables: 4 out of 4
Constraints: 3 out of 3

Total objects: 7
Successful count: 7
Failed count: 0
Invalid count: 0

*************************************************************

数据检测

cww=# \d scott.*
            Table "scott.bonus"
 Column |         Type          | Modifiers 
--------+-----------------------+-----------
 ename  | character varying(10) | 
 job    | character varying(9)  | 
 sal    | numeric               | 
 comm   | numeric               | 

             Table "scott.dept"
 Column |         Type          | Modifiers 
--------+-----------------------+-----------
 deptno | numeric(2,0)          | not null
 dname  | character varying(14) | 
 loc    | character varying(13) | 
Indexes:
    "pk_dept" PRIMARY KEY, btree (deptno)
Referenced by:
    TABLE "scott.emp" CONSTRAINT "fk_deptno" FOREIGN KEY (deptno) REFERENCES scott.dept(deptno)

                 Table "scott.emp"
  Column  |            Type             | Modifiers 
----------+-----------------------------+-----------
 empno    | numeric(4,0)                | not null
 ename    | character varying(10)       | 
 job      | character varying(9)        | 
 mgr      | numeric(4,0)                | 
 hiredate | timestamp without time zone | 
 sal      | numeric(7,2)                | 
 comm     | numeric(7,2)                | 
 deptno   | numeric(2,0)                | 
Indexes:
    "pk_emp" PRIMARY KEY, btree (empno)
Foreign-key constraints:
    "fk_deptno" FOREIGN KEY (deptno) REFERENCES scott.dept(deptno)

       Index "scott.pk_dept"
 Column |     Type     | Definition 
--------+--------------+------------
 deptno | numeric(2,0) | deptno
primary key, btree, for table "scott.dept"

        Index "scott.pk_emp"
 Column |     Type     | Definition 
--------+--------------+------------
 empno  | numeric(4,0) | empno
primary key, btree, for table "scott.emp"

    Table "scott.salgrade"
 Column |  Type   | Modifiers 
--------+---------+-----------
 grade  | numeric | 
 losal  | numeric | 
 hisal  | numeric | 

EDB Migration Toolkit 支持数据对象类型

粘贴图片.png

MTK tools常用参数:

MTK 工具可根据需求使用参数灵活使用

-verbose [on|off] Display application log messages on standard output (default: on).

-schemaOnly Import the schema object definitions only.
-dataOnly Import the table data only. 
-sourcedbtype db_type The -sourcedbtype option specifies the source database type. 
-targetdbtype db_type The -targetdbtype option specifies the target database type. 
-allTables Import all tables.
-tables LIST Import comma-separated list of tables.
-constraints Import the table constraints.
-indexes Import the table indexes.
-triggers Import the table triggers.
-allViews Import all Views.
-views LIST Import comma-separated list of Views.
-allProcs Import all stored procedures.
-procs LIST Import comma-separated list of stored procedures.
-allFuncs Import all functions.
-funcs LIST Import comma-separated list of functions.
-allPackages Import all packages.
-packages LIST Import comma-separated list of packages.
-allSequences Import all sequences.
-sequences LIST Import comma-separated list of sequences.
-targetSchema NAME Name of the target schema (default: target schema is named after source schema).
-allDBLinks Import all Database Links.
-allSynonyms It enables the migration of all public and private synonyms from an Oracle database to an Advanced Server database.  If a synonym with the same name already exists in the target database, the existing synonym will be replaced with the migrated version.
-allPublicSynonyms It enables the migration of all public synonyms from an Oracle database to an Advanced Server database.  If a synonym with the same name already exists in the target database, the existing synonym will be replaced with the migrated version.
-allPrivateSynonyms It enables the migration of all private synonyms from an Oracle database to an Advanced Server database.  If a synonym with the same name already exists in the target database, the existing synonym will be replaced with the migrated version.
-allQueues Import all queues from the source database.
-queues LIST Import the selected queues from the source database. LIST is a comma-separated list of queue names.
-skipUserSchemaCreation This option prevents implicit schema creation for a migrated Oracle user. 

-dropSchema [true|false] Drop the schema if it already exists in the target database (default: false).
-truncLoad It disables any constraints on target table and truncates the data from the table before importing new data. This option can only be used with -dataOnly.
-safeMode Transfer data in safe mode using plain SQL statements.
-copyDelimiter Specify a single character to be used as delimiter in copy command when loading table data. Default is \t
-batchSize Specify the Batch Size to be used by the bulk inserts. Valid values are  1-50000, default batch size is 1000, reduce if you run into Out of Memory exception
-cpBatchSize  Specify the Batch Size in MB, to be used in the Copy Command. Valid value is from 1 to 1024, default batch size is 8 MB.
-fetchSize  Specify fetch size in terms of number of rows should be fetched in result set at a time. This option can be used when tables contain millions of rows and you want to avoid out of memory errors.
-filterProp The properties file that contains table where clause.
-skipFKConst Skip migration of FK constraints.
-skipCKConst Skip migration of Check constraints.
-ignoreCheckConstFilter By default MTK does not migrate Check constraints and Default clauses from Sybase, use this option to turn off this filter.
-fastCopy Bypass WAL logging to perform the COPY operation in an optimized way, default disabled.
-customColTypeMapping LIST Use custom type mapping represented by a semi-colon separated list, where each entry is specified using COL_NAME_REG_EXPR=TYPE pair. e.g. .*ID=INTEGER
-customColTypeMappingFile PROP_FILE The custom type mapping represented by a properties file, where each entry is specified using COL_NAME_REG_EXPR=TYPE pair. e.g. .*ID=INTEGER
-offlineMigration [PATH] This performs offline migration and saves the DDL/DML scripts in files for a later execution. By default the script files will be saved under user home folder, if required follow -offlineMigration option with a custom path. 
-logDir LOG_PATH Specify a custom path to save the log file. By default, on Linux the logs will be saved under folder $HOME/.enterprisedb/migration-toolkit/logs. In case of Windows logs will be saved under folder %HOMEDRIVE%%HOMEPATH%\.enterprisedb\migration-toolkit\logs.
-copyViaDBLinkOra This option can be used to copy data using dblink_ora COPY command. This option can only be used in Oracle to EnterpriseDB migration mode.
-singleDataFile Use single SQL file for offline data storage for all tables. This option cannot be used in COPY format.
-allUsers Import all users and roles from the source database.
-users LIST Import the selected users/roles from the source database. LIST is a comma-separated list of user/role names e.g. -users MTK,SAMPLE
-allProfiles Import all profiles from the source database.
-profiles LIST Import the selected profiles from the source database. LIST is a comma-separated list of profile names e.g. -profiles USER_PROFILE,ADMIN_PROFILE
-allRules Import all rules from the source database.
-rules LIST Import the selected rules from the source database. LIST is a comma-separated list of rule names e.g. -rules high_sal_emp,low_sal_emp
-allGroups Import all groups from the source database.
-groups LIST Import the selected groups from the source database. LIST is a comma-separated list of group names e.g. -groups acct_emp,mkt_emp
-allDomains Import all domain, enumeration and composite types from the source database.
-domains LIST Import the selected domain, enumeration and composite types from the source database. LIST is a comma-separated list of domain names e.g. -domains d_email,d_dob, mood
-objecttypes Import the user-defined object types.
-replaceNullChar <CHAR> If null character is part of a column value, the data migration fails over JDBC protocol. This option can be used to replace null character with a user-specified character.
-importPartitionAsTable [LIST] Use this option to import Oracle Partitioned table as a normal table in EnterpriseDB. To apply the rule on a selected set of tables, follow the option by a comma-separated list of table names.
-enableConstBeforeDataLoad Use this option to re-enable constraints (and triggers) before data load. This is useful in the scenario when the migrated table is mapped to a partition table in EnterpriseDB.
-checkFunctionBodies [true|false] When set to false, it disables validation of the function body during function creation, this is to avoid errors if function contains forward references. Applicable when target database is Postgres/EnterpriseDB, default is true.
-retryCount VALUE Specify the number of re-attempts performed by MTK to migrate objects that failed due to cross-schema dependencies. The VALUE parameter should be greater than 0, default is 2.
-analyze  It invokes ANALYZE operation against a target Postgres or Postgres Plus Advanced Server database. The ANALYZE collects statistics for the migrated tables that are utilized for efficient query plans.
-vacuumAnalyze  It invokes VACUUM and ANALYZE operations against a target Postgres or Postgres Plus Advanced Server database. The VACUUM reclaims dead tuple storage whereas ANALYZE collects statistics for the migrated tables that are utilized for efficient query plans.
-loaderCount VALUE Specify the number of jobs (threads) to perform data load in parallel. The VALUE parameter should be greater than 0, default is 1.
-logFileSize VALUE It represents the maximum file size limit (in MB) before rotating to a new log file, defaults to 50MB.
-logFileCount VALUE It represents the number of files to maintain in log file rotation history, defaults to 20. Specify a value of zero to disable log file rotation.
-useOraCase It preserves the identifier case while migrating from Oracle, except for functions, procedures and packages unless identifier names are given in quotes. 
-logBadSQL It saves the DDL scripts for the objects that fail to migrate, in a .sql file in log folder.
-targetDBVersion It represents the major.minor version of the target database. This option is applicable for offline migration mode and is used to validate certain migration options as per target db version [default is 9.6 for EnterpriseDB database]

MTK 可用 ./runMTK.sh -help 查看更详细功能介绍

目录
相关文章
|
网络安全 Nacos 数据安全/隐私保护
nacos常见问题之使用默认用户名密码提示错误如何解决
Nacos是阿里云开源的服务发现和配置管理平台,用于构建动态微服务应用架构;本汇总针对Nacos在实际应用中用户常遇到的问题进行了归纳和解答,旨在帮助开发者和运维人员高效解决使用Nacos时的各类疑难杂症。
|
2月前
|
SQL 存储 关系型数据库
PL/pgSQL 入门教程(一):语法篇
本教程为PL/pgSQL入门首篇,系统讲解其核心基础与语法规则。涵盖函数创建、块结构、变量声明、参数传递、返回类型及排序规则等关键知识点,助你掌握在PostgreSQL中编写高效存储过程与函数的必备技能,提升数据库逻辑处理能力。
|
安全 数据挖掘 定位技术
工厂内部导航系统:高精度定位与智能路径规划的技术实现
工厂内部导航系统其核心功能包括实时定位、智能路径规划、车辆警告及数据分析,显著提升了物流效率和管理水平。系统具备高精度定位、灵活部署及跨平台兼容等技术优势,并已在实际项目中取得显著成效。
731 11
工厂内部导航系统:高精度定位与智能路径规划的技术实现
|
搜索推荐 关系型数据库 MySQL
MySQL中的模糊匹配技巧:无需ES的高效实现
在数据库应用中,模糊匹配是一个常见的需求,尤其在处理搜索功能时。虽然Elasticsearch(ES)等搜索引擎在处理文本搜索方面表现出色,但在一些场景下,直接使用MySQL数据库实现模糊匹配也是一个经济且高效的选择。本文将分享如何在不引入ES的情况下,利用MySQL实现模糊匹配的五大步骤和十个实战案例。
1129 1
|
安全 数据安全/隐私保护
同态加密含义以及应用场景
文章探讨了同态加密技术的含义、发展历程、技术路线以及在安全求交、隐匿查询、多方联合计算和建模等隐私计算场景中的应用,并分析了其在实际应用中面临的关键问题和研究发展方向,同时指出了同态加密可能导致的计算精度损失和效率降低。
1387 0
同态加密含义以及应用场景
|
人工智能 Oracle 关系型数据库
阿里云宣布与数据库厂商EnterpriseDB(EDB)达成深度合作 提供优秀的Oracle兼容性
自2015年起,阿里云已经与EnterpriseDB公司就云数据库产品进行业务合作,基于阿里云飞天架构及EDB Postgres Advanced Server推出 云数据库PPAS版。针对Postgres市场的持续升温,阿里云将与EDB公司加强源代码级别的技术合作,为全球用户提供基于云计算架构,更优秀的Postgres云数据库服务及产品。
4671 0
|
Oracle 关系型数据库 数据库
|
自然语言处理 网络协议 编译器
gRPC和Ice远程调用协议对比
gRPC和Ice远程调用协议对比
450 0
|
设计模式 Java 应用服务中间件
掌握这30个设计模式真实案例,挑战年薪60W不是梦
Design Patterns: Elements of Reusable Object-Oriented Software(以下简称《设计模式》),一书由Erich Gamma、Richard Helm、Ralph Johnson和John Vlissides合著(Addison-Wesley,1995)。这四位作者常被称为“四人组(Gang of Four)”,而这本书也就被称为“四人组(或 GoF)”书。他们首次给我们总结出一套软件开发可以反复使用的经验,帮助我们提高代码的可重用性、系统的可维护性等,解决软件开发中的复杂问题。
608 3
|
消息中间件 监控 Cloud Native
基于SpringCloud体系实现的一套支持云原生的分布式微服务架构,提供OAuth2/JWT权限认证、分布式事务、灰度、限流、链路追踪等功能,支持Docker容器化部署、镜像交付、K8S容器编排
lion是基于Spring Cloud体系实现的一套支持云原生的分布式微服务架构,为了让中小型公司解决当下技术瓶颈,快速将现有应用服务架构拆分改造为分布式微服务架构,进入 All-in-Cloud 时代,只需在本架构上进行相关业务开发即可,大大减少了分布式微服务架构的门槛,仅在本框架上做&quot;减法&quot;的目的,使架构师及开发人员不必过多的关注架构本身,只需专注于业务开发
基于SpringCloud体系实现的一套支持云原生的分布式微服务架构,提供OAuth2/JWT权限认证、分布式事务、灰度、限流、链路追踪等功能,支持Docker容器化部署、镜像交付、K8S容器编排

热门文章

最新文章