本问将介绍如何使用 Oracle到postgresql的迁移使用ora2pg工具。配置环境后,你将了解如何编写配置文件,进行数据库迁移任务。
DBD::Oracle 和`DBD::Pg.perl -MCPAN -e install DBD::Oracleperl -MCPAN -e install DBD::Pgcd ora2pg perl Makefile.PLgmake && gmake installtar -zxvf DBI-1.641.tar.gzcd DBI-1.641perl Makefile.PLgmake && gmake installperl版本5.10以上,安装依赖包 perl-ExtUtils-CBuilder perl-ExtUtils-MakeMaker(注:win下的strawberry-perl自带这两个包)
yum install -y perl perl-ExtUtils-CBuilder perl-ExtUtils-MakeMaker安装DBI
tar -zxvf DBI-1.641.tar.gzcd DBI-1.641perl Makefile.PLmake && make install安装Oracle客户端(略)
rpm -ivh oracle-instantclient12.2-basic-12.2.0.1.0-1.x86_64.rpm rpm -ivh oracle-instantclient12.2-devel-12.2.0.1.0-1.x86_64.rpm rpm -ivh oracle-instantclient12.2-jdbc-12.2.0.1.0-1.x86_64.rpm rpm -ivh oracle-instantclient12.2-sqlplus-12.2.0.1.0-1.x86_64.rpm安装DBD-Oracle
tar -zxvf DBD-Oracle-1.74.tar.gzcd DBD-Oracle-1.74perl Makefile.PLmake && make install安装DBD:PG
#使用官方提供的yum安装方式#Linux系统版本centos7 64位版本#Pg版本为12yum install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpmyum install postgresql12yum install postgresql12-serveryum install postgresql12-contrib#配置开机启动与启动systemctl enable postgresql-12systemctl start postgresql-12#修改配置文件#打开并编辑文件“/var/lib/pgsql/12/data/postgresql.conf”。将“#listen_addresses = ‘localhost’”改为“listen_addresses = ‘*’”#打开并编辑文件“/var/lib/pgsql/12/data/pg_hba.conf”。在文件的末尾添加“host all all 0.0.0.0/0 md5”#设置pgsql的环境变量安装Ora2PG
tar -zxvf ora2pg-22.1.tar.bz2cd ora2pg-22.1/perl Makefile.PLmake && make install编写conf文件,执行指令将Oracle转化为Postgresql
ora2pg -c ora2pg_table.confPG_VERSION 12ORACLE_HOME /home/oracle/app/oracle/product/11.2.0/dbhome_1ORACLE_DSN dbi:Oracle:host=xx.xx.x.xxx;service_name=xxx;port=xxORACLE_USER xxxxxxORACLE_PWD ######USER_GRANTS 1SCHEMA xxxxTYPE TABLEPG_NUMERIC_TYPE 0PG_INTEGER_TYPE 1DEFAULT_NUMERIC float#SKIP fkeys pkeys ukeys indexes checksNLS_LANG AMERICAN_AMERICA.UTF8OUTPUT /home/ora2pg/xxxxx.sql使用ORACLE_HOME作为环境变量,ora2pg可以通过该参数找到Perl的DBD模块连接ORACLE数据库所需要的依赖包。
/home/oracle/app/oracle/product/11.2.0/dbhome_1D:\app\Administrator\product\11.2.0\dbhome_1该参数与Oracle数据库具体配置有关,主要为数据库的host,sid,port等参数
dbi:Oracle:host=oradb_host.myhost.com;sid=DB_SID;port=1521dbi:Oracle:DB_SID#On 18c this could be for example:dbi:Oracle:host=192.168.1.29;service_name=pdb1;port=1521#for the second notation the SID should be declared in the well known#file $ORACLE_HOME/network/admin/tnsnames.ora or in the path given to#the TNS_ADMIN environment variable.#For MySQL the DSN will lool like this:dbi:mysql:host=192.168.1.10;database=sakila;port=3306the 'sid' part is replaced by 'database'.USER和PWD表示Oracle数据库的用户名和密码,选用账号的时候尽量选择DBA权限的账号,以防止数据库扫描时出现的权限问题。GRANT,该值必须设置为0,否者将无效。事务隔离类型,如果需要改变导出数据的默认隔离级别,需要设置该参数,总共由4中事务隔离类型:
在Oarcle数据库和Postgresql数据库中间,使用一台机器,这台机器实现将Oracle数据库的元数据导出,并根据数据库中的表格、序列、视图等内容生成符合Postgresql语法的DDL(data definition language),并保存为相应的 *.sql文件按
our %TYPE = ( 'NUMBER' => 'numeric',# CHAR types limit of 2000 bytes with defaults to 1 if no length# is specified. PG char type has max length set to 8104 so it# should match all needs 'CHAR' => 'char', 'NCHAR' => 'char',# VARCHAR types the limit is 2000 bytes in Oracle 7 and 4000 in# Oracle 8. PG varchar type has max length iset to 8104 so it# should match all needs 'VARCHAR' => 'varchar', 'NVARCHAR' => 'varchar', 'VARCHAR2' => 'varchar', 'NVARCHAR2' => 'varchar', 'STRING' => 'varchar',# The DATE data type is used to store the date and time# information. PG type timestamp should match all needs. 'DATE' => 'timestamp',# Type LONG is like VARCHAR2 but with up to 2Gb. PG type text# should match all needs or if you want you could use blob 'LONG' => 'text',# Character data of variable length 'LONG RAW' => 'bytea',# Types LOB and FILE are like LONG but with up to 4Gb. PG type# text should match all needs or if you want you could use blob# (large object) 'CLOB' => 'text',# A large object containing single-byte characters 'NCLOB' => 'text',# A large object containing national character set data 'BLOB' => 'bytea',# Binary large object# The full path to the external file is returned if destination type is text.# If the destination type is bytea the content of the external file is returned. 'BFILE' => 'bytea',# Locator for external large binary file# The RAW type is presented as hexadecimal characters. The# contents are treated as binary data. Limit of 2000 bytes# PG type text should match all needs or if you want you could# use blob (large object)、 'RAW' => 'bytea', 'ROWID' => 'oid', 'UROWID' => 'oid', 'FLOAT' => 'double precision', 'DEC' => 'decimal', 'DECIMAL' => 'decimal', 'DOUBLE PRECISION' => 'double precision', 'INT' => 'numeric', 'INTEGER' => 'numeric', 'BINARY_INTEGER' => 'integer', 'PLS_INTEGER' => 'integer', 'REAL' => 'real', 'SMALLINT' => 'smallint', 'BINARY_FLOAT' => 'double precision', 'BINARY_DOUBLE' => 'double precision', 'TIMESTAMP' => 'timestamp', 'BOOLEAN' => 'boolean', 'INTERVAL' => 'interval', 'XMLTYPE' => 'xml', 'TIMESTAMP WITH TIME ZONE' => 'timestamp with time zone', 'TIMESTAMP WITH LOCAL TIME ZONE' => 'timestamp with time zone', 'SDO_GEOMETRY' => 'geometry' );| ORACLE | POSTGRESQL |
|---|---|
| Database Link | Foreign Data Wrapper |
| External Table | Foreign Table |
| Synonym | View / Set search_path |
| Global Temporary Table | Unlogged Table / Temp Table |
| Virtual column | View / Function / Trigger |
| Connect by | With Recursive |
| Reverse Index | Functional Index |
| Index Organized Table (IOT table) | Cluster the table according to an Index |
CREATE PUBLIC DATABASE LINK remote_service USING 'remote_db';SELECT * FROM employees@remote_service;CREATE SERVER remote_service FOREIGN DATA WRAPPER oracle_fdw OPTIONS (dbserver 'remote_db');CREATE USER MAPPING FOR current_user SERVER remote_service OPTIONS (user 'scott', password 'tiger');CREATE FOREIGN TABLE employees_fdw (<columns_list>) SERVER remote_service OPTIONS(schema 'HR',table 'EMPLOYEES');假设有这么一张表
cat /data/ext/file_ext.csv1234,ALBERT,GRANT,211235,ALFRED,BLUEOS,261236,BERNY,JOLYSE,34CREATE OR REPLACE DIRECTORY ext_dir AS '/data/ext/';CREATE TABLE ext_table ( empno VARCHAR2(4), firstname VARCHAR2(20), lastname VARCHAR2(20), age VARCHAR2(2)) ORGANIZATION EXTERNAL (DEFAULT DIRECTORY ext_dir ACCESS PARAMETERS (…LOCATION ('file_ext.csv')));CREATE FOREIGN TABLE ext_table( empno VARCHAR(4), firstname VARCHAR(20), lastname VARCHAR(20), age VARCHAR(2))SERVER ext_dir OPTIONS (filename '/data/ext/file_ext.csv', format 'csv', delimiter ',');Synonyms are not supported in PostgreSQL
CREATE PUBLIC SYNONYM emp_table FOR hr.employees [@ dblink];有两种方式search path和view
--search path – session level – no permanent effect, it needs to be set for every connectionSET search_path TO other_schema;--search path – role or database level – it takes permanent effect--@postgresqlalter database <database_name> set search_path = "other_schema";--@database_namealter role <role_name> set search_path = "other_schema";--view:CREATE VIEW public.emp_table AS SELECT * FROM hr.employees;ALTER VIEW public.emp_table OWNER TO hr;GRANT ALL ON public.emp_table TO PUBLIC;Option 1: Unlogged Table:
CREATE GLOBAL TEMPORARY TABLE MY_CONTRACT_MONTH( ID NUMBER(10), CMONTH DATE)ON COMMIT DELETE ROWS;CREATE UNLOGGED TABLE MY_CONTRACT_MONTH( ID number, CMONTH timestamp, pid bigint default pg_backend_pid());ALTER TABLE MY_CONTRACT_MONTH ENABLE ROW LEVEL SECURITY;ALTER TABLE MY_CONTRACT_MONTH FORCE ROW LEVEL SECURITY;CREATE POLICY cm_pid ON MY_CONTRACT_MONTH TO <role_name> USING (pid = (selectpg_backend_pid()));在Postgresql中Role即User,若表格不保存日志,对于 Unlogged 表,必须实现行级安全,以防止会话查询其他会话数据。还需要实施一项作业,以消除非活动会话的未记录表上的数据
DELETE FROM smdr.contract_months cmWHERE not exists (select 1 from pg_stat_activity psa where psa.pid = cm.pid);Option 2: Temp Table:
CREATE GLOBAL TEMPORARY TABLE MY_CONTRACT_MONTH( ID NUMBER(10), CMONTH DATE)ON COMMIT DELETE ROWS;--Oracle 永久存储临时表的定义,就像常规表的定义一样。CREATE GLOBAL TEMPORARY TABLE MY_CONTRACT_MONTH( ID NUMERIC, CMONTH TIMESTAMP)ON COMMIT DELETE ROWS;CREATE TEMPORARY TABLE 语句创建一个临时表,该表在会话或当前事务结束时自动删除(ON COMMIT DROP 选项。在转换过程中,您需要从应用程序代码中提取 CREATE TEMPORARY TABLE 语句,存储过程、触发器等并执行它们一次以创建临时表定义。Oracle 不支持 ON COMMIT DROP,因此如果需要此选项,则需要明确每次提交后执行 DROP TABLE 语句 ON COMMIT PRESERVE ROWS 是 PostgreSQL 中的默认值,ON COMMIT DELETE ROWS 是 Oracle 中的默认值
CREATE TABLE VIRT_COL_TABLE ( id NUMBER, first_name VARCHAR2(10), last_name VARCHAR2(10), salary NUMBER(9,2), comm1 NUMBER(3), comm2 NUMBER(3), salary1 AS (ROUND(salary*(1+comm1/100),2)), salary2 NUMBER GENERATED ALWAYS AS (ROUND(salary*(1+comm2/100),2)) VIRTUAL);CREATE TABLE virt_col_table ( id bigint NOT NULL, first_name varchar(10), last_name varchar(10), salary double precision, comm1 smallint, comm2 smallint, salary1 bigint, salary2 bigint);Ora2pg通过一个包含触发器的定义的额外文件 VIRTUAL_COLUMNS_(...).sql,来实现原始Virtual Column的默认值
DROP TRIGGER IF EXISTS virt_col_VIRT_COL_TABLE_trigger ON VIRT_COL_TABLE CASCADE;CREATE OR REPLACE FUNCTION fct_virt_col_VIRT_COL_TABLE_trigger() RETURNS trigger AS $BODY$BEGINNEW.SALARY2 = ROUND(NEW.SALARY*(1+NEW.COMM2/100),2);NEW.SALARY1 = ROUND(NEW.SALARY*(1+NEW.COMM1/100),2);RETURN NEW;end $BODY$ LANGUAGE 'plpgsql' SECURITY DEFINER;CREATE TRIGGER virt_col_VIRT_COL_TABLE_triggerBEFORE INSERT OR UPDATE ON VIRT_COL_TABLE FOR EACH ROWEXECUTE PROCEDURE fct_virt_col_VIRT_COL_TABLE_trigger();在Oracle通过connect by (子句) 以实现层级遍历,而在PostgreSQL中使用WITH RECURSIVE (子句)
CREATE TABLE taxonomy ( key NUMBER(11) NOT NULL CONSTRAINT taxPkey PRIMARY KEY, value VARCHAR2(255),taxHier NUMBER(11));ALTER TABLE taxonomy ADD CONSTRAINT taxTaxFkey FOREIGN KEY (taxHier) REFERENCEStax(key);SELECT value FROM taxonomy CONNECT BY PRIOR key = taxHier START WITH key = 0;WITH RECURSIVE cte AS ( SELECT key, value, 1 AS level FROM taxonomy WHERE key = 0 UNION ALL SELECT t.key, t.value, c.level + 1 FROM cte c JOIN taxonomy t ON t.taxHier = c.key )SELECT value FROM cte ORDER BY level;当反向索引应用于 TEXT 列时,此解决方法有效。
CREATE TABLE REV_TEMP ( Id NUMBER(10) NOT NULL PRIMARY KEY, Description VARCHAR2(512) NOT NULL);CREATE INDEX REV_TEMP_N1 ON REV_TEMP(Description) REVERSE;CREATE TABLE REV_TEMP ( Id NUMERIC(10) NOT NULL PRIMARY KEY, Description VARCHAR(512) NOT NULL);CREATE INDEX REV_TEMP_N1 ON REV_TEMP(REVERSE(Description));Oracle 数据库默认使用堆表。 可以使用 ORGANIZATION INDEX 子句创建索引组织表,Oracle 数据库始终使用主键作为集群键。
CREATE TABLE IOT_TEMP ( Id NUMBER(10) NOT NULL PRIMARY KEY, Description VARCHAR2(512) NOT NULL) ORGANIZATION INDEX;PostgreSQL 只使用堆表。 但是,使用 CLUSTER 子句将堆表的内容与索引对齐。
CREATE TABLE IOT_TEMP ( Id NUMERIC(10) NOT NULL PRIMARY KEY, Description VARCHAR(512) NOT NULL);CREATE INDEX IOT_TEMP_N1 ON IOT_TEMP(ID);SELECT JOB,WHAT,INTERVAL,SCHEMA_USER FROM ALL_JOBS WHERE SCHEMA_USER='SJCK'