{"id":2761,"date":"2025-01-06T15:34:54","date_gmt":"2025-01-06T15:34:54","guid":{"rendered":"https:\/\/bynatree.com\/?p=2761"},"modified":"2025-01-06T15:34:54","modified_gmt":"2025-01-06T15:34:54","slug":"database-migration-from-postgres-to-mysql-using-pg2mysql-tool","status":"publish","type":"post","link":"https:\/\/divaind.com\/ie1\/2025\/01\/06\/database-migration-from-postgres-to-mysql-using-pg2mysql-tool\/","title":{"rendered":"Database Migration from postgres to mysql using pg2mysql tool"},"content":{"rendered":"<h2><b>Introduction<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">In today&#8217;s dynamic corporate climate, data migration between database systems is a deliberate move that coincides with changing organizational demands. This block describes the migration of PostgreSQL to MySQL. This change necessitates meticulous preparation to handle possible issues and guarantee smooth interaction with existing business procedures.<\/span><\/p>\n<p>&nbsp;<\/p>\n<h2><b>Basic Differences<\/b><\/h2>\n<h3><b>PostgreSQL<\/b><\/h3>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><b>Advanced Feature Set<\/b><span style=\"font-weight: 400;\">: PostgreSQL is known for its rich feature set, including support for complex data types, advanced indexing, and strong compliance with SQL standards.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><b>Robust Concurrency<\/b><span style=\"font-weight: 400;\">: Utilizes MVCC (Multi-Version Concurrency Control) to manage high concurrency with minimal locking.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><b>Extensible<\/b><span style=\"font-weight: 400;\">: Highly extensible with support for custom datatypes, indexes, and procedural languages.<\/span><\/li>\n<\/ul>\n<h3><b>MySQL<\/b><\/h3>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><b>Performance-Oriented<\/b><span style=\"font-weight: 400;\">: MySQL is optimized for high performance, especially in read-heavy applications, making it a popular choice for web applications.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><b>Cost-Effective<\/b><span style=\"font-weight: 400;\">: Often associated with lower licensing fees and operational costs.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><b>Extensive Ecosystem<\/b><span style=\"font-weight: 400;\">: Supported by a large community and numerous plugins for enhanced functionality.<\/span><\/li>\n<\/ul>\n<h2><b>Data Type Differences<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">Understanding the data type differences between PostgreSQL and MySQL is critical for a successful move. Here are some important distinctions:<\/span><\/p>\n<p>&nbsp;<\/p>\n<table>\n<tbody>\n<tr>\n<td><b>PostgreSQL<\/b><\/td>\n<td><b>MySQL<\/b><\/td>\n<td><b>Notes<\/b><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">SERIAL<\/span><\/td>\n<td><span style=\"font-weight: 400;\">INT AUTO_INCREMENT<\/span><\/td>\n<td><span style=\"font-weight: 400;\">MySQL uses AUTO_INCREMENT for auto-incrementing fields.<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">BOOLEAN<\/span><\/td>\n<td><span style=\"font-weight: 400;\">TINYINT(1)<\/span><\/td>\n<td><span style=\"font-weight: 400;\">MySQL doesn\u2019t have a BOOLEAN type; uses TINYINT(1) instead.<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">BYTEA<\/span><\/td>\n<td><span style=\"font-weight: 400;\">BLOB \/ BINARY<\/span><\/td>\n<td><span style=\"font-weight: 400;\">MySQL uses BLOB or BINARY for binary data; careful attention needed for conversions.<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">TEXT<\/span><\/td>\n<td><span style=\"font-weight: 400;\">TEXT<\/span><\/td>\n<td><span style=\"font-weight: 400;\">Both systems support TEXT, but handling and performance may vary.<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">JSON<\/span><\/td>\n<td><span style=\"font-weight: 400;\">JSON<\/span><\/td>\n<td><span style=\"font-weight: 400;\">Both support a JSON type but with different manipulation functions.<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">ARRAY<\/span><\/td>\n<td><span style=\"font-weight: 400;\">N\/A<\/span><\/td>\n<td><span style=\"font-weight: 400;\">MySQL lacks native ARRAY type; use JSON or join tables for similar functionality.<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">UUID<\/span><\/td>\n<td><span style=\"font-weight: 400;\">VARCHAR(36)<\/span><\/td>\n<td><span style=\"font-weight: 400;\">MySQL typically stores UUIDs as VARCHAR(36).<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">TIMESTAMP&#8221;<\/span><\/td>\n<td><span style=\"font-weight: 400;\">TIMESTAMP<\/span><\/td>\n<td><span style=\"font-weight: 400;\">Both support TIMESTAMP, but default behaviors and timezone handling may differ.<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<p><b>Migration Tools from PostgreSQL to MySQL:<\/b><\/p>\n<p><span style=\"font-weight: 400;\">Several solutions can help with the conversion from PostgreSQL to MySQL, preserving data integrity while decreasing manual labor. Here are some popular options:<\/span><\/p>\n<p>&nbsp;<\/p>\n<h3><span style=\"font-weight: 400;\">SQLines:<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">SQLines&#8217; features include data transfer, database schema (DDL) conversion, procedures, and scripting. It supports a large number of PostgreSQL and MySQL versions .<\/span><\/p>\n<p><span style=\"font-weight: 400;\">It may require tweaking for complicated schema configurations. Performance might vary according to data amount.<\/span><\/p>\n<h3><\/h3>\n<h3><span style=\"font-weight: 400;\">MySQL Workbench Migration Wizard:<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">Features: A step-by-step wizard guides you through database conversion, and it supports a variety of RDBMS systems, including PostgreSQL .<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Limitations: On big datasets, GUI difficulties and performance bottlenecks might degrade the user experience. Some manual interventions may be necessary.<\/span><\/p>\n<p>&nbsp;<\/p>\n<h3><span style=\"font-weight: 400;\">Hevo Data:<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">Features: Provides an automated pipeline for data migration that requires little setup. It also allows for continuous data integration between PostgreSQL and MySQL via live sync.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Limitations: A paid tool requiring license fees. There may be limits for certain sophisticated data types or formats.<\/span><\/p>\n<p>&nbsp;<\/p>\n<h3><span style=\"font-weight: 400;\">pg2mysql:<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">Features: A script for converting PostgreSQL dump files to MySQL format, including CREATE TABLE, INSERT INTO, CREATE INDEX, and ALTER TABLE commands.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Limitations include known concerns with complicated schema items such as triggers and stored procedures. Requires manual intervention for certain data type conversions.<br \/>\n<\/span><\/p>\n<blockquote><p><span style=\"font-weight: 400;\">In this blog we are going to discuss the pg2mysql tool. It&#8217;s an open source migration tool.<\/span><\/p><\/blockquote>\n<h2><span style=\"font-weight: 400;\">Data Migration using pg2mysql:<\/span><\/h2>\n<h3><span style=\"font-weight: 400;\">Installing pg2mysql from github repository.<\/span><\/h3>\n<pre class=\"theme:solarized-dark lang:sh decode:true \">[postgres@mysql-01 ~]$ wget https:\/\/github.com\/dolthub\/pg2mysql\/archive\/refs\/heads\/main.zip\n--2024-12-31 12:07:12--  https:\/\/github.com\/dolthub\/pg2mysql\/archive\/refs\/heads\/main.zip\nResolving github.com (github.com)... 20.207.73.82\nConnecting to github.com (github.com)|20.207.73.82|:443... connected.\nHTTP request sent, awaiting response... 302 Found\nLocation: https:\/\/codeload.github.com\/dolthub\/pg2mysql\/zip\/refs\/heads\/main [following]\n--2024-12-31 12:07:15--  https:\/\/codeload.github.com\/dolthub\/pg2mysql\/zip\/refs\/heads\/main\nResolving codeload.github.com (codeload.github.com)... 20.207.73.88\nConnecting to codeload.github.com (codeload.github.com)|20.207.73.88|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: unspecified [application\/zip]\nSaving to: 'main.zip'\n\nmain.zip                                    [ &lt;=&gt;                                                                          ]  18.72K  --.-KB\/s    in 0.04s\n\n2024-12-31 12:07:16 (531 KB\/s) - 'main.zip' saved [19169]\n\n<\/pre>\n<p><span style=\"font-weight: 400;\">Unzip main.zip file after installation:<br \/>\n<\/span><\/p>\n<pre class=\"theme:solarized-dark lang:sh decode:true\">[postgres@mysql-01 ~]$ unzip main.zip\nArchive:  main.zip\ndd956ff207178dc67750568a375f50025cc37b38\n   creating: pg2mysql-main\/\n  inflating: pg2mysql-main\/LICENSE\n  inflating: pg2mysql-main\/README.md\n  inflating: pg2mysql-main\/pg2mysql.pl\n   creating: pg2mysql-main\/test\/\n  inflating: pg2mysql-main\/test\/auto-increment.bats\n  inflating: pg2mysql-main\/test\/basic-types.bats\n  inflating: pg2mysql-main\/test\/begin-end.bats\n  inflating: pg2mysql-main\/test\/cli-args.bats\n  inflating: pg2mysql-main\/test\/common.bash\n  inflating: pg2mysql-main\/test\/indexes-keys-constraints.bats\n  inflating: pg2mysql-main\/test\/unsupported-types.bats\n\n[postgres@mysql-01 ~]$ cd pg2mysql-main\/\n[postgres@mysql-01 pg2mysql-main]$ ll\ntotal 36\n-rw-r--r-- 1 postgres postgres 11358 Oct  9  2023 LICENSE\n-rwxr-xr-x 1 postgres postgres 20249 Oct  9  2023 pg2mysql.pl\n-rw-r--r-- 1 postgres postgres  1778 Oct  9  2023 README.md\ndrwxr-xr-x 2 postgres postgres   186 Oct  9  2023 test\n<\/pre>\n<p><span style=\"font-weight: 400;\">Creating a dummy database in postgres and creating four tables: employees, departments, project_assignments, products.<\/span><\/p>\n<pre class=\"theme:solarized-dark lang:pgsql decode:true \">postgres=# create database dummy;\nCREATE DATABASE\npostgres=# \\c dummy\nYou are now connected to database \"dummy\" as user \"postgres\".\ndummy=#CREATE TABLE employees (\ndummy=#    employee_id INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,\ndummy=#    first_name VARCHAR(50) NOT NULL,\ndummy=#    last_name VARCHAR(50) NOT NULL,\ndummy=#    email VARCHAR(100) UNIQUE NOT NULL,\ndummy=#    hire_date DATE NOT NULL,\ndummy=#    salary NUMERIC(10, 2) CHECK(salary &gt; 0)\ndummy=#);\nCREATE TABLE\ndummy=#CREATE TABLE departments (\ndummy=#    department_id INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,\ndummy=#    department_name VARCHAR(100) NOT NULL\ndummy=#);\nCREATE TABLE\ndummy=#CREATE TABLE project_assignments (\ndummy=#    employee_id INT,\ndummy=#    project_id INT,\ndummy=#    assignment_date DATE NOT NULL,\ndummy=#    PRIMARY KEY (employee_id, project_id)\ndummy=#);\nCREATE TABLE\ndummy=#CREATE TABLE products (\ndummy=#    product_id INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,\ndummy=#    product_name VARCHAR(100) NOT NULL,\ndummy=#    price NUMERIC(10, 2) DEFAULT 0.00,\ndummy=#    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP\ndummy=#);\nCREATE TABLE\n\n<\/pre>\n<p><span style=\"font-weight: 400;\">Checking tables:<\/span><\/p>\n<pre class=\"theme:solarized-dark lang:pgsql decode:true \">dummy=# \\dt+\n                                            List of relations\n Schema |        Name         | Type  |  Owner   | Persistence | Access method |    Size    | Description\n--------+---------------------+-------+----------+-------------+---------------+------------+-------------\n public | departments         | table | postgres | permanent   | heap          | 8192 bytes |\n public | employees           | table | postgres | permanent   | heap          | 8192 bytes |\n public | products            | table | postgres | permanent   | heap          | 8192 bytes |\n public | project_assignments | table | postgres | permanent   | heap          | 8192 bytes |\n(4 rows)<\/pre>\n<p><span style=\"font-weight: 400;\">Verification:<\/span><\/p>\n<pre class=\"theme:solarized-dark lang:pgsql decode:true\">dummy=# \\d\n                List of relations\n Schema |        Name         | Type  |  Owner\n--------+---------------------+-------+----------\n public | departments         | table | postgres\n public | employees           | table | postgres\n public | products            | table | postgres\n public | project_assignments | table | postgres\n(4 rows)\n<\/pre>\n<h3><span style=\"font-weight: 400;\">Taking backup of database in postgres &amp; convert to MySQL schema:<\/span><\/h3>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">Here I used the pg_dump to take the backup in the SQL format and convert the file to MySQL compatible format using the pg2mysql script. It&#8217;s written in perl.<\/span><\/p>\n<pre class=\"theme:solarized-dark lang:pgsql decode:true \">[postgres@mysql-01 ~]$ pg_dump -U postgres -p 5432 -d dummy -f \/var\/lib\/pgsql\/download\/dummy.sql\n[postgres@mysql-01 ~]$ \/var\/lib\/pgsql\/pg2mysql-main\/.\/pg2mysql.pl &lt; \/var\/lib\/pgsql\/download\/dummy.sql &gt; \/var\/lib\/pgsql\/download\/dummy1.sql 2&gt;warnings.txt<\/pre>\n<h3><span style=\"font-weight: 400;\">Restoring database in mysql:<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">Execute the SQL file in MySQL to restore the database.<\/span><\/p>\n<pre class=\"theme:solarized-dark lang:sh decode:true \">[root@mysql-01 pgsql]# mysql -u root -p &lt; \/var\/lib\/pgsql\/download\/dummy1.sql\nEnter password:\n\n<\/pre>\n<h3><span style=\"font-weight: 400;\">Verification Steps:<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">Note: In mysql schema is called database.<\/span><span style=\"font-weight: 400;\"><br \/>\n<\/span><\/p>\n<pre class=\"theme:solarized-dark lang:mysql decode:true \">mysql&gt; \\u public\nReading table information for completion of table and column names\nYou can turn off this feature to get a quicker startup with -A\n\nDatabase changed\nmysql&gt; show tables;\n+---------------------+\n| Tables_in_public    |\n+---------------------+\n| departments         |\n| employees           |\n| products            |\n| project_assignments |\n+---------------------+\n4 rows in set (0.00 sec)\n\nmysql&gt; desc departments;\n+-----------------+--------------+------+-----+---------+-------+\n| Field           | Type         | Null | Key | Default | Extra |\n+-----------------+--------------+------+-----+---------+-------+\n| department_id   | int          | NO   | PRI | NULL    |       |\n| department_name | varchar(100) | NO   |     | NULL    |       |\n+-----------------+--------------+------+-----+---------+-------+\n2 rows in set (0.07 sec)\n\nmysql&gt; desc employees;\n+---------------+---------------+------+-----+---------+-------+\n| Field         | Type          | Null | Key | Default | Extra |\n+---------------+---------------+------+-----+---------+-------+\n| employee_id   | int           | NO   | PRI | NULL    |       |\n| first_name    | varchar(50)   | NO   |     | NULL    |       |\n| last_name     | varchar(50)   | NO   |     | NULL    |       |\n| email         | varchar(100)  | NO   | UNI | NULL    |       |\n| hire_date     | date          | NO   |     | NULL    |       |\n| salary        | decimal(10,2) | YES  |     | NULL    |       |\n| department_id | int           | YES  | MUL | NULL    |       |\n+---------------+---------------+------+-----+---------+-------+\n7 rows in set (0.00 sec)\n\nmysql&gt; desc products;\n+--------------+---------------+------+-----+-------------------+-------------------+\n| Field        | Type          | Null | Key | Default           | Extra             |\n+--------------+---------------+------+-----+-------------------+-------------------+\n| product_id   | int           | NO   | PRI | NULL              |                   |\n| product_name | varchar(100)  | NO   |     | NULL              |                   |\n| price        | decimal(10,2) | YES  |     | 0.00              |                   |\n| created_at   | timestamp     | YES  |     | CURRENT_TIMESTAMP | DEFAULT_GENERATED |\n+--------------+---------------+------+-----+-------------------+-------------------+\n4 rows in set (0.00 sec)\n\nmysql&gt; desc project_assignments;\n+-----------------+------+------+-----+---------+-------+\n| Field           | Type | Null | Key | Default | Extra |\n+-----------------+------+------+-----+---------+-------+\n| employee_id     | int  | NO   | PRI | NULL    |       |\n| project_id      | int  | NO   | PRI | NULL    |       |\n| assignment_date | date | NO   |     | NULL    |       |\n+-----------------+------+------+-----+---------+-------+\n3 rows in set (0.00 sec)<\/pre>\n<h2><span style=\"font-weight: 400;\">Limitations of pg2mysql<\/span><\/h2>\n<p><span style=\"font-weight: 400;\">pg2mysql is a useful tool for basic migrations, however it has a few limitations:<\/span><\/p>\n<ul>\n<li><span style=\"font-weight: 400;\">Conversion Scope: Supports just tables. Triggers, views, and stored procedures are not transformed, potentially leading to incomplete migrations.<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Data Type Handling: Some data types are not correctly transformed and may necessitate manual modifications. For example, a character with no defined length changes to LONGTEXT, impacting key definitions.<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Performance: huge datasets can be slow to process, and the tool may struggle with huge files or complicated schema dependencies.<\/span><\/li>\n<li><span style=\"font-weight: 400;\">Manual Adjustments: Significant manual intervention is required to fix SQL syntax discrepancies and data type concerns. It is significantly dependent on pre-processing and post-processing processes.<\/span><\/li>\n<li><span style=\"font-weight: 400;\">This blog is to enlighten on various available tools or options to perform the migration of PostgreSQL to MySQL. The pg2mysql is not a fully matured product and it needs more changes to make the migration seamless.<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400;\">Credit:Credit goes to:James Grant Lightbox Technolgoies <a href=\"http:\/\/www.lightbox.org\/\">http:\/\/www.lightbox.org\/<\/a>\u00a0, <a href=\"https:\/\/github.com\/ChrisLundquist\/pg2mysql\">https:\/\/github.com\/ChrisLundquist\/pg2mysql<\/a><\/span><\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Introduction In today&#8217;s dynamic corporate climate, data migration between database systems is a deliberate move that coincides with changing organizational demands. This block describes the migration of PostgreSQL to MySQL. This change necessitates meticulous preparation to handle possible issues and guarantee smooth interaction with existing business procedures. &nbsp; Basic Differences PostgreSQL Advanced Feature Set: PostgreSQL&hellip;<\/p>\n","protected":false},"author":1,"featured_media":2762,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[24],"tags":[95,103,111,113,178,197,202,250,266,285,319],"class_list":["post-2761","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-mysql","tag-cloud-database","tag-data-transfer","tag-database-conversion","tag-database-migration","tag-it-solutions","tag-migration-tools","tag-mysql","tag-pg2mysql","tag-postgresql","tag-postgresql-to-mysql","tag-sql-databases","category-24","description-off"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/divaind.com\/ie1\/wp-json\/wp\/v2\/posts\/2761","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/divaind.com\/ie1\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/divaind.com\/ie1\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/divaind.com\/ie1\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/divaind.com\/ie1\/wp-json\/wp\/v2\/comments?post=2761"}],"version-history":[{"count":0,"href":"https:\/\/divaind.com\/ie1\/wp-json\/wp\/v2\/posts\/2761\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/divaind.com\/ie1\/wp-json\/wp\/v2\/media\/2762"}],"wp:attachment":[{"href":"https:\/\/divaind.com\/ie1\/wp-json\/wp\/v2\/media?parent=2761"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/divaind.com\/ie1\/wp-json\/wp\/v2\/categories?post=2761"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/divaind.com\/ie1\/wp-json\/wp\/v2\/tags?post=2761"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}