The Architecture of a Seamless Magento 1 to Magento 2 Database Migration

Understanding the Core Differences: Magento 1 vs. Magento 2 Database Schema

Migrating a Magento 1 database to Magento 2 is not a simple ETL process. The underlying database schemas are fundamentally different, reflecting Magento 2’s architectural overhaul. Key areas of divergence include EAV (Entity-Attribute-Value) model implementation, table structures for core entities like products, customers, and orders, and the introduction of new concepts like “shared catalogs” and “company accounts” in Magento 2 Commerce. A direct, unassisted data dump and import will fail catastrophically. We must analyze the schema differences and plan a transformation strategy.

Magento 1’s EAV implementation, while flexible, led to performance bottlenecks due to complex joins and large tables (e.g., catalog_product_entity_varchar, catalog_product_entity_int). Magento 2 refactors this, often denormalizing data into dedicated columns for frequently accessed attributes and employing a more structured approach for less common ones. Understanding the mapping between M1 EAV attributes and M2 columns is paramount. This involves analyzing the eav_attribute tables in M1 and comparing them to the corresponding M2 entity tables and their attribute definitions.

Pre-Migration Data Cleansing and Preparation

Before any migration tools are even considered, a rigorous data cleansing process on the Magento 1 database is essential. This significantly reduces the complexity and error rate of the migration. Common issues include:

• Duplicate Customer Accounts: Often caused by inconsistent email addresses or variations in name/address.
• Orphaned Data: Products without categories, orders without customers, etc.
• Corrupted Data: Invalid characters, incorrect data types, or malformed entries in custom attributes.
• Unused Attributes/Entities: EAV attributes that are no longer used by any products or are obsolete.
• Large/Bloated Tables: Particularly log_... tables, which can be safely truncated or archived if historical logging isn’t critical for the migration.

A good starting point for identifying potential issues is to run SQL queries against the M1 database. For instance, checking for duplicate customer emails:

SELECT email, COUNT(*) FROM customer_entity GROUP BY email HAVING COUNT(*) > 1;

Or identifying products without any assigned categories:

SELECT DISTINCT
    e.entity_id
FROM
    catalog_product_entity e
LEFT JOIN
    catalog_category_product ccp ON e.entity_id = ccp.product_id
WHERE
    ccp.product_id IS NULL;

These queries should be adapted and expanded to cover various data integrity checks. Any identified issues must be addressed either by manual correction, custom scripts, or by leveraging data quality tools before proceeding.

Leveraging the Magento Migration Toolkit (MMT)

Magento provides the official Magento Migration Toolkit (MMT) as a primary tool for this process. While it automates much of the data transfer, it’s crucial to understand its architecture and how to configure and extend it. MMT consists of several components:

• Data Migration Console: A command-line tool that orchestrates the migration process.
• Configuration Files: XML files that define the mapping between M1 and M2 database structures, attribute sets, data transformations, and migration steps.
• Scripts: PHP scripts that handle specific data transformations and custom logic.

The core of MMT’s configuration lies in its config.xml and map.xml files. The config.xml specifies database connection details for both M1 and M2, as well as settings for the migration itself. The map.xml is where the schema mapping and attribute transformations are defined.

A typical config.xml snippet for MMT:

<?xml version="1.0"?>
<config>
    <source>
        <database>
            <host>m1.db.example.com</host>
            <username>m1_user</username>
            <password>m1_password</password>
            <name>magento1_db</name>
            <port>3306</port>
        </database>
    </source>
    <destination>
        <database>
            <host>m2.db.example.com</host>
            <username>m2_user</username>
            <password>m2_password</password>
            <name>magento2_db</name>
            <port>3306</port>
        </database>
    </destination>
    <migration>
        <settings>
            <directory>./migration_data</directory>
            <log_file>./migration.log</log_file>
            <max_parallel_processes>4</max_parallel_processes>
        </settings>
        <steps_mode>destructive</steps_mode>
    </migration>
</config>

The map.xml file is far more extensive. It defines how tables and columns from M1 are mapped to M2. For EAV attributes, this involves mapping M1 attribute codes to M2 attribute codes and specifying the target entity type. For example:

<!-- Example mapping for a custom product attribute -->
<eav>
    <attribute>
        <source_table>catalog_product_entity_varchar</source_table>
        <source_attribute_code>my_custom_attribute</source_attribute_code>
        <target_attribute_code>my_custom_attribute_m2</target_attribute_code>
        <entity_type>catalog_product</entity_type>
        <attribute_type>varchar</attribute_type>
        <backend_type>varchar</backend_type>
    </attribute>
    <!-- ... other attribute mappings ... -->
</eav>

<!-- Example mapping for a core table -->
<table>
    <source_table>catalog_product_entity</source_table>
    <target_table>catalog_product_entity</target_table>
    <fields>
        <entity_id>entity_id</entity_id>
        <sku>sku</sku>
        <created_at>created_at</created_at>
        <updated_at>updated_at</updated_at>
        <type_id>type_id</type_id>
        <attribute_set_id>attribute_set_id</attribute_set_id>
    </fields>
</table>
<!-- ... other table mappings ... -->

It’s crucial to generate a comprehensive map.xml. This often involves scripting to parse M1’s EAV attribute definitions and M2’s attribute metadata to create the mappings automatically. The MMT itself provides scripts to help generate initial mapping files based on your M1 installation.

Customizing MMT for Complex Migrations

While MMT handles standard data, real-world Magento 1 installations invariably have custom modules, custom attributes, and unique data structures. MMT allows for customization through:

• Custom Scripts: PHP scripts can be written to handle complex data transformations that cannot be expressed in XML mapping. These scripts are referenced within the MMT configuration.
• Data Transformation Rules: MMT supports defining rules for transforming data values during migration (e.g., converting date formats, mapping specific values).
• Delta Migrations: For minimizing downtime, MMT supports delta migrations to transfer only the data that has changed since the initial full migration.

Consider a scenario where a custom module in M1 stores product-specific configuration in a separate table, my_module_product_config, which needs to be migrated to a new structure in M2, perhaps as a custom attribute or a dedicated table. You would typically:

• Define the target structure in M2 (e.g., create a new attribute or table).
• Write a custom PHP script that reads from my_module_product_config in M1, performs the necessary transformations, and inserts data into the M2 structure.
• Configure MMT to execute this script at the appropriate stage of the migration.

A simplified example of referencing a custom script in MMT’s configuration (often within a specific migration step definition):

<!-- Within a migration step definition -->
<step code="custom_product_config_migration">
    <description>Migrate custom product configuration</description>
    <handler>
        <type>script</type>
        <script>scripts/custom_product_config.php</script>
        <params>
            <param name="source_table" value="my_module_product_config"/>
            <param name="target_attribute_set" value="default"/>
        </params>
    </handler>
</step>

The custom PHP script (e.g., scripts/custom_product_config.php) would then contain the logic to connect to both databases, fetch data, transform it, and save it to the M2 database. This script would typically extend MMT’s base migration classes to leverage its infrastructure.

Execution Strategy and Downtime Minimization

A successful migration requires a well-defined execution strategy. This typically involves several phases:

• Initial Full Migration: Migrate all historical data. This can take a significant amount of time depending on the data volume. Perform this on a staging environment.
• Data Verification: Thoroughly validate the migrated data in M2. This includes checking product counts, customer counts, order totals, attribute values, and running sample reports.
• Delta Migrations: Once the full migration is complete and verified, schedule a period of read-only mode on the M1 store. Then, run delta migrations to capture all changes (new orders, updated products, new customers) that occurred since the full migration.
• Final Cutover: After the delta migration, perform the final cutover. This involves:
  • Putting the M1 store into maintenance mode (or completely shutting it down).
  • Running a final, very quick delta migration for any last-minute changes.
  • Updating DNS records to point to the new M2 servers.
  • Performing final smoke tests on the live M2 site.

The goal is to minimize the time the store is unavailable. This is achieved by performing the bulk of the migration offline and using delta migrations for the final synchronization. The exact downtime window will depend on the data volume and the efficiency of your migration scripts and infrastructure.

For large datasets, consider optimizing the migration process:

• Parallel Processing: MMT supports parallel migration of different data entities. Configure max_parallel_processes in config.xml appropriately for your server resources.
• Database Tuning: Ensure both M1 and M2 database servers are well-tuned for read/write performance.
• Staging Environment: Always perform migrations on a staging environment that closely mirrors production. This allows for testing and refinement without impacting live operations.
• Incremental Backups: Implement a robust backup strategy for both M1 and M2 databases throughout the migration process.

Post-Migration Validation and Optimization

The migration is not complete until the Magento 2 store is live and functioning correctly. Post-migration validation is critical:

• Functional Testing: Test all critical user flows: product browsing, adding to cart, checkout, customer account management, order history.
• Data Integrity Checks: Run reports and queries to verify data consistency. Compare key metrics (e.g., total orders, revenue, customer count) between M1 and M2.
• Performance Testing: Magento 2 is generally more performant than M1, but poorly migrated data or incorrect configurations can negate this. Conduct load testing to identify bottlenecks.
• Indexing: Ensure all Magento 2 indexes (catalog, search, etc.) are re-indexed after the migration.
• Third-Party Integrations: Verify that all third-party modules and integrations are compatible with M2 and are functioning correctly with the migrated data.

Common post-migration issues include missing product images (if not migrated correctly), incorrect pricing, broken customer data, and search functionality problems. Addressing these requires a deep understanding of both Magento 1 and Magento 2 architectures and the specific data transformations performed during the migration.