Comparing data from two big MySQL tables

Email this to someoneShare on Google+Share on FacebookTweet about this on TwitterShare on LinkedIn

Problem
You have two big tables in MySQL (>640K records), that maybe differ in the number of fields, but you want to make sure that the data in the common fields in both tables are the same.

Solution

  1. Use mysql to export the data from the first table in a csv file, selecting only the common fields.
    We use the /tmp folder on the server to make sure we have the right permissions to create the file:
  2. If the tables are in different databases remember to switch db:

    Export the second table in the second file:
  3. now use the diff. You can use any of the following options:
    • diff -q first_table.txt second_table.txt
    • diff first_table.txt second_table.txt > diff.txt