Importing .rdb files into Redis with Protocol Streams

March 01, 2016

As a pref­ace, read this for the moti­va­tion and con­cept behind Redis pro­to­col streams (as usual, Redis doc­u­men­ta­tion is amaz­ing­ly well writ­ten and a joy to read).

redis-rdb-tools is a util­i­ty that cre­ates a pro­to­col stream from a .rdb file. How­ev­er, the orig­i­nal repo has a Uni­code decod­ing bug that pre­vents it from work­ing prop­er­ly. Thank­ful­ly, some­one forked it and patched it, and I can con­firm that the patch works for me. To install (make sure you’re on Python 2.x, not 3.x):

$ pip install git+https://github.com/lesandr/redis-rdb-tools@1f7bcf366073adf5510ad18f1efe0bf46ae5e0c1

(I’m installing the spe­cif­ic patch commit because it’s a fork and who knows what’ll happen to it in the future.)

Then, to import the file, just a simple one-liner:

$ rdb --command protocol /path/to/dump.rdb | redis-cli --pipe

If suc­cess­ful, you’ll see some­thing like:

All data transferred. Waiting for the last reply...
Last reply received from server.
errors: 0, replies: 4341259

as well as this in your Redis server logs:

95086:M 01 Mar 21:53:42.071 * 10000 changes in 60 seconds. Saving...
95086:M 01 Mar 21:53:42.072 * Background saving started by pid 98223
98223:C 01 Mar 21:53:44.277 * DB saved on disk

Notes:

  1. Make sure you already have a Redis server run­ning. The --pipe flag is only avail­able for Redis 2.6 onwards.
  2. If you want to inspect the pro­to­col stream visu­al­ly before import­ing, you can leave out the piping to redis-cli and it will pipe the stream to STDOUT (or you could pipe it to a text file).

The alter­na­tive way to import is to copy the .rdb file to the loca­tion spec­i­fied in your redis.conf (or modify redis.conf to point to your .rdb file). How­ev­er, I think using pro­to­col streams is a cooler solu­tion. 😎