(go-redis-migrator) Migrate keys between Redis hosts or clusters using GO

Go Redis Migrator


I recently have been both working with GO and Redis a lot. When I came across the need to copy all the keys in a cluster to a single host (for use with development and testing), I set out to find a simple key migrator utility that supported clusters. Turns out, there aren’t any I could find! Clustering just came to Redis in version 3.0 and is still pretty new.

I used this opportunity to write some GO. GO had a package that supported Redis clustering and is fast enough to work with even large deployments… So I set to work. The result is go-redis-migrator. It’s pretty fast at 6,367 keys copied per second in my test!


First, get the dependencies: go get -v

Then, make a build: go build

Finally, run the binary to see the help: ./go-redis-migrator


./go-redis-migrator -copyData -sourceHosts=,,,, -destinationHosts=,,,,, -keyFile=keyList.txt 
Migrated 57 keys.

Fetch a filtered key list from a host:

./go-redis-migrator -getKeys -keyFilter=sessions:\* -sourceHosts=

Fetch a filtered key list from a cluster:

./go-redis-migrator -getKeys -keyFilter=sessions:\* -sourceHosts=,

From a cluster to a single host:

./go-redis-migrator -copyData -sourceHosts=, -destinationHosts=

Copying keys specified in a file from a host to a host:

./go-redis-migrator -copydata=true -sourceHosts= -destinationHosts= -keyFile=./onlyMoveTheseKeys.txt

Simply run the binary to get the following help:

- Redis Key Migrator - 

Migrates all or some of the keys from a Redis host or cluster to a specified host or cluster.  Supports migrating TTLs.
go-redis-migrator can also be used to list the keys in a cluster.  Run with the -getKey and -sourceHosts= flags to do so.

You must run this program with an operation before it will do anything.

  -getKeys=false: Fetches and prints keys from the source host.
  -copyData=false: Copies all keys in a list specified by -keyFile= to the destination cluster from the source cluster.
  -keyFile="": The file path which contains the list of keys to migrate.  One per line.
  -keyFilter="*": The filter for which keys to migrate.  Does not work when -keyFile is also specified.
  -destinationHosts="": A list of source cluster host:port servers seperated by spaces. Use a single host without a ',' if there is no cluster. EX),
  -sourceHosts="": A list of source cluster host:port servers seperated by commas. Use a single host without a ',' if there is no cluster. EX),

Contribution Wanted

  • It would be nice to have pre-compiled binaries for people to download for their OS and architecture.
  • Moving to SCAN to migrate keys would support super large deployments better.
  • Multi-threading the copy process for super large migrations.

In case you missed it. Get it on Github here!