[Solved]-Redis is slow to get large strings


Redis is not designed to store very large objects. You are not supposed to store your entire collection in a single string in Redis, but rather use Redis list or set as a container for your objects.

Furthermore, the pickle format is not optimized for space … you would need a more compact format. Protocol Buffers, MessagePack, or even plain JSON, are probably better for this. You should consider to apply a light compression algorithm before storing your data (like Snappy, LZO, Quicklz, LZF, etc …).

Finally, the performance is probably network bound. On my machine, retrieving a 20 MB object from Redis takes 85 ms (not 3 seconds). Now, if I run the same test using a remote server, it takes 1.781 seconds, which is expected on this 100 Mbit/s network. The duration is fully dependent on the network bandwidth.

Last point: be sure to use a recent Redis version – a number of optimization have been done to deal with large objects.


It’s most likely just the size of the string. I’d look at whether your objects are being serialized efficiently.

Leave a comment