Redis 4.0 GA is out!

27,184 views
Skip to first unread message

Salvatore Sanfilippo

unread,
Jul 14, 2017, 9:42:15 AM7/14/17
to redi...@googlegroups.com
Dear Redis users,

finally after the canonical 1 year of developments, Redis 4.0.0 is out.
I'm very happy about 4.0, because this release finally overcomes
several very important limits of Redis. There are many things in the
4.2 plan, so Redis with 4.0 is not magically perfect. Yet it is a very
important, measurable, step forward, because of the following
features:

1) PSYNC2, the new replication engine. The way the replication
handshake and changes propagation happens between masters and slaves
was significantly changed. Now slaves promoted to masters are able to
accept the old slaves (reconfigured to point to the new master)
without the need to do a full resynchronization. Similarly slaves can
be stopped, upgraded and restarted, and can continue with the master
with just a partial resynchronization. People that run Redis
operations will appreciate this change...

2) Redis modules: finally it is possible to use Redis as a framework
to write networked services without reinventing everything from
scratch. Modules can extend Redis with new functionalities and data
structures, implement types which are exactly like the native types
(persisted in the RDB, rewritten in the AOF, ...), and there is
experimental support for blocking and threaded slow operations.

3) Redis is a better cache with 4.0, because it supports LFU other
than LRU as eviction algorithm, and in general the eviction algorithm
was improved in different ways.

4) DEL, FLUSHDB and FLUSHALL can now be ran in a different thread
calling UNLINK (instead of DEL), and using the ASYNC option of FLUSH
commands. This, together with the new SWAPDB command, open interesting
possibilities to atomically switch a given dataset to another one,
makes certain workloads using only DEL as O(N) commands a lot less
latency sensitive, and can improve also replication setup times,
because the slave, if configured to do so, can delete the old dataset
in background after a full resync.

5) It is now possible to enable a new AOF mode that rewrites the AOF
in terms of an RDB preamble + an AOF tail: faster rewrites, faster
reloads of the data, less latency issues.

6) The Raspberry Pi is now one a supported platform: every new Redis
release is tested on the Pi, the code was audited and fixed for
unaligned accesses, stack traces are supported on ARM.

7) The new MEMORY command, and especially MEMORY DOCTOR, allows to
have a better observability on the Redis memory usage. MEMORY USAGE
can precisely report a key memory utilization.

8) Redis Cluster was improved (but 4.2 will focus mostly on Cluster...
so the big news are headed): NAT/Containers support, better failure
detection (much lower traffic in big clusters), less CPU overhead in
cluster nodes compared to Redis 3.2.

9) Active memory defragmentation. Redis is able to defragment the
memory while online if the Jemalloc allocator is used (the default on
Linux). Useful for workloads where the allocator cannot keep the
fragmentation low enough, so the only possibility is for Redis and the
allocator to collaborate in order to defragment the memory.

A big thank you to the Redis community, providing help in issues, help
channels, fixing documentation, finding bugs, sending PRs. This Redis
release is surely the one that benefitted the most from external help.
A special thank you to my colleagues at Redis Labs that supported me
in all the possible ways, helped me in the hardest moments (see
Heisenbugs...), and that were extremely active in using, testing,
fixing and designing with me the modules API.

Please report 4.0.0 problems ASAP if you find any, I expect a few
patch releases coming before we actually reach the same level of
stability of Redis 3.2.

Enjoy!
Salvatore

--
Salvatore 'antirez' Sanfilippo
open source developer - Redis Labs https://redislabs.com

"If a system is to have conceptual integrity, someone must control the
concepts."
— Fred Brooks, "The Mythical Man-Month", 1975.

Yiftach Shoolman

unread,
Jul 14, 2017, 9:57:28 AM7/14/17
to redi...@googlegroups.com
Indeed, big effort, great work, well done Salvatore! 

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.



--

Yiftach Shoolman
+972-54-7634621

iandyh

unread,
Jul 14, 2017, 10:29:28 AM7/14/17
to Redis DB
Thanks for the great work.

Really looking forward to testing it. 

Andy

Alexander Berezovsky

unread,
Jul 14, 2017, 10:42:52 AM7/14/17
to Redis DB
Amazing Salvatore! It's been great to see how features announced on the RedicConf becoming our new reality. 

Itamar Haber

unread,
Jul 14, 2017, 12:15:49 PM7/14/17
to Redis DB
An historical version by all accounts.

Personal context: I've been waiting for v4 since Salvatore's '16 visit to the holy land (circa https://redislabs.com/videos/redisday-tlv-2016/), after the brainstorming sessions where modules became a reality, so 18 months or so. Roughly, IIRC, the length of an Oliphant's gestation period :)

Mazal tov, we have a beautiful new baby, now it is our turn to love it and help it grow up to its fullest potential. And beyond.

Cheers,

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.



--

Itamar Haber | Chief OSS Education Officer
Redis Labs ~/redis

Mobile: +972 (54) 567 9692
Twitter: @itamarhaber
Skype: itamar.haber

Jonathan L.

unread,
Jul 14, 2017, 2:24:11 PM7/14/17
to Redis DB
This is amazing Salvatore! Thanks!

Antônio Guilherme Ferreira Viggiano

unread,
Jul 21, 2017, 12:47:13 PM7/21/17
to Redis DB
Very nice!

I hope to see more useful redis modules coming out after 4.0.0


On Friday, July 14, 2017 at 10:42:15 AM UTC-3, Salvatore Sanfilippo wrote:

Stefano Fratini

unread,
Jul 21, 2017, 11:48:00 PM7/21/17
to Redis DB
The modules available now have mostly been developed by Redis Labs and are already very useful (try the full text search one for example)

They natively don't work with Redis Cluster though OR they may work but the data may not shard, for example

Jay Rolette

unread,
Jul 23, 2017, 8:36:29 AM7/23/17
to redis
On Fri, Jul 21, 2017 at 10:48 PM, Stefano Fratini <ma...@stefanofratini.it> wrote:
The modules available now have mostly been developed by Redis Labs and are already very useful (try the full text search one for example)

They natively don't work with Redis Cluster though OR they may work but the data may not shard, for example

Make sure you check the license on those before you use them. Unfortunately, most of them do not follow the same license as Redis itself. Most of them are AGPL v3, which is a problem for many.

Jay 

Salvatore Sanfilippo

unread,
Jul 23, 2017, 8:59:12 AM7/23/17
to redi...@googlegroups.com
A clarification on the licensing part. As you know Redis is BSD
licensed. However, like with clients, there are no "official modules",
modules totally follow the Bazar model.
In essence the Redis project will just enlist at Redis.io *every*
module that is not evil or alike, and has a Github page, as long as
the license is any OSI-approved license.
For this reason Redis Labs, like any other player in the Redis
landscape, choses the license according to their needs. If one day
there will be modules directly shipped with Redis, they'll be released
under the BSD license like Redis, as you can see from the example
modules shipped inside Redis right now (which are just programming
examples btw).
> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to redis-db+u...@googlegroups.com.
> To post to this group, send email to redi...@googlegroups.com.
> Visit this group at https://groups.google.com/group/redis-db.
> For more options, visit https://groups.google.com/d/optout.



Antônio Guilherme Ferreira Viggiano

unread,
Jul 23, 2017, 10:06:06 AM7/23/17
to Redis DB
Sorry, I didn't mean current modules aren't useful, I mean there aren't a lot of them yet.

Having written one myself, I guess it's because the documentation isn't that extensive, and because not every programmer is familiar with C. 

For the first part I would like to see more blog posts or walkthroughs explaining step-by-step how to create a redis module (such as this one made by redis labs), and for the second one perhaps more examples with bindings to other languages, such as this one in rust.

Besmir Zanaj

unread,
Jul 24, 2017, 2:27:11 AM7/24/17
to Redis DB
Great job indeed!!

Jay Rolette

unread,
Jul 24, 2017, 8:12:10 AM7/24/17
to redis
Yes, no disagreement that RedisLabs (or anyone else publishing a Redis module) has the right to choose their own license according to their needs, philosophy or whims. I brought it up for a couple of reasons:

1) It's different than Redis and it would be easy for folks to mistakenly assume that a module running as part of the Redis process probably has similar licensing. Redis isn't treading new ground on that front, but it is a little surprising that the modules RedisLabs is publishing have such a dramatically different license.

2) I personally had some concerns about licensing of future features. Now that Redis has modules, it seems likely that any feature that can be developed as a module will be (and probably should be) developed as a module. 

With all of the modules coming from the company you work at (or at least are sponsored by) being published under AGPL, that was looking like a bad sign where otherwise very useful extensions to Redis are walled off for many.

Best regards and thanks for all that you've done with Redis over the years,
Jay


> To post to this group, send email to redi...@googlegroups.com.
> Visit this group at https://groups.google.com/group/redis-db.
> For more options, visit https://groups.google.com/d/optout.



--
Salvatore 'antirez' Sanfilippo
open source developer - Redis Labs https://redislabs.com

"If a system is to have conceptual integrity, someone must control the
concepts."
       — Fred Brooks, "The Mythical Man-Month", 1975.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.

Itamar Haber

unread,
Jul 24, 2017, 8:15:37 AM7/24/17
to Redis DB
@Jay - we've (Redis Labs) thought long and hard about the selection of license, and you can read what our CTO wrote about it at https://redislabs.com/blog/why-redis-labs-modules-are-agpl/.

That said, I'm not sure why "AGPL was looking like a bad sign" to you - can you expand?

Jay Rolette

unread,
Jul 24, 2017, 9:17:38 AM7/24/17
to redis
Hi Itamar,

Thanks for the link. The explanation is helpful and very reasonable. The last paragraph in the link is key, IMO, to not locking many out of being able to use your modules. 

At every company I've ever been at, there is no hesitation in pushing fixes and enhancements to open source modules upstream. Even ignoring any legal or altruistic reasons, it makes business sense to avoid having to re-merge changes constantly. 

The problem for companies is the risk of having viral licenses being applied to _their_ code. It's very much a legal gray area around what constitutes a derivative. The FSF has a very different interpretation of it than many (most?). In the US, it hasn't been tested in the courts yet so there's a lot of risk there that has to be considered.

Linux mostly avoids this by being explicit that user space programs are not considered derivative works. gcc states explicitly that the programs it compiles are not derivative works, so also not an issue. LGPL licensed programs make it clear that you can link safely.

Is there anyway to get that statement (or something to the same effect) into the license in your github repo? It would eliminate the risks for users of your module. The LICENSE file there is the stock AGPL v3 text.

Cheers,
Jay

Dvir Volk

unread,
Jul 26, 2017, 6:39:55 AM7/26/17
to redi...@googlegroups.com
Hi Jay. Your idea is interesting and we're considering adding something.

Regardless, do you see people having similar concerns about using MongoDB? 
If not, why do you think that is? Their blog post clarification? Or just the fact that redis modules are libraries that gets people more concerned as opposed to a "monolith" database?

Cheers,
Dvir


> To post to this group, send email to redi...@googlegroups.com.
> Visit this group at https://groups.google.com/group/redis-db.
> For more options, visit https://groups.google.com/d/optout.



--
Salvatore 'antirez' Sanfilippo
open source developer - Redis Labs https://redislabs.com

"If a system is to have conceptual integrity, someone must control the
concepts."
       — Fred Brooks, "The Mythical Man-Month", 1975.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.

To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.

To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.



--

Itamar Haber | Chief OSS Education Officer
Redis Labs ~/redis

Mobile: +972 (54) 567 9692
Twitter: @itamarhaber
Skype: itamar.haber

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.

To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.

Jay Rolette

unread,
Jul 27, 2017, 8:32:43 AM7/27/17
to redis
Hi Dvir,

I appreciate Redis Labs considering adjusting the license to be explicit. IANAL, but it seems risky to assume the courts will accept that a blog post (which, btw, may not be there at some point in the future) to modify the license that is embedded in the code repository.

I haven't looked specifically at the MongoDB license, but if it is AGPL, it should be the same issue for companies. I know MongoDB gets around some of the AGPL issues by licensing the drivers with an Apache license. Not exactly sure how much that helps for services targeted specifically by AGPL though.

In the end, it comes down to how conservative the company and their legal team wants to be.

Jay


> To post to this group, send email to redi...@googlegroups.com.
> Visit this group at https://groups.google.com/group/redis-db.
> For more options, visit https://groups.google.com/d/optout.



--
Salvatore 'antirez' Sanfilippo
open source developer - Redis Labs https://redislabs.com

"If a system is to have conceptual integrity, someone must control the
concepts."
       — Fred Brooks, "The Mythical Man-Month", 1975.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.

To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.

To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.



--

Itamar Haber | Chief OSS Education Officer
Redis Labs ~/redis

Mobile: +972 (54) 567 9692
Twitter: @itamarhaber
Skype: itamar.haber

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.

To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.

To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.

Dvir Volk

unread,
Jul 27, 2017, 9:28:46 AM7/27/17
to redis
Thanks Jay,
Yes, Mongo is AGPL.

And BTW, we're also publishing all our client libraries for modules as BSD. 



> To post to this group, send email to redi...@googlegroups.com.
> Visit this group at https://groups.google.com/group/redis-db.
> For more options, visit https://groups.google.com/d/optout.



--
Salvatore 'antirez' Sanfilippo
open source developer - Redis Labs https://redislabs.com

"If a system is to have conceptual integrity, someone must control the
concepts."
       — Fred Brooks, "The Mythical Man-Month", 1975.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.

To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.

To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.



--

Itamar Haber | Chief OSS Education Officer
Redis Labs ~/redis

Mobile: +972 (54) 567 9692
Twitter: @itamarhaber
Skype: itamar.haber

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.

To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.

To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.

To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages