Today I checked again, and to my surprise I got:
For the first time segment (273 days period) I averaged 5.98 IP bans a day. From that day to today I got: 5853-1633 = 4220 new bans. The time period from November 2014 to today is 573 days.
That means in the last year the banning grew to 7.36 bans a day (~20% increase). And we need to remember that this is also with the previous backlist.
So today, beside applauding fail2ban's relentless work, I changed the port of the SSH server.
To my surprise from this morning, until now, there is radio silence from the fail2ban new bans. I guess most scanners don't do a port scanning first, and they just try to find default or weirdly configured SSH servers.
So here's my second tip. Change your SSH server port.