I have never used Kafka or had to maintain queues with more than a hundred jobs per hour. Most of the jobs queues I've had to maintain are doing simple things like sending emails or updating a third party API.
I've always resorted to simpler distributed redis based job queues such as kue [0], bull [1], celery [2] or resque [3].
I would like to learn more about distributed queues and architectures from first principles. Where is a good place to go?
[0] https://github.com/Automattic/kue [1] https://github.com/OptimalBits/bull [2] http://www.celeryproject.org/ [3] https://github.com/resque/resque