58
How to monitor your Redis-based job queue system?

How to monitor your Redis-based job queue system?

4 years ago
Anonymous $9jpehmcKty

https://medium.com/@alexthered/how-to-monitor-your-redis-based-job-queue-system-c518b34a7e7a

Hiep DoanBlockedUnblockFollowFollowingMay 4Photo by Michał Parzuchowski on UnsplashDistributed job queue is a very popular pattern to process heavy batch work, which can require much 'bigger' (read: expensive) hardware resource to complete.

Let's illustrate with an example. Your system needs to send a newsletter to all of your users (e.g 100k users) on weekly basis. Since it is more important that all users receive an email, rather than they need to receive it at the same time, or exactly at a moment, it's probably useful to split the task into, for example, 100 tasks of sending email (each for 1000 users).