beanstalk队列中的作业名称重复

Question

我们的系统处理队列中的许多作业，有时这些作业尚未完成处理。我们的系统有可能将与当前处理的作业名称相同的作业放在一起。

是否有一个检查器会告诉我们在将它们添加到队列中之前，具有相同名称的作业已经在队列中了？

谢谢你们！

Answer 1

Beanstalkd does not have the facility to look things up within it - it's a job-queue, not a giant array. Other things can be used in conjunction with it though, that do allow for random access to data to record if something has already been done.

If you know all the jobs have a specific identifier, you could put them into Redis or Memcached, probably with some form of prefix, and possibly an expiry beyond which they won't be stored.

Redis also allows for other data structures that could help as well, such as Bloom Filters and the Redis-native Hyperloglog.