Question

I'm used to use webjob on Azure triggering an Azure queue. It works like a charm.

static void Main(string[] args)
{
    JobHost host = new JobHost();
    host.RunAndBlock();
}

public static void ProcessQueueMessage([QueueTrigger("logqueue")] string logMessage, TextWriter logger)
{
    logger.WriteLine(logMessage);
}

What's really good with queueTrigger is until the process triggered by a message isnot done, the message is keep invisible (not delete). So If you turn off the webjob (for webjob update for example) The message will be visible (after a little timeout) in the queue to be process by the updated webjob (perfect).

Now I wanna do the same thing but on a worker role. Today I do like this.

while (true)
{
     var cloudMessage = await sourceImportationQueue.GetMessageAsync();
     if (cloudMessage != null)
           sourceImportationQueue.DeleteMessage(cloudMessage);
      // process my job (few hours)
     else
           await Task.Delay(1000 * 5);
}

But if I stop the worker during is job, I lost the message. So how can I do like webJob triggering ?

Answer 1

最后我找到了一个简单的解决方案。在运行几个小时的工作之前，我启动一个任务KeepHiddenMessageAsync，用超时更新消息。在超时结束之前，将完成消息的新更新。如果出现问题，则将达到消息超时并且消息将变为可见。

        private bool jobIsComplete;

        private void Run()
        {
             while (true)
            {
                 jobIsComplete = false;
                 //get the message
                 var cloudMessage = await queue.GetMessageAsync();

                 if (cloudMessage != null)
                        //run the task to keep the message until end of the job and worker role stopping for an update for example 
                       var keepHiddenMessageTask = KeepHiddenMessageAsync(cloudMessage);

                        //
                        // process my job (few hours)
                        //

                      jobIsComplete = true;
                      await keepHiddenMessageTask;
                      await _queue.DeleteMessageAsync(cloudMessage);
                 else
                       await Task.Delay(1000 * 5);
            }
        }

        private async Task KeepHiddenMessageAsync(CloudQueueMessage iCloudQueueMessage)
        {
            while (true)
            {
                //Update message and hidding during 5 new minutes
                await _queue.UpdateMessageAsync(iCloudQueueMessage, TimeSpan.FromMinutes(5), MessageUpdateFields.Visibility);

                //Wait 4 minutes
                for (int i = 0; i < 60 * 4; i++)
                {
                    if (JobIsComplete)
                        return;
                    else
                        await Task.Delay(1000);
                }
            }
        }

Answer 2

By default once a queue message has been retrieved it becomes invisible for 5 minutes. After this delay, if the message has not been deleted from the queue it will become visible again so that it can be processed once again.

In your code sample you are deleting the message as soon as you get it from the queue. If you want to make it safe the message should only be deleted at the end of your process. Have you tried to move sourceImportationQueue.DeleteMessage(cloudMessage); at the end of the processing job?

Answer 3

如果不使用某种持久性存储来跟踪您的工作进度，则可能无法解决此问题。正如已经确定的那样，您将在作业开始之前删除该消息，因此如果作业因任何原因失败（包括停止该角色），则消息将丢失。消息的最长锁定时间为5分钟，这意味着当作业仍在运行时，消息将再次出现，如果删除操作已移至最后，则由于丢失锁定而失败。

如果长时间运行的作业由多个较小的步骤组成，这些步骤都不超过5分钟，那么您可以定期调用RenewLock（）来保留对消息的锁定并阻止它重新出现在队列中。只要锁永不过期，在这种情况下，最终的DeleteMessage就会成功。但这可能不太适合您的情况。

一种可能的解决方案是将作业状态写入例如Azure表，并在整个作业处理过程中记录状态。您的工作者角色循环将检查表中是否有任何尚未完成的作业，并继续存在任何已存在的作业，如果没有找到，则检查服务总线是否有任何新作业。这个解决方案也可以让你有机会从他们到达的地点获取失败的工作，而不是从头开始你的2小时工作。

Azure WorkerRole trigger on queue like WebJob

3 个答案: