使用Node.js从Azure WebJob轮询Azure服务总线队列

使用Node.js从Azure WebJob轮询Azure服务总线队列,azure,azure-web-app-service,azureservicebus,azure-webjobs,Azure,Azure Web App Service,Azureservicebus,Azure Webjobs,正在尝试使用Node.js中编写的WebJob轮询Azure Service Bus队列。我创造了两个网络职位。第一个是随需应变的,向队列发送10条唯一的消息。第二个作业是连续作业,轮询队列中的消息 遇到以下问题: 投票很慢。收到10条信息平均需要10分钟左右。请参阅下面的示例日志详细信息。在这种速度下基本上无法使用。所有延迟都是从receiveQueueMessage获取响应造成的。响应时间从0秒到120秒不等,平均为60秒 这些信息是按随机顺序接收的。不是先进先出 有时消息会被接收两次,即使

正在尝试使用Node.js中编写的WebJob轮询Azure Service Bus队列。我创造了两个网络职位。第一个是随需应变的,向队列发送10条唯一的消息。第二个作业是连续作业,轮询队列中的消息

遇到以下问题:

  • 投票很慢。收到10条信息平均需要10分钟左右。请参阅下面的示例日志详细信息。在这种速度下基本上无法使用。所有延迟都是从
    receiveQueueMessage
    获取响应造成的。响应时间从0秒到120秒不等,平均为60秒

  • 这些信息是按随机顺序接收的。不是先进先出

  • 有时消息会被接收两次,即使它们是在ReceiveAndDelete模式下读取的(我尝试过没有默认为ReceiveAndDelete的read mode参数,使用
    {isReceiveAndDelete:true}
    {isPeekLock:false}
    ,结果相同)

  • 当队列为空时,它应该将接收请求保持打开状态一天,但在230秒后它总是返回无消息错误。根据文件,最长为24天,因此我不知道230秒是从哪里来的:

  • 服务总线中阻塞接收操作的最大超时 排队时间为24天。但是,基于REST的超时具有最大值 55秒

    基本上没有广告宣传的效果。我做错了什么

    发送消息测试作业:

    var uuid = require('node-uuid');
    var azure = require('azure');
    var serviceBus = azure.createServiceBusService(process.env.busSearchConnectionString);
    var messagesToSend = 10;
    
    sendMessage(0);
    
    function sendMessage(count)
    {
        var message = {
            body: 'test message',
            customProperties: {
                message_number: count,
                sent_date: new Date
            },
            brokerProperties: {
                MessageId: uuid.v4() //ensure that service bus doesn't think this is a duplicate message
            }
        };
    
        serviceBus.sendQueueMessage(process.env.busSearchQueueName, message, function(err) {
    
            if (!err) {
                console.log('sent test message number ' + count.toString());
            } else {
                console.error('error sending message: ' + err);
            }
    
        });
    
        //wait 5 seconds to ensure messages are received by service bus in correct order
        if (count < messagesToSend) {
            setTimeout(function(newCount) {
                //send next message
                sendMessage(newCount);
            }, 5000, count+1);
        }
    }    
    
    日志输出示例:

    [05/06/2015 21:50:14 > 8c2504: SYS INFO] Status changed to Running
    [05/06/2015 21:50:14 > 8c2504: INFO] listener job started
    [05/06/2015 21:51:23 > 8c2504: INFO] received a response in 1s seconds
    [05/06/2015 21:51:23 > 8c2504: INFO] received test message number 0
    [05/06/2015 21:51:25 > 8c2504: INFO] received a response in 2s seconds
    [05/06/2015 21:51:26 > 8c2504: INFO] received test message number 4
    [05/06/2015 21:51:27 > 8c2504: INFO] received a response in 1s seconds
    [05/06/2015 21:51:27 > 8c2504: INFO] received test message number 7
    [05/06/2015 21:51:28 > 8c2504: INFO] received a response in 0s seconds
    [05/06/2015 21:51:29 > 8c2504: INFO] received test message number 9
    [05/06/2015 21:51:49 > 8c2504: INFO] received a response in 20s seconds
    [05/06/2015 21:51:49 > 8c2504: INFO] received test message number 1
    [05/06/2015 21:53:35 > 8c2504: INFO] received a response in 106s seconds
    [05/06/2015 21:53:35 > 8c2504: INFO] received test message number 1
    [05/06/2015 21:54:26 > 8c2504: INFO] received a response in 50s seconds
    [05/06/2015 21:54:26 > 8c2504: INFO] received test message number 5
    [05/06/2015 21:54:35 > 8c2504: INFO] received a response in 9s seconds
    [05/06/2015 21:54:35 > 8c2504: INFO] received test message number 9
    [05/06/2015 21:55:28 > 8c2504: INFO] received a response in 53s seconds
    [05/06/2015 21:55:28 > 8c2504: INFO] received test message number 2
    [05/06/2015 21:57:26 > 8c2504: INFO] received a response in 118s seconds
    [05/06/2015 21:57:26 > 8c2504: INFO] received test message number 6
    [05/06/2015 21:58:28 > 8c2504: INFO] received a response in 61s seconds
    [05/06/2015 21:58:28 > 8c2504: INFO] received test message number 8
    [05/06/2015 22:00:35 > 8c2504: INFO] received a response in 126s seconds
    [05/06/2015 22:00:35 > 8c2504: INFO] received test message number 3
    [05/06/2015 22:04:25 > 8c2504: INFO] received a response in 230s seconds
    [05/06/2015 22:04:25 > 8c2504: INFO] error requesting message: No messages to receive
    [05/06/2015 22:08:16 > 8c2504: INFO] received a response in 230s seconds    
    [05/06/2015 22:04:25 > 8c2504: INFO] error requesting message: No messages to receive
    

    问题是我使用的队列已分区(在Azure门户中创建队列时的默认选项)。一旦我创建了一个没有分区的新队列,一切都按预期工作,没有延迟(除了长时间轮询时奇怪的230秒超时)。因此,node.js库基本上不适用于分区队列。完全浪费了很多天才弄明白。将把这个留给其他人。

    关闭服务总线队列的分区标志对我来说也很有效

    对于分区队列,一些消息的延迟超过30分钟。 一个简单的dotnetwebclient可以毫不延迟地下载所有消息。然而,一旦nodejs应该下载消息,只有第一条消息会被毫无问题地下载,随后出现延迟。使用nodejs更改http代理选项keepalive和socket timeout并没有改善这种情况

    停止nodejs后,我不得不等待几分钟,DotNet客户端才真正开始正常工作。这可以复制好几次。我还发现简单的dotnetwebclient程序在连续启动和停止几次之后也出现了类似的问题


    无论如何,您的帖子向我展示了解决方案:关闭分区标志:)

    尝试使用amqp从azure service bus分区队列读取消息,这将适用于分区主题/队列,您甚至不必进行大量轮询

    const AMQPClient = require('amqp10').Client;
    const Policy = require('amqp10').Policy;
    
    const protocol = 'amqps';
    const keyName = 'RootManageSharedAccessKey';
    const sasKey = 'your_key_goes_here';
    const serviceBusHost = 'namespace.servicebus.windows.net';
    const uri = `${protocol}://${encodeURIComponent(keyName)}:${encodeURIComponent(sasKey)}@${serviceBusHost}`;
    const queueName = 'partitionedQueueName';
    const client = new AMQPClient(Policy.ServiceBusQueue);
    client.connect(uri)
    .then(() => Promise.all([client.createReceiver(queueName)]))
    .spread((receiver) => {
        console.log('--------------------------------------------------------------------------');
        receiver.on('errorReceived', (err) => {
            // check for errors
            console.log(err);
        });
        receiver.on('message', (message) => {
            console.log('Received message');
            console.log(message);
            console.log('----------------------------------------------------------------------------');
        });
    })
    .error((e) => {
        console.warn('connection error: ', e);
    });
    

    嘿,乔。我知道这与问题无关,但因为这是一个新问题。你是如何建立一个新的WebJob的?我在Azure中的旧门户或新门户的任何菜单上都找不到任何选项。。。我感到悲伤和绝望。。。谷歌对此没有帮助:(你需要设置一个“web应用程序”首先,它是web应用下的一个选项。然后,您可以使用git continuous integration部署到它。非常感谢!我已经通过Github部署了一个web api作为web应用。我将在那里添加WebJobs。您在节点端的接收速度如何?每秒有多少消息?我看到的是5或6毫秒/秒max@Joel哟你可以使用amqp
    const AMQPClient = require('amqp10').Client;
    const Policy = require('amqp10').Policy;
    
    const protocol = 'amqps';
    const keyName = 'RootManageSharedAccessKey';
    const sasKey = 'your_key_goes_here';
    const serviceBusHost = 'namespace.servicebus.windows.net';
    const uri = `${protocol}://${encodeURIComponent(keyName)}:${encodeURIComponent(sasKey)}@${serviceBusHost}`;
    const queueName = 'partitionedQueueName';
    const client = new AMQPClient(Policy.ServiceBusQueue);
    client.connect(uri)
    .then(() => Promise.all([client.createReceiver(queueName)]))
    .spread((receiver) => {
        console.log('--------------------------------------------------------------------------');
        receiver.on('errorReceived', (err) => {
            // check for errors
            console.log(err);
        });
        receiver.on('message', (message) => {
            console.log('Received message');
            console.log(message);
            console.log('----------------------------------------------------------------------------');
        });
    })
    .error((e) => {
        console.warn('connection error: ', e);
    });