NodeJS可写流去抖动

时间:2016-01-21 12:42:59

标签: node.js mongodb stream timeout debouncing

我最近实现了一个可写流,它将来自多个可读流的聚合数据持久保存到mongo数据库中。由于流的高吞吐量,我希望实现一些去抖功能。

我正在努力确定这样做的好方法,麻烦来自需要在管道中施加背压。基本上,如果去抖动超时执行,可写的' next()'只有在超时功能完成后才需要调用回调,但是如果超时尚未执行,则可写入需要' next()'立即保持流动。

我觉得我在这里错过了一些简单的东西,希望这里有人可以让我自己踢。

_write()逻辑解释/ TLDR:

// Queues a mongo bulk upsert operation.
queueMongoUpsertOperation(someData);

// This should only be called NOW if the timeout isn't executing
// as we should wait for it to finish the save before reading
// more data IF it is saving.
// Best/SAFEST way to block this call when the timeout is executing
next()

// Simple debounce.
clearTimeout(timeout);

timeout = setTimeout(function(){

    // Execute the actual asyncronous save from the queued upsert operations.
    executeMongoBulkUpsert(function(){

        // Because the next() above is somehow blocked when the timeout
        // is executed, we should now called next() from within
        // the debounced function.
        next();

    });

}, 1000);

0 个答案:

没有答案