Question

我正在使用Axios在后端进行大型数据提取：

await Axios({
      method: 'GET',
      url,
      headers: {
        'content-type': 'application/json',
        Authorization: `Bearer ${forgeAccessToken}`,
      },
    });

NodeJS崩溃。内存问题：

  1|rest-backenq  | FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

我将内存从4GB增加到16GB。

没有效果。

我无法控制从中获取数据的外部API。

该如何解决？

Answer 1

我认为您的文件正在下载到内存中，请使用request模块而不是Axios来下载文件

var fs = require('fs');
var request = require('request');
var progress = require('request-progress');

// The options argument is optional so you can omit it 
progress(request({
    url : url,
    headers: {
        'content-type': 'application/json',
        'Authorization': `Bearer ${forgeAccessToken}`
    }
}), {
    // throttle: 2000,                    // Throttle the progress event to 2000ms, defaults to 1000ms 
    // delay: 1000,                       // Only start to emit after 1000ms delay, defaults to 0ms 
    // lengthHeader: 'x-transfer-length'  // Length header to use, defaults to content-length 
})
.on('progress', function (state) {
    // The state is an object that looks like this: 
    // { 
    //     percent: 0.5,               // Overall percent (between 0 to 1) 
    //     speed: 554732,              // The download speed in bytes/sec 
    //     size: { 
    //         total: 90044871,        // The total payload size in bytes 
    //         transferred: 27610959   // The transferred payload size in bytes 
    //     }, 
    //     time: { 
    //         elapsed: 36.235,        // The total elapsed seconds since the start (3 decimals) 
    //         remaining: 81.403       // The remaining seconds to finish (3 decimals) 
    //     } 
    // } 
    console.log('progress', state);
})
.on('error', function (err) {
    // Do something with err 
})
.on('end', function () {
    // Do something after request finishes 
})
.pipe(fs.createWriteStream('IE11.Win8.1.For.Windows.VirtualBox.zip'));

有关此问题的更多信息， What is way to download big file in NodeJS?

根据外部数据请求，NodeJ的内存不足

1 个答案: