Question

我正在尝试读取500 MB文本文件以通过HttpWebRequest发送内容。根据我的要求，我无法以块的形式发送数据。代码如下：

using (StreamReader reader = new StreamReader(filename))
{
    postData = reader.ReadToEnd();
}

byte[] byteArray = Encoding.UTF8.GetBytes(postData);
request.ContentType = "text/plain";
request.ContentLength = byteArray.Length;

Stream dataStream = request.GetRequestStream();
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Close();

WebResponse response = request.GetResponse();
Console.WriteLine(((HttpWebResponse)response).StatusDescription);

dataStream = response.GetResponseStream();

using (StreamReader reader = new StreamReader(dataStream))
{
    responseFromServer = reader.ReadToEnd();
}

Console.WriteLine(responseFromServer);
dataStream.Close();
response.Close();

读取这么大的文件会让我失去内存异常。有没有办法可以做到这一点？

Answer 1

所有文件都以块的形式传输 - 这就是以太网数据包;它只是一大块数据。我敢打赌，这个要求实际上意味着“这个文件必须在一个Web服务调用中传输。”

假设是这种情况，您将从磁盘读取数据到64KB缓冲区，然后将缓冲区写入请求。

request.ContentType = "text/plain";
request.ContentLength = byteArray.Length;
Stream dataStream = request.GetRequestStream();
int BUFFER_SIZE = 65536;
byte[] buffer = new byte[BUFFER_SIZE];
using (StreamReader reader = new StreamReader(filename)) {
    int count = 0;
    while (true) {
        int count = reader.Read(buffer, 0, BUFFER_SIZE);
        dataStream.Write(buffer, 0, count);
        if (count < BUFFER_SIZE) break;
    }
}
dataStream.Close();

Answer 2

听起来你可能遇到this documented issue with HttpWebRequest。根据知识库文章，尝试将HttpWebRequest.AllowWriteStreamBuffering属性设置为false。

内存不足异常 - 读取HttpWebRequest的大型文本文件

2 个答案: