HttpWebRequest FileUpload导致OutOfMemoryException

时间：2010-12-08 14:40:36

标签： vb.net file-upload httpwebrequest

我总是得到这个OutOfMemoryException：

 System.OutOfMemoryException was not handled.
 Message=Exception of type 'System.OutOfMemoryException' was thrown.
 Source=System
 StackTrace:
    at System.Net.ScatterGatherBuffers.AllocateMemoryChunk(Int32 newSize)
    at System.Net.ScatterGatherBuffers.Write(Byte[] buffer, Int32 offset, Int32 count)
    at System.Net.ConnectStream.InternalWrite(Boolean async, Byte[] buffer, Int32 offset, Int32 size, AsyncCallback callback, Object state)
    at System.Net.ConnectStream.Write(Byte[] buffer, Int32 offset, Int32 size)
    …

...当我执行以下源代码时，内存使用量正在稳步增加（根据Windows任务管理器）：

Imports System.Net
Imports System.Text
Imports System.IO

Public Class HttpFileUploader

    Public Function uploadFile(ByVal containa As CookieContainer, ByVal uri As String, ByVal filePath As String, ByVal fileParameterName As String, ByVal contentType As String, ByVal otherParameters As Specialized.NameValueCollection) As String

        Dim boundary As String = "---------------------------" & DateTime.Now.Ticks.ToString("x")
        Dim newLine As String = System.Environment.NewLine
        Dim boundaryBytes As Byte() = System.Text.Encoding.ASCII.GetBytes(newLine & "--" & boundary & newLine)
        Dim request As Net.HttpWebRequest = Net.WebRequest.Create(uri)

        request.ContentType = "multipart/form-data; boundary=" & boundary
        request.Method = "POST"
        request.CookieContainer = containa
        request.AllowAutoRedirect = True
        request.Timeout = -1

        Using requestStream As IO.Stream = request.GetRequestStream()

            Dim formDataTemplate As String = "Content-Disposition: form-data; name=""{0}""{1}{1}{2}"

            For Each key As String In otherParameters.Keys

                requestStream.Write(boundaryBytes, 0, boundaryBytes.Length)
                Dim formItem As String = String.Format(formDataTemplate, key, newLine, otherParameters(key))
                Dim formItemBytes As Byte() = System.Text.Encoding.UTF8.GetBytes(formItem)
                requestStream.Write(formItemBytes, 0, formItemBytes.Length)

            Next key

            requestStream.Write(boundaryBytes, 0, boundaryBytes.Length)

            Dim headerTemplate As String = "Content-Disposition: form-data; name=""{0}""; filename=""{1}""{2}Content-Type: {3}{2}{2}"
            Dim header As String = String.Format(headerTemplate, fileParameterName, filePath, newLine, contentType)
            Dim headerBytes As Byte() = System.Text.Encoding.UTF8.GetBytes(header)
            requestStream.Write(headerBytes, 0, headerBytes.Length)

            Using fileStream As New IO.FileStream(filePath, IO.FileMode.Open, IO.FileAccess.Read)

                Dim buffer(4096) As Byte
                Dim bytesRead As Int32 = fileStream.Read(buffer, 0, buffer.Length)

                Do While (bytesRead > 0)
                    ' the exception is triggered by the following line:     
                    requestStream.Write(buffer, 0, bytesRead)
                    bytesRead = fileStream.Read(buffer, 0, buffer.Length)
                Loop

            End Using

            Dim trailer As Byte() = System.Text.Encoding.ASCII.GetBytes(newLine & "--" + boundary + "--" & newLine)
            requestStream.Write(trailer, 0, trailer.Length)
            requestStream.Close()

        End Using

        Dim response As Net.WebResponse = Nothing
        Dim responseText = ""

        Try

            response = request.GetResponse()

            Using responseStream As IO.Stream = response.GetResponseStream()

                Using responseReader As New IO.StreamReader(responseStream)

                    responseText = responseReader.ReadToEnd()

                End Using

            End Using

        Catch exception As Net.WebException

            MsgBox(exception.Message)

        Finally

            response.Close()
            response = Nothing
            request = Nothing

        End Try

        Return responseText

    End Function

End Class

我已在注释中标记了触发异常的行。

我使用10个线程上传不同的文件（大小不超过250 MB）。

为什么我的代码耗尽内存？

3 个答案:

答案 0 :(得分：2)

这个线程似乎在大字节流上消除了一些的亮点：

http://social.msdn.microsoft.com/Forums/en-US/csharpgeneral/thread/1af59645-cdef-46a9-9eb1-616661babf90/

当使用10个线程一次发送250mb文件时，实际上可能内存不足！您使用的是哪种规格机器？

更新

我全力以赴，所以这就是我所做的。我创建了一个电子邮件帐户，但在阅读有关上传API的复杂性时感到无聊。因此我不确定这段代码是否有效。但是，它运行。我注意到在上面运行你的代码，在文件读取期间似乎没有做任何事情。但是，只要调用GetResponseStream，就会在那里上传文件。嗯，我们希望在我们的流阅读过程中上传它。

所以我尝试了Jeffrey Richter's Blog上的建议，并快速查看here以及其他一些上传示例。现在我在上面说过，我没有深入研究RapidShares上传api的复杂性，而不是流媒体操作本身。

请参阅下面的代码。除了为标题信息引入MemoryStream之外，它也是相同的，我也删除了页脚内容（以使字节数正确）。

    Dim ms As New MemoryStream()
    Dim formDataTemplate As String = "Content-Disposition: form-data; name=""{0}""{1}{1}{2}"

    For Each key As String In otherParameters.Keys
        ms.Write(boundaryBytes, 0, boundaryBytes.Length)
        Dim formItem As String = String.Format(formDataTemplate, key, newLine, otherParameters(key))
        Dim formItemBytes As Byte() = System.Text.Encoding.UTF8.GetBytes(formItem)
        ms.Write(formItemBytes, 0, formItemBytes.Length)
    Next key

    ms.Write(boundaryBytes, 0, boundaryBytes.Length)

    Dim headerTemplate As String = "Content-Disposition: form-data; name=""{0}""; filename=""{1}""{2}Content-Type: {3}{2}{2}"
    Dim header As String = String.Format(headerTemplate, fileParameterName, filePath, newLine, contentType)
    Dim headerBytes As Byte() = System.Text.Encoding.UTF8.GetBytes(header)
    ms.Write(headerBytes, 0, headerBytes.Length)

    Dim length As Long = ms.Length
    length += New FileInfo(filePath).Length
    request.ContentLength = length

    Using requestStream As IO.Stream = request.GetRequestStream()
        Dim bheader() As Byte = ms.ToArray()
        requestStream.Write(bheader, 0, bheader.Length)
        Using fileStream As New IO.FileStream(filePath, IO.FileMode.Open, IO.FileAccess.Read)

            Dim buffer(4096) As Byte
            Dim bytesRead As Int32 = fileStream.Read(buffer, 0, buffer.Length)

            Do While (bytesRead > 0)
                requestStream.Write(buffer, 0, bytesRead)
                bytesRead = fileStream.Read(buffer, 0, buffer.Length)
            Loop
        End Using
        requestStream.Close()
    End Using

这一点的重点是预先分配ContentLength属性，以便立即开始流式传输。它不是在GetResponseStream进行工作，而是在requestStream.Write命令上完成工作。计算MemoryStream中的字节，然后写入，然后将文件的大小添加到length变量并设置为ContentLength。这有用吗？！

答案 1 :(得分：2)

以下代码正在运行：

Imports System.Net
Imports System.Text
Imports System.IO

Public Class HttpFileUploader

    Public Function uploadFile(ByVal containa As CookieContainer, ByVal uri As String, ByVal filePath As String, ByVal fileParameterName As String, ByVal contentType As String, ByVal otherParameters As Specialized.NameValueCollection) As String

        Dim boundary As String = "---------------------------" & DateTime.Now.Ticks.ToString("x")
        Dim newLine As String = System.Environment.NewLine
        Dim boundaryBytes As Byte() = System.Text.Encoding.ASCII.GetBytes(newLine & "--" & boundary & newLine)
        Dim request As Net.HttpWebRequest = Net.WebRequest.Create(uri)

        request.ContentType = "multipart/form-data; boundary=" & boundary
        request.Method = "POST"
        request.CookieContainer = containa
        request.AllowAutoRedirect = True
        request.Timeout = -1
        request.KeepAlive = True
        request.AllowWriteStreamBuffering = False

        Dim ms As New MemoryStream()
        Dim formDataTemplate As String = "Content-Disposition: form-data; name=""{0}""{1}{1}{2}"

        For Each key As String In otherParameters.Keys
            ms.Write(boundaryBytes, 0, boundaryBytes.Length)
            Dim formItem As String = String.Format(formDataTemplate, key, newLine, otherParameters(key))
            Dim formItemBytes As Byte() = System.Text.Encoding.UTF8.GetBytes(formItem)
            ms.Write(formItemBytes, 0, formItemBytes.Length)
        Next key

        ms.Write(boundaryBytes, 0, boundaryBytes.Length)

        Dim headerTemplate As String = "Content-Disposition: form-data; name=""{0}""; filename=""{1}""{2}Content-Type: {3}{2}{2}"
        Dim header As String = String.Format(headerTemplate, fileParameterName, filePath, newLine, contentType)
        Dim headerBytes As Byte() = System.Text.Encoding.UTF8.GetBytes(header)
        ms.Write(headerBytes, 0, headerBytes.Length)

        Dim length As Long = ms.Length
        length += New FileInfo(filePath).Length
        request.ContentLength = length

        Using requestStream As IO.Stream = request.GetRequestStream()
            Dim bheader() As Byte = ms.ToArray()
            requestStream.Write(bheader, 0, bheader.Length)
            Using fileStream As New IO.FileStream(filePath, IO.FileMode.Open, IO.FileAccess.Read)

                Dim buffer(4096) As Byte
                Dim bytesRead As Int32 = fileStream.Read(buffer, 0, buffer.Length)

                Do While (bytesRead > 0)
                    requestStream.Write(buffer, 0, bytesRead)
                    bytesRead = fileStream.Read(buffer, 0, buffer.Length)
                Loop
            End Using
            requestStream.Close()
        End Using

        Dim response As Net.WebResponse = Nothing
        Dim responseText = ""

        Try

            response = request.GetResponse()

            Using responseStream As IO.Stream = response.GetResponseStream()

                Using responseReader As New IO.StreamReader(responseStream)

                    responseText = responseReader.ReadToEnd()

                End Using

            End Using

        Catch exception As Net.WebException

            MsgBox(exception.Message)

        Finally

            response.Close()
            response = Nothing
            request = Nothing
        End Try

        Return responseText

    End Function

End Class

很多感谢Tom和来自http://blogs.msdn.com/b/johan/archive/2006/11/15/are-you-getting-outofmemoryexceptions-when-uploading-large-files.aspx的人!!!

答案 2 :(得分：0)

Dim WebClient1 As New WebClient
WebClient1.UploadFile(...blablabla...)
相关问题
最新问题