Question

我被分配来修复一个给系统内存异常的问题。我是这个项目的新手，我正在尝试调试代码以找到问题的确切原因。当数据从oracle db打开并压缩数据集时，会发生此问题。数据集包含10个以上的表，在一个表中包含超过10万个数据，在同一个数据集中包含其他表中的相似数字。代码在Compression中编写如下，

public DataSetObject(DataSet dataSet)
        {
            if (dataSet != null)
            {
                using (dataSet)
                {
                    foreach (DataTable table in dataSet.Tables)
                    {
                        _totalRowCount += table.Rows.Count;
                    }

                    dataSet.RemotingFormat = SerializationFormat.Xml;
                    _dataSetAsString = Compressor.ConvertToString(dataSet);
                }
            }
        }

其中Compressor是实用程序calss，编写代码如下，

dsSurrogate = new DataSetSurrogate(dataSet); 
                formatter = new BinaryFormatter(); 
                formatter.TypeFormat = FormatterTypeStyle.XsdString; 

                using (MemoryStream ms = new MemoryStream())
                {
                    formatter.Serialize(ms, dsSurrogate); 
                    ms.Seek(0, SeekOrigin.Begin);

                    byteArray = ms.ToArray();
                }

                return Convert.ToBase64String(byteArray);

突然之后它会抛出异常，因为跨线程操作无效。并在日志文件中记录eception，如下所示，

System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.

Server stack trace: 
   at System.Text.StringBuilder.ToString()
   at System.IO.BinaryReader.ReadString()
   at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.ReadObjectString(BinaryHeaderEnum binaryHeaderEnum)
   at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.Run()
   at System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize(HeaderHandler handler, __BinaryParser serParser, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)
   at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream, HeaderHandler handler, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)
   at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.UnsafeDeserializeMethodResponse(Stream serializationStream, HeaderHandler handler, IMethodCallMessage methodCallMessage)
   at System.Runtime.Remoting.Channels.CoreChannel.DeserializeBinaryResponseMessage(Stream inputStream, IMethodCallMessage reqMsg, Boolean bStrictBinding)
   at System.Runtime.Remoting.Channels.BinaryClientFormatterSink.DeserializeMessage(IMethodCallMessage mcm, ITransportHeaders headers, Stream stream)
   at System.Runtime.Remoting.Channels.BinaryClientFormatterSink.SyncProcessMessage(IMessage msg)

Answer 1

你可以直接序列化到base64流，后者可以在后台序列化到MemoryStream或者一些临时文件（如果前者不够）：

            using (MemoryStream ms = new MemoryStream())
            // or use temporary FileStream for this if running out of memory
            {
                CryptoStream base64Stream = new CryptoStream(ms, new ToBase64Transform(), CryptoStreamMode.Write);
                formatter.Serialize(base64Stream, dsSurrogate); 
                ms.Seek(0, SeekOrigin.Begin);

                return ms.ToArray();
            }

Answer 2

您似乎没有压缩代码中的任何内容。

无论如何，你可以做几件事

在您的app / web config

gcAllowVeryLargeObjects

使您的项目仅在x64 CPU上运行，您可以从项目的属性中更改

This读得很好“.NET Framework 4.5包含针对客户端和服务器应用程序的新垃圾收集器增强功能”。

压缩Base64string中的大数据集

2 个答案: