Question

尝试通过字符串编写器将数据表数据转换为xml时。它给出了内存异常。

我的数据表包含大约20000条记录。

以下是我正在尝试的代码

DataTable dt = GetData();// contains 20000 records.

    StringWriter sw = new StringWriter();
    dt.WriteXml(sw);

    XmlDocument sd = new XmlDocument();
    sd.LoadXml(sw.ToString());

请有人帮助我。

Answer 1

您可以做的一件事是减少内存使用，即删除中间StringBuilder和string表示，并通过传递{DataTable直接从XmlDocument写入XmlWriter {1}}从XmlDocument.CreateNavigator.AppendChild()返回DataTable.WriteXml(XmlWriter)：

XmlDocument sd = new XmlDocument();
using (XmlWriter writer = sd.CreateNavigator().AppendChild())
{
    dt.WriteXml(writer);
}

这也应该更高效，因为不再需要解析中间XML字符串。

以下是我用于在x86进程上验证此方法的测试代码：

class TestClass
{
    internal static void Test()
    {
        // Verify the old and new algorithms generate identical XML
        foreach (var i in new[] { 0, 1, 2, 100 })
        {
            var old1 = TestStringWriteAndParse(i);
            var new1 = TestDirectWrite(i);

            if (old1.OuterXml != new1.OuterXml)
            {
                throw new InvalidOperationException("old1.OuterXml != new1.OuterXml");
            }
        }

        // Find a number of records that generate an out-of-memory exception whcn converting to an intermediate StringBuilder and string:
        uint size = 20000;
        try
        {
            while (size < int.MaxValue / 2)
            {
                TestStringWriteAndParse((int)size);
                size = checked(size * 2);
                GC.Collect();
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine(ex);
        }

        GC.Collect();

        // Verify that number of records can be written directly using  XmlDocument.CreateNavigator.AppendChild()

        try
        {
            TestDirectWrite((int)size);
            Console.WriteLine("SUCCEEDED in writing {0} DataTable records to an XmlDocument", size);
        }
        catch (Exception ex)
        {
            Console.WriteLine("FAILED in writing {0} DataTable records to an XmlDocument:", size);
            Console.WriteLine(ex);
            throw;
        }
    }

    static DataTable GetData(int count)
    {
        var table = new DataTable();
        table.TableName = "Test";

        table.Columns.Add("Name", typeof(string));
        table.Columns.Add("Height", typeof(double));
        table.Columns.Add("NetWorth", typeof(decimal));

        for (int i = 0; i < count; i++)
        {
            DataRow row = table.NewRow();
            row["Name"] = "Bob Cratchit " + i.ToString();
            row["Height"] = 6.023;
            row["NetWorth"] = 101.01 + (10 * i);
            table.Rows.Add(row);
        }
        return table;
    }

    static XmlDocument TestDirectWrite(int count)
    {
        DataTable dt = GetData(count);// contains 20000 records.

        XmlDocument sd = new XmlDocument();
        using (XmlWriter writer = sd.CreateNavigator().AppendChild())
        {
            dt.WriteXml(writer);
        }

        return sd;
    }

    static XmlDocument TestStringWriteAndParse(int count)
    {
        DataTable dt = GetData(count);// contains 20000 records.

        StringWriter sw = new StringWriter();
        dt.WriteXml(sw);

        XmlDocument sd = new XmlDocument();
        sd.LoadXml(sw.ToString());

        return sd;
    }
}

在我的情况下，原始代码在新代码成功时尝试序列化1280000条记录失败。

Answer 2

您可以尝试使用文件流而不是字符串编写器。

FileStream stream = File.OpenRead(path);
XMLDocument doc = new XMLDocuemnt();
doc.Load(stream);

将datatable转换为xml，提供outofmemory异常

2 个答案: