动态扩展数组以导入数据的好方法是什么?

时间:2014-12-17 22:19:46

标签: c++ arrays vector heap deque

我正在寻找帮助来创建动态扩展阵列以从.csv文件导入数据。我不想看到文件有多大并在源代码中编辑变量/提示用户,我只想要导入数据然后以各种方式进行操作。首先,我的代码是:

#include <fstream>
#include <sstream>
#include <iostream>

int main()
{

//declare variables and arrays
long rows = 170260;
int cols = 5;
double **rawData = new double*[rows]; //on heap because of size
for(long pi = 0; pi < rows; ++pi) //create an array of pointers
{
         rawData[pi] = new double[cols];
}
char buff[200];
double deltaT;
double carDeltaV;
double *carV = new double[rows]; //on heap because of size

//import raw data
std::cout << "Importing filedata.csv...";

std::ifstream rawInput("filedata.csv");

for(long r = 0; r < rows; ++r)
{
      rawInput.getline(buff, 200);
      std::stringstream ss(buff);

      for(int c = 0; c < cols; ++c) 
      {
            ss.getline(buff, 40, ',');
            rawData[r][c] = atof(buff);
      }
}

std::cout << "Done." << std::endl;

//create speed matrix
carV[0] = 0;

std::cout << std::endl << "Creating speed matrix...";

for (long i = 1; i < rows; ++i) 
{

    deltaT = rawData[i][0] - rawData[i-1][0];
    carDeltaV = rawData[i-1][3] * deltaT;
    carV[i] = carDeltaV + carV[i-1];
}

std::cout << "Done." << std::endl;

//write data to csv file
std::cout << std::endl << "Writing data to file...";

std::ofstream outputData;
outputData.open("outputdata.csv");

for(long r = 0; r < rows; ++r)
{
         outputData << rawData[r][0] << "," << rawData[r][3]/.00981 << ",";
         outputData << carV[r] << std::endl;
}

outputData.close();
std::cout << "Done." << std::endl;

//delete pointers
std::cout << std::endl << "Clearing memory...";

for(long pj = 0; pj < rows; ++pj)
{
         delete [] rawData[pj];
}
delete [] rawData;
delete [] carV;

std::cout << "Done." << std::endl;

std::cin.get();
return 0;

}

注意:colums的数量总是为5.行是我未知的。我将导入的示例如下所示:

0.001098633,0.011430004,0.002829004,-0.004371409,0.00162947
0.001220703,0.00606778,0.001273052,0.003497127,0.002359922
0.001342773,0.003104446,-0.000848701,0.012385657,-0.008119254

还有更多内容,但这应该足以理解我想要实现的目标。我对矢量有点了解,但矢量矢量的概念对我来说有点混乱,我试图实现它没有成功。另外,我读到一个双端队列可能就是我想要的东西?我对这些没有任何经验,而且在我看来,对我的应用程序来说可能有些过分,因为我只是向一个方向追加到一组数据。

免责声明:我几乎是C ++的新手,所以如果你认为有任何概念超出我的技能水平,请告诉我,以便我可以阅读。

有什么建议吗?

编辑:根据请求,这是我尝试使用向量的方法。

std::vector<double> rawDataRow;
std::vector< std::vector<double> > rawDataMatrix;

//import raw data loop
std::ifstream rawInput("test.csv");

for(int i = 1; i > 0; ) {
          rawInput.getline(buff, 200);
          std::stringstream ss(buff);

          for(int c = 0; c < cols; ++c) {
                  ss.getline(buff, 40, ',');
                  value = atof(buff);
                  rawDataRow.push_back(value);

                  std::cout << rawDataRow[0] << std::endl;
          }
          timeDiff = timeAfter - timeBefore;
          timeBefore = timeAfter;
          timeAfter = rawDataRow[0];

          rawDataMatrix.push_back(rawDataRow);
}

其中&#34;我&#34;在eof上将被设置为0。

3 个答案:

答案 0 :(得分:0)

总结讨论中出现的问题:

你不能有一个数组向量,请参见:Correct way to work with vector of arrays你可以有一个指向数组的指针向量。但是,在这一点上,我不会搞乱所有的内存处理。

最好的方法是使用向量保存代码,除非您应该在循环中放置rawDataRow的定义以在每次迭代时清除其内容。

std::vector< std::vector<double> > rawDataMatrix;

//import raw data loop
std::ifstream rawInput("test.csv");

for(int i = 1; i > 0; ) {
      std::vector<double> rawDataRow;
      rawInput.getline(buff, 200);
      std::stringstream ss(buff);

      // do the rest
}

答案 1 :(得分:0)

看来你的生活太过艰难。但是,关键的实现是,始终需要检查之前的输入以某种形式使用它。一旦你这样做,事情很容易就会到位。

为了方便地处理一行的输入,我要定义的第一件事就是这个简单的操纵器会忽略一个逗号:

std::istream& comma(std::istream& in) {
    if ((in >> std::ws).peek() == ',') {
        in.ignore(); // the happy case: just skip over the comma
    }
    else if (!in.peek() == std::char_traits<char>::eof()) {
        in.setstate(std::ios_base::failbit); // unhappy: not the end and not a comma
    }
    return in;
}

有了这个,读取线条并将它们分成单元格相当容易:

std::vector<std::vector<double>> result;
for (std::string line; std::getline(in, line); ) {
    std::istringstream lin(line);
    std::vector<double> row;
    for (double d; d >> lin >> comma; ) {
        row.push_back(d);
    }
    if (!lin.eof()) {
        in.setstate(std::ios_base::failbit);
    }
    std::result.push_back(row);
}
if (!in.eof()) {
    std::cout << "there was an input error\n";
}
else {
    // result contains the result of reading...
}

我还没有对代码进行过测试,我猜测某些地方存在拼写错误,但一般方法应该正常工作......

答案 2 :(得分:0)

首先,您应该将程序分为三个部分:

  1. 从输入文件中读取数据
  2. 处理数据
  3. 将数据写入输出文件
  4. 你的主程序基本上应该是这样的:

    int main() {
      vector<InputRecord> data = read_from_csv("filedata.csv");
      vector<double> speeds = compute_speeds(data);
      write_to_csv("result.csv", data, speeds);
      return 0;
    }
    

    现在您需要定义InputRecord是什么。你说它是5个双打的阵列,但这不是最好的描述。应该更像这样:

    struct InputRecord {
      double timestamp;
      double field2;
      double field3;
      double location;
      double field5;
    };
    

    使用此数据结构,您可以编写data[0].timestamp而不是data[0][0],这意味着您不再需要评论。

    这是我为此任务编写的完整代码。它与你的相似,应该是一个好的起点。请注意,此代码根本不进行显式内存管理。

    #include <cstdio>
    #include <cstdlib>
    #include <fstream>
    #include <iostream>
    #include <string>
    #include <vector>
    
    using std::string;
    using std::vector;
    
    struct InputRecord {
      double timestamp;
      double field2;
      double field3;
      double location;
      double field5;
    };
    
    vector<InputRecord> read_from_csv(const char *filename) {
      std::ifstream in(filename);
      vector<InputRecord> data;
    
      if (!in.is_open()) {
        throw std::ios_base::failure(string()
            + "cannot open input file \"" + filename + "\".");
      }
    
      string line;
      while (std::getline(in, line)) {
        InputRecord rec;
        char end_of_line;
        if (std::sscanf(line.c_str(), "%lf,%lf,%lf,%lf,%lf%c",
            &rec.timestamp, &rec.field2, &rec.field3,
            &rec.location, &rec.field5, &end_of_line) != 5) {
          throw std::ios_base::failure(string()
              + "input file \"" + filename + "\" "
              + "contains invalid data: \"" + line + "\"");
        }
        data.push_back(rec);
      }
      if (in.bad()) {
        throw std::ios_base::failure(string() + "error while reading data");
      }
      return data;
    }
    
    vector<double> calculate_speeds(const vector<InputRecord> &data) {
      vector<double> speeds;
    
      speeds.push_back(0.0);
      for (std::size_t i = 1; i < data.size(); i++) {
        double delta_t = data[i].timestamp - data[i - 1].timestamp;
        double delta_s = data[i].location - data[i - 1].location;
        speeds.push_back(delta_s / delta_t);
      }
      return speeds;
    }
    
    void write_to_csv(const char *filename, const vector<InputRecord> &data,
        const vector<double> &speeds) {
      std::ofstream out(filename);
    
      if (!out.is_open()) {
        throw std::ios_base::failure(string()
            + "cannot open output file \"" + filename + "\".");
      }
      for (std::size_t i = 0; i < data.size(); i++) {
        out << data[i].timestamp << "," << speeds[i] << "\n";
      }
      if (out.bad()) {
        throw std::ios_base::failure(string() + "error while writing data");
      }
    }
    
    int main() {
      vector<InputRecord> data = read_from_csv("in.csv");
      vector<double> speeds = calculate_speeds(data);
      write_to_csv("out.csv", data, speeds);
      return 0;
    }