Question

我试图将一个非常大的CSV文件导入我的数据库（本地）该文件是230MB，大约880万行我遇到的问题不是打开CSV或者不知道如何导入它，文件打开，导入大约500,000行，然后退出并没有错误或超时或任何东西，我只是看到我的网页。

这是代码：

    try {
    $conn = new PDO("mysql:host=$servername;dbname=adresses_database", $username, $password);
    // set the PDO error mode to exception
    $conn->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
    echo "Connected successfully"; 
    $row = 1;
if (($handle = fopen("bagadres.csv", "c+")) !== FALSE) {
    while (($data = fgetcsv($handle, '', ";")) !== FALSE) {
        if (!isset($write_position)) {        // move the line to previous position, except the first line
                $write_position = 0;
        $num = count($data); // $num is 15
        $row++; //i dont need this?
        $stmt = $conn->prepare("INSERT INTO adresses (openbareruimte, huisnummer, huisletter, huisnummertoevoeging, postcode, woonplaats, gemeente, provincie, object_id, object_type, nevenadres, x, y, lon, lat) VALUES (:openbareruimte, :huisnummer, :huisletter, :huisnummertoevoeging, :postcode, :woonplaats, :gemeente, :provincie, :object_id, :object_type, :nevenadres, :x, :y, :lon, :lat)");
        $stmt->bindParam(':openbareruimte', $data[0]);
        $stmt->bindParam(':huisnummer', $data[1]);
        $stmt->bindParam(':huisletter', $data[2]);
        $stmt->bindParam(':huisnummertoevoeging', $data[3]);
        $stmt->bindParam(':postcode', $data[4]);
        $stmt->bindParam(':woonplaats', $data[5]);
        $stmt->bindParam(':gemeente', $data[6]);
        $stmt->bindParam(':provincie', $data[7]);
        $stmt->bindParam(':object_id', $data[8]);
        $stmt->bindParam(':object_type', $data[9]);
        $stmt->bindParam(':nevenadres', $data[10]);
        $stmt->bindParam(':x', $data[11]);
        $stmt->bindParam(':y', $data[12]);
        $stmt->bindParam(':lon', $data[13]);
        $stmt->bindParam(':lat', $data[14]);
    $stmt->execute();
} else {
                $read_position = ftell($handle); // get actual line
                fseek($handle, $write_position); // move to previous position
                fputs($handle, $line);           // put actual line in previous position
                fseek($handle, $read_position);  // return to actual position
                $write_position += strlen($line);    // set write position to the next loop
            }
            fflush($handle);                         // write any pending change to file
        ftruncate($handle, $write_position);     // drop the repeated last line
        flock($handle, LOCK_UN);   
    }
   fclose($handle);
}
    }
catch(PDOException $e)
    {
    echo "Connection failed: " . $e->getMessage();
    }

$conn = null;

我到目前为止寻找stackoverflow和PHP手册的帮助，我也搜索了它是否是一个mysql错误。但我无法弄清楚这一点，（关于使用linux mint 18的mysql设置的任何建议）

Answer 1

我强烈建议您使用MySQL LOAD DATA INFILE，这可能是将CSV数据导入MySQL表的最快和最有效的方法。您设置的命令如下所示：

LOAD DATA INFILE 'bagadres.csv' 
INTO TABLE adresses
FIELDS TERMINATED BY ',' 
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS

如果您的字段未用引号括起来，或者用引号以外的其他内容括起来，则删除或修改ENCLOSED BY子句。此外，IGNORE 1 ROWS将忽略第一行，假设您的文件的第一行是标题行（即不是实际数据而是列标签），这将是有意义的。

SQL导入CSV文件与PHP

1 个答案: