通过php将csv文件上传到MySQL数据库

时间：2015-06-30 18:03:54

标签： php mysql csv pdo mysqli

我正在创建一个网页，我的非技术客户端可以上传一个包含25,000多行的csv文件。我找到了一种使用fgetcsv（）的方法，并创建一个如下所示的查询：

INSERT INTO `usbcData` (first, last, usbc, year) values
("Joshua", "Doom", "1324-1324", "2015");
INSERT INTO `usbcData` (first, last, usbc, year) values
("Turd", "Fergason", "1324-1325", "2014");
INSERT INTO `usbcData` (first, last, usbc, year) values
("Bruce", "Wayne", "1324-1326", "2013");
<Insert 20,000 more INSERTS here.>

问题是这需要太长时间。 20-30分钟太长了。所以现在我想让MySQL的LOAD DATA INFILE工作。当我从/home/joshua/file.csv（我在我的笔记本电脑上运行GNU / Linux）中包含文件时，我可以使用phpmyadmin，但不能使用我的php代码。我尝试使用带有和不带local关键字，pdo扩展名，mysqli扩展名和系统调用的查询，但都没有用。

我按照本指南：Error 1148 MySQL The used command is not allowed with this MySQL version设置my.cnf以允许本地关键字正常工作以及配置我的pdo连接以允许使用local关键字。

[mysqld]
local-infile 

[mysql]
local-infile

这是php脚本，用户上传的csv文件发送到该脚本。它包含几种尝试使用获取MySQL的LOAD DATA INFILE工作的方法：

<?php
ini_set('display_errors', 'On');
error_reporting(E_ALL | E_STRICT);

echo "The file name is ".$_FILES['userfile']['name']."<br>";
echo "The file type is ".$_FILES['userfile']['type']."<br>";
echo "The size of the file is ".$_FILES['userfile']['size']." bytes <br>";
echo "The file is stored on the server as ".$_FILES['userfile']['tmp_name']."<br>";

$file_name = $_FILES['userfile']['name'];
$file_size = $_FILES['userfile']['size'];
$file_tmp = $_FILES['userfile']['tmp_name'];
$file_type = $_FILES['userfile']['type'];

$user = "root";
$pass = "passord";

try {
  $dbh = new PDO('mysql:host=localhost;dbname=usbc', $user, $pass,
               array(PDO::MYSQL_ATTR_LOCAL_INFILE => true,
                     PDO::ATTR_PERSISTENT => true)
);
} catch (Exception $e) {
  die("Unable to connect: " . $e->getMessage());
}


$dbh->query("TRUNCATE TABLE `usbcData`");

if (chmod($file_tmp, 0777)) {
    echo "chmod worked on $file_tmp <br>";
} else {
    echo "chmod didn't work on $file_tmp <br>";
}
$sql = 'load data infile "'.$file_tmp.'" into table `usbcData` ';
$sql .= 'fields terminated by "," ';
$sql .= 'lines terminated by "\n" ';
$sql .= '(first, last, usbc, year) ';
$dbh->query($sql);
//$result = $dbh->query($sql);
echo $sql."<br>";

$sql = 'load data infile "'.$file_name.'" into table `usbcData` ';
$sql .= 'fields terminated by "," ';
$sql .= 'lines terminated by "\n" ';
$sql .= '(first, last, usbc, year) ';
$dbh->query($sql);
//$result = $dbh->query($sql);
echo $sql."<br>";

$sql = 'load data local infile "'.$file_tmp.'" into table `usbcData` ';
$sql .= 'fields terminated by "," ';
$sql .= 'lines terminated by "\n" ';
$sql .= '(first, last, usbc, year) ';
$dbh->query($sql);
//$result = $dbh->query($sql);
echo $sql."<br>";

$sql = 'load data local infile "'.$file_name.'" into table `usbcData` ';
$sql .= 'fields terminated by "," ';
$sql .= 'lines terminated by "\n" ';
$sql .= '(first, last, usbc, year) ';
$dbh->query($sql);
//$result = $dbh->query($sql);
echo $sql."<br>";

$dbh = null;

$mysqli = new mysqli("localhost", "root", "password", "usbc");
if ($mysqli->connect_errno) {
    echo "Failed to connect to MySQL: (" . $mysqli->connect_errno . ") " . $mysqli->connect_error;
}

$sql = 'load data infile "'.$file_tmp.'" into table `usbcData` ';
$sql .= 'fields terminated by "," ';
$sql .= 'lines terminated by "\n" ';
$sql .= '(first, last, usbc, year) ';
$mysqli->query($sql);
//$result = $dbh->query($sql);
echo $sql."<br>";

$sql = 'load data infile "'.$file_name.'" into table `usbcData` ';
$sql .= 'fields terminated by "," ';
$sql .= 'lines terminated by "\n" ';
$sql .= '(first, last, usbc, year) ';
$mysqli->query($sql);
//$result = $dbh->query($sql);
echo $sql."<br>";

$sql = 'load data local infile "'.$file_tmp.'" into table `usbcData` ';
$sql .= 'fields terminated by "," ';
$sql .= 'lines terminated by "\n" ';
$sql .= '(first, last, usbc, year) ';
$mysqli->query($sql);
//$result = $dbh->query($sql);
echo $sql."<br>";

$sql = 'load data local infile "'.$file_name.'" into table `usbcData` ';
$sql .= 'fields terminated by "," ';
$sql .= 'lines terminated by "\n" ';
$sql .= '(first, last, usbc, year) ';
$mysqli->query($sql);
//$result = $dbh->query($sql);
echo $sql."<br>";




$dbUser = "root";
$dbHost = "localhost";
$dbPass = "password";
$dbName = "usbc";

$sql = "LOAD DATA INFILE '$file_tmp' INTO TABLE `usbcData` FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n';";
echo $sql."<br>";
system("mysql -u $dbUser -h $dbHost --password=$dbPass --local_infile=1 -e \"$sql\" $dbName");

$sql = "LOAD DATA INFILE '$file_name' INTO TABLE `usbcData` FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n';";
echo $sql."<br>";
system("mysql -u $dbUser -h $dbHost --password=$dbPass --local_infile=1 -e \"$sql\" $dbName");

$sql = "LOAD DATA local INFILE '$file_tmp' INTO TABLE `usbcData` FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n';";
echo $sql."<br>";
system("mysql -u $dbUser -h $dbHost --password=$dbPass --local_infile=1 -e \"$sql\" $dbName");

$sql = "LOAD DATA local INFILE '$file_name' INTO TABLE `usbcData` FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n';";
echo $sql."<br>";
system("mysql -u $dbUser -h $dbHost --password=$dbPass --local_infile=1 -e \"$sql\" $dbName");

?>

输出如下：

The file name is MonthlyCSVDataFileforIndianaYouth.csv
The file type is text/csv
The size of the file is 759494 bytes 
The file is stored on the server as /tmp/phpT9cyBK
/tmp/phpT9cyBK is writable
chmod worked on /tmp/phpT9cyBK 
load data infile "/tmp/phpT9cyBK" into table `usbcData` fields terminated by "," lines terminated by "\n" (first, last, usbc, year) 
load data infile "MonthlyCSVDataFileforIndianaYouth.csv" into table  `usbcData` fields terminated by "," lines terminated by "\n" (first, last,  usbc, year) 
load data local infile "/tmp/phpT9cyBK" into table `usbcData` fields terminated by "," lines terminated by "\n" (first, last, usbc, year) 
load data local infile "MonthlyCSVDataFileforIndianaYouth.csv" into table `usbcData` fields terminated by "," lines terminated by "\n" (first, last, usbc, year) 
load data infile "/tmp/phpT9cyBK" into table `usbcData` fields terminated by "," lines terminated by "\n" (first, last, usbc, year) 
load data infile "MonthlyCSVDataFileforIndianaYouth.csv" into table `usbcData` fields terminated by "," lines terminated by "\n" (first, last, usbc, year) 
load data local infile "/tmp/phpT9cyBK" into table `usbcData` fields terminated by "," lines terminated by "\n" (first, last, usbc, year) 
load data local infile "MonthlyCSVDataFileforIndianaYouth.csv" into table `usbcData` fields terminated by "," lines terminated by "\n" (first, last, usbc, year) 
LOAD DATA INFILE '/tmp/phpT9cyBK' INTO TABLE `usbcData` FIELDS TERMINATED BY ',' LINES TERMINATED BY ' ';
LOAD DATA INFILE 'MonthlyCSVDataFileforIndianaYouth.csv' INTO TABLE `usbcData` FIELDS TERMINATED BY ',' LINES TERMINATED BY ' ';
LOAD DATA local INFILE '/tmp/phpT9cyBK' INTO TABLE `usbcData` FIELDS TERMINATED BY ',' LINES TERMINATED BY ' ';
LOAD DATA local INFILE 'MonthlyCSVDataFileforIndianaYouth.csv' INTO TABLE `usbcData` FIELDS TERMINATED BY ',' LINES TERMINATED BY ' ';

任何帮助都会很好。 ：）

2 个答案:

答案 0 :(得分：0)

如果您尝试通过脚本语言（如PHP）发送文件，则不需要

LOCAL关键字。尝试在没有LOCAL的情况下运行您的查询。

如果这不起作用，请尝试将文件复制到某处并使用LOAD DATA INFILE中的新路径，最后删除该文件。

答案 1 :(得分：0)

我终于开始工作了！我接受了Robo的建议并尝试使用

将文件复制到我的工作目录中 
$file_copy = getcwd()."/uploaded-file.csv";
copy ($file_tmp, $file_copy);

起初没有用，因为Apache / php在用户“http”下运行，用户“http”没有写入我的工作目录的权限。 所以我把用户http添加到了joshua组，因为/ home / joshua下的所有目录都有joshua组。 

#add user http to group joshua
sudo gpasswd -a http joshua

然后我退出并重新登录，以确保权限已更新。 然后我尝试使用这个MySQL查询上传正确的文件：

$sql  = "load data CONCURRENT infile \"$file_copy\" ";
$sql .= "into table `usbcData` ";
$sql .= "fields terminated by \",\" ";
$sql .= "lines terminated by \"\n\" ";
$sql .= "(first, last, usbc, year) ";
echo $sql."<br>";
$dbh->query($sql);

但是当我运行php脚本时，表usbcData表没有更新。我尝试在phpmyadmin中运行sql查询，它给了我一个权限错误。显然我的MySQL实现是在用户“mysql”下运行的，它也没有正确的权限来访问具有joshua组的文件。所以我把mysql添加到了joshua小组

#add user mysql to group joshua
sudo gpasswd -a mysql joshua

我退出并重新登录，它运作正常！呜啊！感谢上帝的stackoverflow，因为否则我就没有工作。附： Robo，如果我有足够的代表，我会投票给你答案。

相关问题
最新问题