使用SUM的MySQL LEFT JOIN很慢

时间：2012-09-07 03:39:07

标签： mysql performance query-optimization sum left-join

我需要想法来优化此查询：

SELECT U.Id
    , U.Name
    , SUM(P.Credit) AS CreditAmount
    , SUM(P.Debit) AS DebitAmount
    FROM users U
    LEFT JOIN payments P ON P.UserId = U.Id
    WHERE U.StatusId = 1 /* Active Users Only */
    GROUP BY U.Id

有关表格的一些信息：

  1. users表：10,000+条记录
  2. payments表：2,000,000条记录，编入UserId列

    3. 查询最多需要2分钟。通过调查查询，我发现SUM函数是查询速度慢的原因。尝试在没有SUM的情况下执行查询只需不到2秒。

    是否有空间来改进此查询？感谢

    编辑1

    这是我在使用EXPLAIN：

    时得到的 
    id  select_type table   type    possible_keys   key     key_len     ref             rows    Extra
1   SIMPLE      U       ALL                                                         8304    Using where; Using temporary; Using filesort
1   SIMPLE      P       ref     UserId          UserId  5           Database.U.Id   361     ""

    查询中正在使用索引payments.UserId。有什么想法吗？

    编辑2 表信息：

    users   "CREATE TABLE `users` (
  `Id` int(11) NOT NULL auto_increment,
  `StatusId` int(11) default NULL,
  `Name` varchar(60) default NULL,
  `TimeZone` varchar(100) default NULL,
  PRIMARY KEY  (`Id`),
  KEY `StatusId` (`StatusId`),
  CONSTRAINT `FK_User_Status` FOREIGN KEY (`StatusId`) REFERENCES `userStatus` (`Id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8"

payments    "CREATE TABLE `payments` (
  `Id` int(11) NOT NULL auto_increment,
  `UserId` int(11) default NULL,  
  `Description` varchar(200) default NULL,
  `Debit` decimal(11,2) default NULL,
  `Credit` decimal(11,2) default NULL,
  `Date` datetime default NULL,
  PRIMARY KEY  (`Id`),
  KEY `UserId` (`UserId`),
  CONSTRAINT `FK_Payment_User` FOREIGN KEY (`UserId`) REFERENCES `users` (`Id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8"

3 个答案:

答案 0 :(得分：1)

如果添加以下索引：

CREATE INDEX ix_status ON users (StatusId, Id);
CREATE INDEX ix_userid ON payments (UserId, Credit, Debit);

并添加FORCE INDEX选项：

SELECT U.Id
    , U.Name
    , SUM(P.Credit) AS CreditAmount
    , SUM(P.Debit) AS DebitAmount
    FROM 
        users U FORCE INDEX (ix_status)
    LEFT JOIN 
        payments P FORCE INDEX (ix_userid) ON P.UserId = U.Id
    WHERE 
        U.StatusId = 1 /* Active Users Only */
    GROUP BY
        U.Id;

您的查询几乎肯定会表现得更好。

答案 1 :(得分：1)

Kael，这是我运行的脚本，以及它的输出，以显示我在第一个答案中提供的索引和查询，将查询时间从大约14.3秒缩短到0.75秒：

DROP TABLE IF EXISTS payments;
DROP TABLE IF EXISTS users;
DROP TABLE IF EXISTS userStatus;

CREATE TABLE userStatus (
    Id int(11) NOT NULL auto_increment,
    PRIMARY KEY  (Id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE users (
    Id int(11) NOT NULL auto_increment,
    StatusId int(11) default NULL,
    Name varchar(60) default NULL,
    TimeZone varchar(100) default NULL,
    PRIMARY KEY  (Id),
    KEY StatusId (StatusId),
    CONSTRAINT FK_User_Status FOREIGN KEY (StatusId) REFERENCES userStatus (Id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE payments (
    Id int(11) NOT NULL auto_increment,
    UserId int(11) default NULL,  
    Description varchar(200) default NULL,
    Debit decimal(11,2) default NULL,
    Credit decimal(11,2) default NULL,
    Date datetime default NULL,
    PRIMARY KEY  (Id),
    KEY UserId (UserId),
    CONSTRAINT FK_Payment_User FOREIGN KEY (UserId) REFERENCES users (Id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

DROP TABLE IF EXISTS ids;
CREATE TABLE IF NOT EXISTS ids (
    id INT UNSIGNED AUTO_INCREMENT NOT NULL,
    PRIMARY KEY (id)
) ENGINE=MyISAM; -- must be MyISAM
INSERT INTO ids SET id = NULL; # inserts a 1
UPDATE ids SET id=0;
ALTER TABLE ids AUTO_INCREMENT=1;
INSERT INTO ids SELECT NULL FROM ids; #      1      1
INSERT INTO ids SELECT NULL FROM ids; #      2      2
INSERT INTO ids SELECT NULL FROM ids; #      4      4
INSERT INTO ids SELECT NULL FROM ids; #      8      8
INSERT INTO ids SELECT NULL FROM ids; #     10     16
INSERT INTO ids SELECT NULL FROM ids; #     20     32
INSERT INTO ids SELECT NULL FROM ids; #     40     64
INSERT INTO ids SELECT NULL FROM ids; #     80    128
INSERT INTO ids SELECT NULL FROM ids; #    100    256
INSERT INTO ids SELECT NULL FROM ids; #    200    512
INSERT INTO ids SELECT NULL FROM ids; #    400  1,024
INSERT INTO ids SELECT NULL FROM ids; #    800  2,048
INSERT INTO ids SELECT NULL FROM ids; #   1000  4,096
INSERT INTO ids SELECT NULL FROM ids; #   2000  8,192
INSERT INTO ids SELECT NULL FROM ids; #   4000 16,384

INSERT INTO 
userStatus
(id)
SELECT
id
FROM
ids
WHERE
id > 0
ORDER BY
id
LIMIT 2;

INSERT INTO 
users
(Id, StatusId)
SELECT
id,
IF(RAND() < .9, 1, 2) -- 90% of users are 'Active'
FROM
ids
WHERE
id > 0 AND
id <= 20000
ORDER BY
id;

INSERT INTO
payments
(Id, UserId, Debit, Credit, `Date`)
SELECT
NULL,
i1.id,
ROUND(RAND() * 1000, 2),
ROUND(RAND() * 1000, 2),
NOW() - INTERVAL FLOOR(RAND() * 86400 * 1000) SECOND
FROM
ids i1,
ids i2 
WHERE
i1.id > 0 AND
i1.id <= 20000 AND
i2.id < 100
ORDER BY
RAND();

EXPLAIN
SELECT U.Id
    , U.Name
    , SUM(P.Credit) AS CreditAmount
    , SUM(P.Debit) AS DebitAmount
    FROM users U
    LEFT JOIN payments P ON P.UserId = U.Id
    WHERE U.StatusId = 1 /* Active Users Only */
    GROUP BY U.Id;

SELECT U.Id
    , U.Name
    , SUM(P.Credit) AS CreditAmount
    , SUM(P.Debit) AS DebitAmount
    FROM users U
    LEFT JOIN payments P ON P.UserId = U.Id
    WHERE U.StatusId = 1 /* Active Users Only */
    GROUP BY U.Id;

SELECT U.Id
    , U.Name
    , SUM(P.Credit) AS CreditAmount
    , SUM(P.Debit) AS DebitAmount
    FROM users U
    LEFT JOIN payments P ON P.UserId = U.Id
    WHERE U.StatusId = 1 /* Active Users Only */
    GROUP BY U.Id;

SELECT U.Id
    , U.Name
    , SUM(P.Credit) AS CreditAmount
    , SUM(P.Debit) AS DebitAmount
    FROM users U
    LEFT JOIN payments P ON P.UserId = U.Id
    WHERE U.StatusId = 1 /* Active Users Only */
    GROUP BY U.Id;

CREATE INDEX ix_status ON users (StatusId, Id);
CREATE INDEX ix_userid ON payments (UserId, Credit, Debit);

EXPLAIN
SELECT U.Id
    , U.Name
    , SUM(P.Credit) AS CreditAmount
    , SUM(P.Debit) AS DebitAmount
    FROM 
        users U FORCE INDEX (ix_status)
    LEFT JOIN 
        payments P FORCE INDEX (ix_userid) ON P.UserId = U.Id
    WHERE 
        U.StatusId = 1 /* Active Users Only */
    GROUP BY
        U.Id;

SELECT U.Id
    , U.Name
    , SUM(P.Credit) AS CreditAmount
    , SUM(P.Debit) AS DebitAmount
    FROM 
        users U FORCE INDEX (ix_status)
    LEFT JOIN 
        payments P FORCE INDEX (ix_userid) ON P.UserId = U.Id
    WHERE 
        U.StatusId = 1 /* Active Users Only */
    GROUP BY
        U.Id;

SELECT U.Id
    , U.Name
    , SUM(P.Credit) AS CreditAmount
    , SUM(P.Debit) AS DebitAmount
    FROM 
        users U FORCE INDEX (ix_status)
    LEFT JOIN 
        payments P FORCE INDEX (ix_userid) ON P.UserId = U.Id
    WHERE 
        U.StatusId = 1 /* Active Users Only */
    GROUP BY
        U.Id;

SELECT U.Id
    , U.Name
    , SUM(P.Credit) AS CreditAmount
    , SUM(P.Debit) AS DebitAmount
    FROM 
        users U FORCE INDEX (ix_status)
    LEFT JOIN 
        payments P FORCE INDEX (ix_userid) ON P.UserId = U.Id
    WHERE 
        U.StatusId = 1 /* Active Users Only */
    GROUP BY
        U.Id;

这是输出：

[ross@titanv ~]$ mysql -vvv < ex.sql temp
--------------
DROP TABLE IF EXISTS payments
--------------

Query OK, 0 rows affected (0.01 sec)

--------------
DROP TABLE IF EXISTS users
--------------

Query OK, 0 rows affected (0.00 sec)

--------------
DROP TABLE IF EXISTS userStatus
--------------

Query OK, 0 rows affected (0.01 sec)

--------------
CREATE TABLE userStatus (
        Id int(11) NOT NULL auto_increment,
        PRIMARY KEY  (Id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
--------------

Query OK, 0 rows affected (0.01 sec)

--------------
CREATE TABLE users (
        Id int(11) NOT NULL auto_increment,
        StatusId int(11) default NULL,
        Name varchar(60) default NULL,
        TimeZone varchar(100) default NULL,
        PRIMARY KEY  (Id),
        KEY StatusId (StatusId),
        CONSTRAINT FK_User_Status FOREIGN KEY (StatusId) REFERENCES userStatus (Id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
--------------

Query OK, 0 rows affected (0.00 sec)

--------------
CREATE TABLE payments (
        Id int(11) NOT NULL auto_increment,
        UserId int(11) default NULL,  
        Description varchar(200) default NULL,
        Debit decimal(11,2) default NULL,
        Credit decimal(11,2) default NULL,
        Date datetime default NULL,
        PRIMARY KEY  (Id),
        KEY UserId (UserId),
        CONSTRAINT FK_Payment_User FOREIGN KEY (UserId) REFERENCES users (Id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
--------------

Query OK, 0 rows affected (0.01 sec)

--------------
DROP TABLE IF EXISTS ids
--------------

Query OK, 0 rows affected (0.00 sec)

--------------
CREATE TABLE IF NOT EXISTS ids (
        id INT UNSIGNED AUTO_INCREMENT NOT NULL,
        PRIMARY KEY (id)
) ENGINE=MyISAM
--------------

Query OK, 0 rows affected (0.01 sec)

--------------
INSERT INTO ids SET id = NULL
--------------

Query OK, 1 row affected (0.00 sec)

--------------
UPDATE ids SET id=0
--------------

Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

--------------
ALTER TABLE ids AUTO_INCREMENT=1
--------------

Query OK, 1 row affected (0.00 sec)
Records: 1  Duplicates: 0  Warnings: 0

--------------
INSERT INTO ids SELECT NULL FROM ids
--------------

Query OK, 1 row affected (0.00 sec)
Records: 1  Duplicates: 0  Warnings: 0

--------------
INSERT INTO ids SELECT NULL FROM ids
--------------

Query OK, 2 rows affected (0.00 sec)
Records: 2  Duplicates: 0  Warnings: 0

--------------
INSERT INTO ids SELECT NULL FROM ids
--------------

Query OK, 4 rows affected (0.00 sec)
Records: 4  Duplicates: 0  Warnings: 0

--------------
INSERT INTO ids SELECT NULL FROM ids
--------------

Query OK, 8 rows affected (0.00 sec)
Records: 8  Duplicates: 0  Warnings: 0

--------------
INSERT INTO ids SELECT NULL FROM ids
--------------

Query OK, 16 rows affected (0.00 sec)
Records: 16  Duplicates: 0  Warnings: 0

--------------
INSERT INTO ids SELECT NULL FROM ids
--------------

Query OK, 32 rows affected (0.01 sec)
Records: 32  Duplicates: 0  Warnings: 0

--------------
INSERT INTO ids SELECT NULL FROM ids
--------------

Query OK, 64 rows affected (0.00 sec)
Records: 64  Duplicates: 0  Warnings: 0

--------------
INSERT INTO ids SELECT NULL FROM ids
--------------

Query OK, 128 rows affected (0.00 sec)
Records: 128  Duplicates: 0  Warnings: 0

--------------
INSERT INTO ids SELECT NULL FROM ids
--------------

Query OK, 256 rows affected (0.00 sec)
Records: 256  Duplicates: 0  Warnings: 0

--------------
INSERT INTO ids SELECT NULL FROM ids
--------------

Query OK, 512 rows affected (0.00 sec)
Records: 512  Duplicates: 0  Warnings: 0

--------------
INSERT INTO ids SELECT NULL FROM ids
--------------

Query OK, 1024 rows affected (0.00 sec)
Records: 1024  Duplicates: 0  Warnings: 0

--------------
INSERT INTO ids SELECT NULL FROM ids
--------------

Query OK, 2048 rows affected (0.00 sec)
Records: 2048  Duplicates: 0  Warnings: 0

--------------
INSERT INTO ids SELECT NULL FROM ids
--------------

Query OK, 4096 rows affected (0.01 sec)
Records: 4096  Duplicates: 0  Warnings: 0

--------------
INSERT INTO ids SELECT NULL FROM ids
--------------

Query OK, 8192 rows affected (0.01 sec)
Records: 8192  Duplicates: 0  Warnings: 0

--------------
INSERT INTO ids SELECT NULL FROM ids
--------------

Query OK, 16384 rows affected (0.02 sec)
Records: 16384  Duplicates: 0  Warnings: 0

--------------
INSERT INTO 
userStatus
(id)
SELECT
id
FROM
ids
WHERE
id > 0
ORDER BY
id
LIMIT 2
--------------

Query OK, 2 rows affected (0.01 sec)
Records: 2  Duplicates: 0  Warnings: 0

--------------
INSERT INTO 
users
(Id, StatusId)
SELECT
id,
IF(RAND() < .9, 1, 2) 
FROM
ids
WHERE
id > 0 AND
id <= 20000
ORDER BY
id
--------------

Query OK, 20000 rows affected (0.11 sec)
Records: 20000  Duplicates: 0  Warnings: 0

--------------
INSERT INTO
payments
(Id, UserId, Debit, Credit, `Date`)
SELECT
NULL,
i1.id,
ROUND(RAND() * 1000, 2),
ROUND(RAND() * 1000, 2),
NOW() - INTERVAL FLOOR(RAND() * 86400 * 1000) SECOND
FROM
ids i1,
ids i2 
WHERE
i1.id > 0 AND
i1.id <= 20000 AND
i2.id < 100
ORDER BY
RAND()
--------------

Query OK, 2000000 rows affected (21.52 sec)
Records: 2000000  Duplicates: 0  Warnings: 0

--------------
EXPLAIN
SELECT U.Id
    , U.Name
    , SUM(P.Credit) AS CreditAmount
    , SUM(P.Debit) AS DebitAmount
    FROM users U
    LEFT JOIN payments P ON P.UserId = U.Id
    WHERE U.StatusId = 1 
    GROUP BY U.Id
--------------

+----+-------------+-------+------+---------------+----------+---------+-----------+-------+-------------+
| id | select_type | table | type | possible_keys | key      | key_len | ref       | rows  | Extra       |
+----+-------------+-------+------+---------------+----------+---------+-----------+-------+-------------+
|  1 | SIMPLE      | U     | ref  | StatusId      | StatusId | 5       | const     | 10274 | Using where |
|  1 | SIMPLE      | P     | ref  | UserId        | UserId   | 5       | temp.U.Id |     1 |             |
+----+-------------+-------+------+---------------+----------+---------+-----------+-------+-------------+
2 rows in set (0.00 sec)

--------------
SELECT U.Id
    , U.Name
    , SUM(P.Credit) AS CreditAmount
    , SUM(P.Debit) AS DebitAmount
    FROM users U
    LEFT JOIN payments P ON P.UserId = U.Id
    WHERE U.StatusId = 1 
    GROUP BY U.Id
--------------

+-------+------+--------------+-------------+
| Id    | Name | CreditAmount | DebitAmount |
+-------+------+--------------+-------------+
|     1 | NULL |     47824.73 |    49580.71 |
|     2 | NULL |     46426.02 |    52019.69 |
...
| 19999 | NULL |     50041.10 |    47696.88 |
| 20000 | NULL |     51923.69 |    50349.38 |
+-------+------+--------------+-------------+
18057 rows in set (14.60 sec)

--------------
SELECT U.Id
    , U.Name
    , SUM(P.Credit) AS CreditAmount
    , SUM(P.Debit) AS DebitAmount
    FROM users U
    LEFT JOIN payments P ON P.UserId = U.Id
    WHERE U.StatusId = 1 
    GROUP BY U.Id
--------------

+-------+------+--------------+-------------+
| Id    | Name | CreditAmount | DebitAmount |
+-------+------+--------------+-------------+
|     1 | NULL |     47824.73 |    49580.71 |
|     2 | NULL |     46426.02 |    52019.69 |
...
| 19999 | NULL |     50041.10 |    47696.88 |
| 20000 | NULL |     51923.69 |    50349.38 |
+-------+------+--------------+-------------+
18057 rows in set (14.10 sec)

--------------
SELECT U.Id
    , U.Name
    , SUM(P.Credit) AS CreditAmount
    , SUM(P.Debit) AS DebitAmount
    FROM users U
    LEFT JOIN payments P ON P.UserId = U.Id
    WHERE U.StatusId = 1 
    GROUP BY U.Id
--------------

+-------+------+--------------+-------------+
| Id    | Name | CreditAmount | DebitAmount |
+-------+------+--------------+-------------+
|     1 | NULL |     47824.73 |    49580.71 |
|     2 | NULL |     46426.02 |    52019.69 |
...
| 19999 | NULL |     50041.10 |    47696.88 |
| 20000 | NULL |     51923.69 |    50349.38 |
+-------+------+--------------+-------------+
18057 rows in set (14.17 sec)

--------------
CREATE INDEX ix_status ON users (StatusId, Id)
--------------

Query OK, 0 rows affected (0.05 sec)
Records: 0  Duplicates: 0  Warnings: 0

--------------
CREATE INDEX ix_userid ON payments (UserId, Credit, Debit)
--------------

Query OK, 0 rows affected (5.09 sec)
Records: 0  Duplicates: 0  Warnings: 0

--------------
EXPLAIN
SELECT U.Id
    , U.Name
    , SUM(P.Credit) AS CreditAmount
    , SUM(P.Debit) AS DebitAmount
    FROM 
        users U FORCE INDEX (ix_status)
    LEFT JOIN 
        payments P FORCE INDEX (ix_userid) ON P.UserId = U.Id
    WHERE 
        U.StatusId = 1 
    GROUP BY
        U.Id
--------------

+----+-------------+-------+------+---------------+-----------+---------+-----------+-------+-------------+
| id | select_type | table | type | possible_keys | key       | key_len | ref       | rows  | Extra       |
+----+-------------+-------+------+---------------+-----------+---------+-----------+-------+-------------+
|  1 | SIMPLE      | U     | ref  | ix_status     | ix_status | 5       | const     | 10274 | Using where |
|  1 | SIMPLE      | P     | ref  | ix_userid     | ix_userid | 5       | temp.U.Id | 10002 | Using index |
+----+-------------+-------+------+---------------+-----------+---------+-----------+-------+-------------+
2 rows in set (0.00 sec)

--------------
SELECT U.Id
    , U.Name
    , SUM(P.Credit) AS CreditAmount
    , SUM(P.Debit) AS DebitAmount
    FROM 
        users U FORCE INDEX (ix_status)
    LEFT JOIN 
        payments P FORCE INDEX (ix_userid) ON P.UserId = U.Id
    WHERE 
        U.StatusId = 1 
    GROUP BY
        U.Id
--------------

+-------+------+--------------+-------------+
| Id    | Name | CreditAmount | DebitAmount |
+-------+------+--------------+-------------+
|     1 | NULL |     47824.73 |    49580.71 |
|     2 | NULL |     46426.02 |    52019.69 |
...
| 19999 | NULL |     50041.10 |    47696.88 |
| 20000 | NULL |     51923.69 |    50349.38 |
+-------+------+--------------+-------------+
18057 rows in set (0.75 sec)

--------------
SELECT U.Id
    , U.Name
    , SUM(P.Credit) AS CreditAmount
    , SUM(P.Debit) AS DebitAmount
    FROM 
        users U FORCE INDEX (ix_status)
    LEFT JOIN 
        payments P FORCE INDEX (ix_userid) ON P.UserId = U.Id
    WHERE 
        U.StatusId = 1 
    GROUP BY
        U.Id
--------------

+-------+------+--------------+-------------+
| Id    | Name | CreditAmount | DebitAmount |
+-------+------+--------------+-------------+
|     1 | NULL |     47824.73 |    49580.71 |
|     2 | NULL |     46426.02 |    52019.69 |
...
| 19999 | NULL |     50041.10 |    47696.88 |
| 20000 | NULL |     51923.69 |    50349.38 |
+-------+------+--------------+-------------+
18057 rows in set (0.75 sec)

--------------
SELECT U.Id
    , U.Name
    , SUM(P.Credit) AS CreditAmount
    , SUM(P.Debit) AS DebitAmount
    FROM 
        users U FORCE INDEX (ix_status)
    LEFT JOIN 
        payments P FORCE INDEX (ix_userid) ON P.UserId = U.Id
    WHERE 
        U.StatusId = 1 
    GROUP BY
        U.Id
--------------

+-------+------+--------------+-------------+
| Id    | Name | CreditAmount | DebitAmount |
+-------+------+--------------+-------------+
|     1 | NULL |     47824.73 |    49580.71 |
|     2 | NULL |     46426.02 |    52019.69 |
...
| 19999 | NULL |     50041.10 |    47696.88 |
| 20000 | NULL |     51923.69 |    50349.38 |
+-------+------+--------------+-------------+
18057 rows in set (0.75 sec)

Bye

答案 2 :(得分：0)

根据此link，我无法加快SUM功能。这里的主要问题是我选择了大量的记录。

我考虑重新设计users表，添加一个新列，例如TotalPayment，它将保留credit表中debit减去payments的数量。我知道它违反了数据库规范化规则，但我想这次重新设计是为了节省性能。

非常感谢你的帮助。

相关问题
最新问题