为什么这个WHERE子句使我的查询慢180倍?

时间:2012-11-15 20:30:48

标签: mysql

以下查询在1.6秒内执行

SET @num :=0, @current_shop_id := NULL, @current_product_id := NULL;

#this query limits the results of the query within it by row number (so that only 250 products get displayed per store)

SELECT * FROM (

#this query adds row numbers to the query within it

SELECT *, @num := IF( @current_shop_id = shop_id, IF(@current_product_id=product_id,@num,@num+1), 0) AS row_number, @current_shop_id := shop_id AS shop_dummy, @current_product_id := product_id AS product_dummy FROM (

SELECT shop, shops.shop_id AS
shop_id, p1.product_id AS
product_id 
    FROM products p1 LEFT JOIN #this LEFT JOIN gets the favorites count for each product
  (
  SELECT fav3.product_id AS product_id, SUM(CASE 
    WHEN fav3.current = 1 AND fav3.closeted = 1 THEN 1
    WHEN fav3.current = 1 AND fav3.closeted = 0 THEN -1
    ELSE 0
    END) AS favorites_count
    FROM favorites fav3
GROUP BY fav3.product_id 

  ) AS fav4 ON p1.product_id=fav4.product_id
    INNER JOIN sex ON sex.product_id=p1.product_id AND
    sex.sex=0 AND
    sex.date >= SUBDATE(NOW(),INTERVAL 1 DAY) 
    INNER JOIN shops ON shops.shop_id = p1.shop_id
    ORDER BY shop, sex.DATE, product_id
    ) AS testtable

) AS rowed_results WHERE
rowed_results.row_number>=0 AND
rowed_results.row_number<(7)

AND shops.shop_id=86添加到最终的WHERE子句会导致查询在292秒内执行:

SET @num :=0, @current_shop_id := NULL, @current_product_id := NULL;

#this query limits the results of the query within it by row number (so that only 250 products get displayed per store)

SELECT * FROM (

#this query adds row numbers to the query within it

SELECT *, @num := IF( @current_shop_id = shop_id, IF(@current_product_id=product_id,@num,@num+1), 0) AS row_number, @current_shop_id := shop_id AS shop_dummy, @current_product_id := product_id AS product_dummy FROM (

SELECT shop, shops.shop_id AS
shop_id, p1.product_id AS
product_id 
    FROM products p1 LEFT JOIN #this LEFT JOIN gets the favorites count for each product
  (
  SELECT fav3.product_id AS product_id, SUM(CASE 
    WHEN fav3.current = 1 AND fav3.closeted = 1 THEN 1
    WHEN fav3.current = 1 AND fav3.closeted = 0 THEN -1
    ELSE 0
    END) AS favorites_count
    FROM favorites fav3
GROUP BY fav3.product_id 

  ) AS fav4 ON p1.product_id=fav4.product_id
    INNER JOIN sex ON sex.product_id=p1.product_id AND
    sex.sex=0 AND
    sex.date >= SUBDATE(NOW(),INTERVAL 1 DAY)
    INNER JOIN shops ON shops.shop_id = p1.shop_id AND
    shops.shop_id=86
    ORDER BY shop, sex.DATE, product_id
    ) AS testtable

) AS rowed_results WHERE
rowed_results.row_number>=0 AND
rowed_results.row_number<(7)

我认为用AND shops.shop_id=86限制商店表可以减少执行时间。相反,执行时间似乎取决于products表中的行数,products.shop_id等于指定的shops.shop_id。 products表中有大约34K行,products.shop_id = 86,执行时间为292秒。对于products.shop_id = 50,大约有28K行,执行时间是210秒。对于products.shop_id = 175,大约有2K行,执行时间是2.8秒。发生了什么事?

1.6秒查询的EXPLAIN EXTENDED为:

id  select_type table   type    possible_keys   key key_len ref rows    filtered    Extra
1   PRIMARY <derived2>  ALL NULL    NULL    NULL    NULL    1203    100.00  Using where
2   DERIVED <derived3>  ALL NULL    NULL    NULL    NULL    1203    100.00  
3   DERIVED sex ALL product_id_2,product_id NULL    NULL    NULL    526846  75.00   Using where; Using temporary; Using filesort
3   DERIVED p1  eq_ref  PRIMARY,shop_id,shop_id_2,product_id,shop_id_3  PRIMARY 4   mydatabase.sex.product_id   1   100.00  
3   DERIVED <derived4>  ALL NULL    NULL    NULL    NULL    14752   100.00  
3   DERIVED shops   eq_ref  PRIMARY PRIMARY 4   mydatabase.p1.shop_id   1   100.00  
4   DERIVED fav3    ALL NULL    NULL    NULL    NULL    15356   100.00  Using temporary; Using filesort

此EXPLAIN EXTENDED的显示警告

-----+
| Note | 1003 | select `rowed_results`.`shop` AS `shop`,`rowed_results`.`shop_id` AS `shop_id`,`rowed_results`.`product_id` AS `product_id`,`rowed_results`.`row_number` AS `row_number`,`rowed_results`.`shop_dummy` AS `shop_dummy`,`rowed_results`.`product_dummy` AS `product_dummy` from (select `testtable`.`shop` AS `shop`,`testtable`.`shop_id` AS `shop_id`,`testtable`.`product_id` AS `product_id`,(@num:=if(((@current_shop_id) = `testtable`.`shop_id`),if(((@current_product_id) = `testtable`.`product_id`),(@num),((@num) + 1)),0)) AS `row_number`,(@current_shop_id:=`testtable`.`shop_id`) AS `shop_dummy`,(@current_product_id:=`testtable`.`product_id`) AS `product_dummy` from (select `mydatabase`.`shops`.`shop` AS `shop`,`mydatabase`.`shops`.`shop_id` AS `shop_id`,`mydatabase`.`p1`.`product_id` AS `product_id` from `mydatabase`.`products` `p1` left join (select `mydatabase`.`fav3`.`product_id` AS `product_id`,sum((case when ((`mydatabase`.`fav3`.`current` = 1) and (`mydatabase`.`fav3`.`closeted` = 1)) then 1 when ((`mydatabase`.`fav3`.`current` = 1) and (`mydatabase`.`fav3`.`closeted` = 0)) then -(1) else 0 end)) AS `favorites_count` from `mydatabase`.`favorites` `fav3` group by `mydatabase`.`fav3`.`product_id`) `fav4` on(((`mydatabase`.`p1`.`product_id` = `mydatabase`.`sex`.`product_id`) and (`fav4`.`product_id` = `mydatabase`.`sex`.`product_id`))) join `mydatabase`.`sex` join `mydatabase`.`shops` where ((`mydatabase`.`sex`.`sex` = 0) and (`mydatabase`.`p1`.`product_id` = `mydatabase`.`sex`.`product_id`) and (`mydatabase`.`shops`.`shop_id` = `mydatabase`.`p1`.`shop_id`) and (`mydatabase`.`sex`.`date` >= (now() - interval 1 day))) order by `mydatabase`.`shops`.`shop`,`mydatabase`.`sex`.`date`,`mydatabase`.`p1`.`product_id`) `testtable`) `rowed_results` where ((`rowed_results`.`row_number` >= 0) and (`rowed_results`.`row_number` < 7)) |
+------
对于292秒查询,

EXPLAIN EXTENDED为:

id  select_type table   type    possible_keys   key key_len ref rows    filtered    Extra
1   PRIMARY <derived2>  ALL NULL    NULL    NULL    NULL    36  100.00  Using where
2   DERIVED <derived3>  ALL NULL    NULL    NULL    NULL    36  100.00  
3   DERIVED shops   const   PRIMARY PRIMARY 4       1   100.00  Using temporary; Using filesort
3   DERIVED p1  ref PRIMARY,shop_id,shop_id_2,product_id,shop_id_3  shop_id 4       11799   100.00  
3   DERIVED <derived4>  ALL NULL    NULL    NULL    NULL    14752   100.00  
3   DERIVED sex eq_ref  product_id_2,product_id product_id_2    5   mydatabase.p1.product_id    1   100.00  Using where
4   DERIVED fav3    ALL NULL    NULL    NULL    NULL    15356   100.00  Using temporary; Using filesort

此EXPLAIN EXTENDED的显示警告

----+ 
| Note | 1003 | select `rowed_results`.`shop` AS `shop`,`rowed_results`.`shop_id` AS `shop_id`,`rowed_results`.`product_id` AS `product_id`,`rowed_results`.`row_number` AS `row_number`,`rowed_results`.`shop_dummy` AS `shop_dummy`,`rowed_results`.`product_dummy` AS `product_dummy` from (select `testtable`.`shop` AS `shop`,`testtable`.`shop_id` AS `shop_id`,`testtable`.`product_id` AS `product_id`,(@num:=if(((@current_shop_id) = `testtable`.`shop_id`),if(((@current_product_id) = `testtable`.`product_id`),(@num),((@num) + 1)),0)) AS `row_number`,(@current_shop_id:=`testtable`.`shop_id`) AS `shop_dummy`,(@current_product_id:=`testtable`.`product_id`) AS `product_dummy` from (select 'shop.nordstrom.com' AS `shop`,'86' AS `shop_id`,`mydatabase`.`p1`.`product_id` AS `product_id` from `mydatabase`.`products` `p1` left join (select `mydatabase`.`fav3`.`product_id` AS `product_id`,sum((case when ((`mydatabase`.`fav3`.`current` = 1) and (`mydatabase`.`fav3`.`closeted` = 1)) then 1 when ((`mydatabase`.`fav3`.`current` = 1) and (`mydatabase`.`fav3`.`closeted` = 0)) then -(1) else 0 end)) AS `favorites_count` from `mydatabase`.`favorites` `fav3` group by `mydatabase`.`fav3`.`product_id`) `fav4` on(((`fav4`.`product_id` = `mydatabase`.`p1`.`product_id`) and (`mydatabase`.`sex`.`product_id` = `mydatabase`.`p1`.`product_id`))) join `mydatabase`.`sex` join `mydatabase`.`shops` where ((`mydatabase`.`sex`.`sex` = 0) and (`mydatabase`.`sex`.`product_id` = `mydatabase`.`p1`.`product_id`) and (`mydatabase`.`p1`.`shop_id` = 86) and (`mydatabase`.`sex`.`date` >= (now() - interval 1 day))) order by 'shop.nordstrom.com',`mydatabase`.`sex`.`date`,`mydatabase`.`p1`.`product_id`) `testtable`) `rowed_results` where ((`rowed_results`.`row_number` >= 0) and (`rowed_results`.`row_number` < 7)) | 
+-----

我正在运行MySQL客户端版本:5.1.56。商店表在shop_id上有一个主要索引:

Action  Keyname Type    Unique  Packed  Column  Cardinality Collation   Null    Comment
 Edit    Drop   PRIMARY BTREE   Yes No  shop_id 163 A

我已经分析了商店表,但这没有帮助。

我注意到如果删除LEFT JOIN,执行时间的差异会下降到0.12秒而不是0.28秒。

Cez的解决方案,即使用1.6秒版本的查询并通过将rowed_results.shop_dummy=86添加到外部查询(如下所示)来删除不相关的结果,在1.7秒内执行。这避免了这个问题,但神秘仍然是为什么292秒的查询是如此缓慢。

SET @num :=0, @current_shop_id := NULL, @current_product_id := NULL;

#this query limits the results of the query within it by row number (so that only 250 products get displayed per store)

SELECT * FROM (

#this query adds row numbers to the query within it

SELECT *, @num := IF( @current_shop_id = shop_id, IF(@current_product_id=product_id,@num,@num+1), 0) AS row_number, @current_shop_id := shop_id AS shop_dummy, @current_product_id := product_id AS product_dummy FROM (

SELECT shop, shops.shop_id AS
shop_id, p1.product_id AS
product_id 
    FROM products p1 LEFT JOIN #this LEFT JOIN gets the favorites count for each product
  (
  SELECT fav3.product_id AS product_id, SUM(CASE 
    WHEN fav3.current = 1 AND fav3.closeted = 1 THEN 1
    WHEN fav3.current = 1 AND fav3.closeted = 0 THEN -1
    ELSE 0
    END) AS favorites_count
    FROM favorites fav3
GROUP BY fav3.product_id 

  ) AS fav4 ON p1.product_id=fav4.product_id
    INNER JOIN sex ON sex.product_id=p1.product_id AND sex.sex=0
    INNER JOIN shops ON shops.shop_id = p1.shop_id
    WHERE sex.date >= SUBDATE(NOW(),INTERVAL 1 DAY) 


    ORDER BY shop, sex.DATE, product_id
    ) AS testtable

) AS rowed_results WHERE
rowed_results.row_number>=0 AND
rowed_results.row_number<(7) AND
rowed_results.shop_dummy=86;

3 个答案:

答案 0 :(得分:1)

根据讨论判断,在指定较低级别的商店时,查询计划程序表现不佳。

rowed_results.shop_dummy=86添加到外部查询以获取您要查找的结果。

答案 1 :(得分:1)

在聊天室之后,实际创建表/列以匹配查询,我提出了以下查询。

我已经开始了我最内心的查询,包括性,产品(对于shop_id)和收藏夹表。由于您在ShopA = Product ID = 1时描述了ProductX,但在ShopB =产品ID = 2(仅示例)时相同的ProductX,每个产品始终是唯一的,并且从不重复。也就是说,我可以在此查询中获取产品和shop_id以及收藏夹数量(如果有的话),但只对product_id进行分组..因为shop_id不会更改我使用MAX()的每个产品。因为你总是以“昨天”和性别(性别= 0女性)的日期来查看,所以我会在(日期,性别,product_id)索引SEX表...我猜你不是每个都添加1000个项目day ...产品显然会在product_id(主键)上有一个索引,而收藏夹应该在product_id上有一个索引。

根据该结果(别名“sxFav”),我们可以通过“Product_ID”直接加入性和产品表,以获取您可能需要的任何其他信息,例如商店名称,添加的日期产品,产品然后,这个结果由shop_id按产品销售,日期和最终产品ID排序(但您可以考虑在内部查询中抓取描述列并将其用作排序)。这导致别名“PreQuery”。

由于商店的订单完全合适,我们现在可以添加@MySQLVariable引用,以便为每个产品分配一个类似于您最初尝试的行号。但是,只有在商店ID发生变化时才重置为1。

SELECT 
      PreQuery.*,
      @num := IF( @current_shop_id = PreQuery.shop_id, @num +1, 1 ) AS RowPerShop, 
      @current_shop_id := PreQuery.shop_id AS shop_dummy 
   from 
      ( SELECT 
              sxFav.product_id, 
              sxFav.shop_id, 
              sxFav.Favorites_Count
           from 
              ( SELECT 
                      sex.product_id,
                      MAX( p.shop_id ) shop_id,
                      SUM( CASE WHEN F.current = 1 AND F.closeted = 1 THEN 1 
                                WHEN F.current = 1 AND F.closeted = 0 THEN -1 
                                ELSE 0 END ) AS favorites_count 
                   from 
                      sex
                         JOIN products p
                            ON sex.Product_ID = p.Product_ID
                         LEFT JOIN Favorites F 
                            ON sex.product_id = F.product_ID 
                   where 
                          sex.date >= subdate( now(), interval 1 day) 
                      and sex.sex = 0 
                   group by 
                      sex.product_id ) sxFav 

              JOIN sex 
                 ON sxFav.Product_ID = sex.Product_ID

              JOIN products p
                 ON sxFav.Product_ID = p.Product_ID
      order by 
         sxFav.shop_id, 
         sex.date, 
         sxFav.product_id ) PreQuery,

     ( select @num :=0, 
              @current_shop_id := 0 ) as SQLVars 

现在,如果您正在寻找特定的“分页”信息(例如每个商店7个条目),请将上面的整个查询包装成类似......

select * from ( entire query above ) where RowPerShop between 1 and 7

(或根据需要在8到14,15和21之间等) 甚至

RowPerShop between RowsPerPage*PageYouAreShowing and RowsPerPage*(PageYouAreShowing +1)

答案 2 :(得分:1)

您应该将shops.shop_id = 86移动到商店的JOIN条件。没有理由把它放在JOIN之外,你首先冒着MySQL JOIN的风险,然后过滤。 JOIN可以执行WHERE子句所做的相同工作,特别是如果您没有引用其他表。

....
INNER JOIN shops ON shops.shop_id = p1.shop_id AND shops.shop_id=86
....

性别加入也是如此:

...
INNER JOIN shops ON shops.shop_id = p1.shop_id
AND sex.date >= SUBDATE(NOW(),INTERVAL 1 DAY)
...

派生表很棒,但它们没有索引。通常这没关系,因为它们通常在RAM中。但是在没有索引的过滤和排序之间,事情可能会加起来。

请注意,在需要更长时间的第二个查询中,表处理顺序会更改。商店表位于慢查询的顶部,p1表在快速查询中检索11799行而不是1行。它也不再使用主键。这可能就是你的问题所在。

3   DERIVED p1  eq_ref  PRIMARY,shop_id,shop_id_2,product_id,shop_id_3  PRIMARY 4   mydatabase.sex.product_id   1   100.00  

3   DERIVED p1  ref PRIMARY,shop_id,shop_id_2,product_id,shop_id_3  shop_id 4       11799   100.00