Question

我的缓存查询缓存了memcached，它很好，但有时我们在mysql中遇到了很多慢查询，好像memcached停止了然后我们的网站停止了。

慢查询图表：http://i.imgur.com/ReyWe.png

此时我们在30秒内完成了大约100次慢速查询。它是什么？我们的询问：

# Query_time: 5.942602  Lock_time: 0.010214 Rows_sent: 10000  Rows_examined: 493139  

SET timestamp=1335194149;  
SELECT story_id FROM dug_stories d WHERE d.story_is_permanent=0  AND  
                       (  
                        (d.story_time>1335190543 AND (d.story_pluses-d.story_minuses) > 5)  
                        OR  
                        ( ( (d.story_pluses-d.story_minuses) > 12 OR (d.story_pluses >= 10 AND d.story_minuses<=0) )  AND d.story_cat!=48 AND d.st  
ory_cat!=131 AND d.story_cat!=55 AND d.story_cat!=44 AND d.story_cat!=126 AND d.story_cat!=53 AND d.story_cat!=370 AND d.story_cat!=381 AND d.stor  
y_cat!=304 AND d.story_cat!=497)  OR (d.story_cat=48 AND (d.story_pluses-d.story_minuses) > 9) OR (d.story_cat=131 AND (d.story_pluses-d.story_min  
uses) > 8) OR (d.story_cat=55 AND (d.story_pluses-d.story_minuses) > 8) OR (d.story_cat=44 AND (d.story_pluses-d.story_minuses) > 9) OR (d.story_c  
at=126 AND (d.story_pluses-d.story_minuses) > 13) OR (d.story_cat=53 AND (d.story_pluses-d.story_minuses) > 8) OR (d.story_cat=370 AND (d.story_pl  
uses-d.story_minuses) > 8) OR (d.story_cat=381 AND (d.story_pluses-d.story_minuses) > 8) OR (d.story_cat=304 AND (d.story_pluses-d.story_minuses)  
> 8) OR (d.story_cat=497 AND (d.story_pluses-d.story_minuses) > 9)  
                       )  
                       ORDER BY d.story_rating DESC, d.story_time DESC LIMIT 0, 10000;

稍微调整后的查询（而不是所有！=类别，我更改为NOT Category IN（要排除的类别列表），但格式化以便于阅读...另外，请注意，SET TIMESTAMP变量不是甚至在查询中使用...它是一个硬编码的值。

SET timestamp=1335194149;  
SELECT 
      story_id 
   FROM 
      dug_stories d 
   WHERE 
          d.story_is_permanent = 0  
      AND (     ( d.story_time > 1335190543 AND d.story_pluses - d.story_minuses > 5 )  
             OR (    (    d.story_pluses - d.story_minuses > 12 
                       OR (   d.story_pluses >= 10 AND d.story_minuses <= 0 ) 
                     )  
                  AND NOT d.story_cat IN ( 44, 48, 53, 55, 126, 131, 304, 370, 381, 497 )
                )
             OR ( d.story_cat = 44  AND d.story_pluses - d.story_minuses > 9 ) 
             OR ( d.story_cat = 48  AND d.story_pluses - d.story_minuses > 9 ) 
             OR ( d.story_cat = 53  AND d.story_pluses - d.story_minuses > 8 ) 
             OR ( d.story_cat = 55  AND d.story_pluses - d.story_minuses > 8 ) 
             OR ( d.story_cat = 126 AND d.story_pluses - d.story_minuses > 13 ) 
             OR ( d.story_cat = 131 AND d.story_pluses - d.story_minuses > 8 ) 
             OR ( d.story_cat = 304 AND d.story_pluses - d.story_minuses > 8 ) 
             OR ( d.story_cat = 370 AND d.story_pluses - d.story_minuses > 8 ) 
             OR ( d.story_cat = 381 AND d.story_pluses - d.story_minuses > 8 ) 
             OR ( d.story_cat = 497 AND d.story_pluses - d.story_minuses > 9 )  
          )  
   ORDER BY 
      d.story_rating DESC, 
      d.story_time DESC 
   LIMIT 
      0, 10000;

非常感谢您的帮助！抱歉我的英语很差:)。

Answer 1

如果rows_examined大于结果集，则慢速日志可能表示索引未被充分利用。

理想情况下，MySQL应该能够为WHERE子句中的所有过滤表达式使用单个索引。由于所有嵌套的AND和OR以及对多个列的引用，您的WHERE子句非常复杂。 MySQL可能无法使用索引来优化此查询，即使您有一个。

通常，这种类型的查询可以通过将其分成几个较小的查询来改进，每个查询都能够使用索引，并使用UNION将结果组合到单个查询中。

例如，如果我在first_name上有索引，在last_name上有索引，则以下查询：

SELECT *
FROM Users
WHERE active = true
  AND ((first_name = 'Marcus') OR (last_name = 'Adams'))

可以这样改进：

SELECT *
FROM Users
WHERE active = true
  AND first_name = 'Marcus'
UNION
SELECT *
FROM Users
WHERE active = true
  AND last_name = 'Adams'

在LIMIT子句中检查的行数多于10000，这也表明ORDER BY子句可能正在使用文件排序，而不是按顺序返回行的查询。这使得MySQL可以完成所有工作，即使您只需要一部分记录。您很可能需要使用覆盖索引来处理所有条件，这可能是不可能的，或者您需要删除ORDER BY子句，或者，如您所示，依靠缓存来节省时间。

查询时间是高负载服务器上的性能测量不佳，因为任何查询在繁忙的服务器上都可能很慢。您应该使用EXPLAIN和SHOW PROFILE来分析您的查询并对其进行改进，以免他们对您的服务器施加太多负担。

允许在服务器上运行太多线程会导致级联效应，从而导致服务器停止运行。如果您使用InnoDB，请确保thread concurrency不是太高。最好让每个人一次等待并服务8个线程（非常快），而不是一次尝试服务100个或无限制的线程（非常慢）。

Answer 2

除了调整查询的可读性之外，我还会尝试重新构建查询我会确保你有一个索引（Story_Is_Permanent，Story_Cat，Story_Time）

我假设你有一个“故事类别”表，其中包含文字描述，以便向用户展示。由于您无论如何都要查询所有类别，但仅基于某些类别，您要么包含或排除...如果包含，则仅基于此类别的PLUSES和MINUSES的净计数。我会将列添加到该类别表中......类似于

DefaultToInclude   integer as 1 or 0
PlusMinusNetCount  integer

然后，加入类别表并简化查询，例如

SELECT 
      story_id 
   FROM 
      dug_stories d
         JOIN YourCategories YC
            on d.story_cat = YC.categoryID
   WHERE 
          d.story_is_permanent = 0  
      AND (     ( d.story_time > 1335190543 AND d.story_pluses - d.story_minuses > 5 )  
             OR (     YC.DefaultToInclude = 0 
                  AND (    d.story_pluses - d.story_minuses > 12 
                       OR (d.story_pluses >= 10 AND d.story_minuses <= 0 ) 
                      )  
                )
             OR (     YC.DefaultToInclude = 1 
                  AND d.story_pluses-d.story_minuses > YC.PlusMinusNetCount )
          )
 ORDER BY 
      d.story_rating DESC, 
      d.story_time DESC 
   LIMIT 
      0, 10000;

<强>的

(     YC.DefaultToInclude = 0 
  AND (    d.story_pluses - d.story_minuses > 12 
        OR (d.story_pluses >= 10 AND d.story_minuses <= 0 ) 
      )  
)

最终会取代

OR (    (    d.story_pluses - d.story_minuses > 12 
          OR (   d.story_pluses >= 10 AND d.story_minuses <= 0 ) 
        )  
    AND d.story_cat!=48 
    AND d.story_cat!=131 
    AND d.story_cat!=55 
    AND d.story_cat!=44 
    AND d.story_cat!=126 
    AND d.story_cat!=53 
    AND d.story_cat!=370 
    AND d.story_cat!=381 
    AND d.story_cat!=304 
    AND d.story_cat!=497 
   )

<强>的

 OR (     YC.DefaultToInclude = 1 
      AND d.story_pluses-d.story_minuses > YC.PlusMinusNetCount )

最终会取代

( OR ( d.story_cat = 44  AND d.story_pluses - d.story_minuses > 9 ) 
  OR ( d.story_cat = 48  AND d.story_pluses - d.story_minuses > 9 ) 
  OR ( d.story_cat = 53  AND d.story_pluses - d.story_minuses > 8 ) 
  OR ( d.story_cat = 55  AND d.story_pluses - d.story_minuses > 8 ) 
  OR ( d.story_cat = 126 AND d.story_pluses - d.story_minuses > 13 ) 
  OR ( d.story_cat = 131 AND d.story_pluses - d.story_minuses > 8 ) 
  OR ( d.story_cat = 304 AND d.story_pluses - d.story_minuses > 8 ) 
  OR ( d.story_cat = 370 AND d.story_pluses - d.story_minuses > 8 ) 
  OR ( d.story_cat = 381 AND d.story_pluses - d.story_minuses > 8 ) 
  OR ( d.story_cat = 497 AND d.story_pluses - d.story_minuses > 9 ) 
)

这样可以灵活地简单地包含/排除具有相应“净”计数的另一个类别，而无需oooopps为您已经很长的查询添加更多OR / AND条件。没有什么，似乎它总是必须通过整个“dug_Stories”表，因为你正在查看一个条件的每个类别和另一个条件的每个其他类别。您的时间戳仅适用于NOT EQUAL类别，但您的其他条件没有任何时间戳限制（除非您的查询中出现错误）。

MySQL慢查询和memcached

2 个答案: