Question

每30分钟我通过cron更新统计表从PHP脚本运行一个相当复杂的SQL查询。

通常更新有效，大约需要2-3分钟处理大约40.000条记录。有时这个查询运行的时间非常长，长达数小时没有结果。在PHPMyAdmin中查看MySQL进程，然后以50.000秒及以上的时间显示查询。

这导致表格不再需要全天更新。为什么会这样？

查询很复杂，但非特殊：

SELECT `a`.`id`
       AS
       `id`,
       `a`.`debitor`
       AS `debitor`,
       `a`.`wnummer`
       AS `wnummer`,
       `a`.`konto`
       AS `konto`,
       `a`.`datum`
       AS `datum`,
       `a`.`name`
       AS `name`,
       `a`.`name2`
       AS `name2`,
       `a`.`land`
       AS `land`,
       `a`.`plz`
       AS `plz`,
       `a`.`ort`
       AS `ort`,
       `a`.`str`
       AS `str`,
       `a`.`beschichterdatum`
       AS `beschichterdatum`,
       `a`.`werkstattdatum`
       AS `werkstattdatum`,
       `a`.`plandatum`
       AS `plandatum`,
       `a`.`kommision`
       AS `kommision`,
       `a`.`status`
       AS `status`,
       (SELECT `protokoll`.`ts`
        FROM   `protokoll`
        WHERE  ( `protokoll`.`auftrag_id` = `a`.`id` )
               AND ( `protokoll`.`status_neu` = `a`.`status` )
        ORDER  BY `protokoll`.`ts` DESC
        LIMIT  1)
       AS `status_ts`,
       `a`.`tourname`
       AS `tourname`,
       `a`.`anlage`
       AS `anlage`,
       `a`.`user`
       AS `user`,
       `a`.`vertreter`
       AS `vertreter`,
       `a`.`druckmodus`
       AS `druckmodus`,
       (SELECT GROUP_CONCAT(DISTINCT `position`.`maschine` ORDER BY
               `position`.`maschine` ASC SEPARATOR ', ') AS
               `maschine`
        FROM   `position`
        WHERE  ( `position`.`auftrag_id` = `a`.`id` )
               AND ( `position`.`status` = 20 )
               AND ( Length(`position`.`maschine`) > 0 ))
       AS
       `maschine`,
       (SELECT GROUP_CONCAT(DISTINCT `position`.`beschichter` ORDER
               BY
               `position`.`beschichter` ASC SEPARATOR ', ') AS
               `beschichter`
        FROM   `position`
        WHERE  ( `position`.`auftrag_id` = `a`.`id` )
               AND ( `position`.`status` = 50 )
               AND ( Length(`position`.`beschichter`) > 0 ))
       AS
       `beschichter`,
       (SELECT DISTINCT `position`.`rueckstandinfo` AS
                        `rueckstandinfo`
        FROM   `position`
        WHERE  ( `position`.`auftrag_id` = `a`.`id` )
        ORDER  BY IF(( Length(`position`.`rueckstandinfo`) > 0 ), 1,
                  0) DESC
                  ,
                  COUNT(*) DESC
        LIMIT  1)
       AS `rueckstandinfo`,
       (SELECT COUNT(*)
        FROM   `position`
        WHERE  ( `position`.`auftrag_id` = `a`.`id` ))
       AS `menge`,
       (SELECT COUNT(*)
        FROM   `position`
        WHERE  ( `position`.`auftrag_id` = `a`.`id` )
               AND ( `position`.`schrott` = 1 ))
       AS `schrott`,
       (SELECT SUM(`position`.`menge` -
                   `position`.`schrott`)
        FROM   `position`
        WHERE  ( ( `position`.`auftrag_id` = `a`.`id` )
                 AND ( ( CASE
                           WHEN ( `position`.`status` < 41 ) THEN (
                           To_days(`a`.`werkstattdatum`) - To_days(NOW()) )
                           WHEN ( `position`.`status` < 66 ) THEN (
                           To_days(`a`.`beschichterdatum`) - To_days(NOW()) )
                           WHEN ( `position`.`status` < 100 ) THEN (
                           To_days(`a`.`plandatum`) - To_days(NOW()) )
                         END ) < 0 ) ))
       AS `rueckstaendig`,
       ( CASE
           WHEN ( `a`.`status` < 41 ) THEN ( To_days(`a`.`werkstattdatum`) -
                                             To_days(NOW()) )
           WHEN ( `a`.`status` < 66 ) THEN (
           To_days(`a`.`beschichterdatum`) - To_days(
           NOW()) )
           WHEN ( `a`.`status` < 100 ) THEN (
           To_days(`a`.`plandatum`) - To_days(NOW())
                                            )
         END )
       AS `kalendertage`
FROM   `auftrag` `a`

这些表有MyISAM引擎。

EXPLAIN输出：

EXPLAIN output

Schema auftrag：

CREATE TABLE `auftrag`
  (
     `id`                  INT(11) NOT NULL AUTO_INCREMENT COMMENT 'Auftrag',
     `debitor`             VARCHAR(255) DEFAULT NULL COMMENT 'Kunde',
     `wnummer`             VARCHAR(255) DEFAULT NULL COMMENT 'W-Nr.',
     `konto`               VARCHAR(255) DEFAULT NULL COMMENT 'Konto',
     `vertreter`           INT(11) NOT NULL DEFAULT '0' COMMENT 'Vertreter',
     `preisliste`          INT(11) NOT NULL DEFAULT '0' COMMENT 'Preisliste',
     `datum`               DATETIME NOT NULL DEFAULT '0000-00-00 00:00:00'
     COMMENT
     'Auftragseingang',
     `name`                VARCHAR(255) DEFAULT NULL COMMENT 'Name',
     `name2`               VARCHAR(255) DEFAULT NULL COMMENT 'Name 2',
     `land`                VARCHAR(255) DEFAULT NULL COMMENT 'Land',
     `plz`                 VARCHAR(255) DEFAULT NULL COMMENT 'PLZ',
     `ort`                 VARCHAR(255) DEFAULT NULL COMMENT 'Ort',
     `str`                 VARCHAR(255) DEFAULT NULL COMMENT 'Strasse',
     `str2`                VARCHAR(255) DEFAULT NULL COMMENT 'Strasse 2',
     `beschichterdatum`    DATETIME NOT NULL DEFAULT '0000-00-00 00:00:00'
     COMMENT
     'Beschichtung',
     `werkstattdatum`      DATETIME NOT NULL DEFAULT '0000-00-00 00:00:00'
     COMMENT
     'Werkstatt',
     `plandatum`           DATE NOT NULL DEFAULT '0000-00-00' COMMENT 'Versand',
     `kiste`               INT(1) NOT NULL DEFAULT '0' COMMENT
     'Anzahl Schleifkisten',
     `anlage`              DATETIME NOT NULL DEFAULT '0000-00-00 00:00:00'
     COMMENT
     'Auftragsanlage',
     `user`                INT(11) NOT NULL DEFAULT '0' COMMENT 'Benutzer',
     `menge_sofortschrott` INT(11) NOT NULL DEFAULT '0',
     `menge_gesamt`        INT(11) NOT NULL DEFAULT '0' COMMENT
     'Menge Lieferschein',
     `menge_realvomkunden` INT(11) NOT NULL DEFAULT '0' COMMENT 'Menge erhalten'
     ,
     `menge_beschrift`     INT(11) NOT NULL DEFAULT '0' COMMENT
     'Menge Beschriftung',
     `bemerkung`           TEXT COMMENT 'Bemerkung',
     `rueckstandinfo`      VARCHAR(255) DEFAULT NULL COMMENT 'Info Rueckstand',
     `bem_c206`            TEXT,
     `bem_c207`            TEXT,
     `bem_c208`            TEXT,
     `bem_c209`            TEXT,
     `kommision`           VARCHAR(255) DEFAULT NULL COMMENT 'Kommission',
     `status`              INT(11) NOT NULL DEFAULT '0' COMMENT 'Status Auftrag'
     ,
     `belegart`            VARCHAR(255) NOT NULL DEFAULT 'Auftrag' COMMENT
     'Belegart',
     `emailapkd`           VARCHAR(255) DEFAULT NULL COMMENT
     'eMail Arbeitsplan Kunde',
     `emailapvt`           VARCHAR(255) DEFAULT NULL COMMENT
     'eMail Arbeitsplan Vertreter',
     `rech_name1`          VARCHAR(255) DEFAULT NULL COMMENT 'Rechnung Name',
     `rech_name2`          VARCHAR(255) DEFAULT NULL COMMENT 'Rechnung Name 2',
     `rech_land`           VARCHAR(255) DEFAULT NULL COMMENT 'Rechnung Land',
     `rech_plz`            VARCHAR(255) DEFAULT NULL COMMENT 'Rechnung PLZ',
     `rech_ort`            VARCHAR(255) DEFAULT NULL COMMENT 'Rechnung Ort',
     `rech_str`            VARCHAR(255) DEFAULT NULL COMMENT 'Rechnung Strasse',
     `rech_str2`           VARCHAR(255) DEFAULT NULL COMMENT
     'Rechnung Strasse 2',
     `gesendet_an`         TEXT COMMENT 'Arbeitsplan gesendet',
     `angebotmail`         DATETIME DEFAULT NULL COMMENT 'Angebot Intern',
     `requestangebot`      SMALLINT(1) NOT NULL DEFAULT '0' COMMENT
     'Angebot',
     `gutschein`           CHAR(1) NOT NULL DEFAULT 'N' COMMENT 'Gutschein',
     `tourname`            VARCHAR(255) DEFAULT NULL COMMENT 'Tourenname Pickup'
     ,
     `druckmodus`          VARCHAR(255) NOT NULL DEFAULT '5/12/""/DL/""'
     COMMENT
     'Schl,Beschicht,Teillief,DL-ZS',
     `fremdkonto`          VARCHAR(255) DEFAULT NULL COMMENT 'Fremdkonto',
     `versandart`          VARCHAR(255) NOT NULL DEFAULT '9' COMMENT
     'Versandart',
     `c201`                VARCHAR(255) DEFAULT NULL,
     `ansprechpartner`     VARCHAR(255) DEFAULT NULL COMMENT 'AP',
     PRIMARY KEY (`id`),
     KEY `status` (`status`),
     KEY `tourname` (`tourname`),
     KEY `wnummer` (`wnummer`),
     KEY `debitor` (`debitor`),
     KEY `user` (`user`)
  )
ENGINE=myisam
AUTO_INCREMENT=91809
DEFAULT CHARSET=utf8

架构位置：

CREATE TABLE `position`
  (
     `id`             INT(11) NOT NULL AUTO_INCREMENT COMMENT 'Position',
     `auftrag_id`     INT(11) NOT NULL DEFAULT '0' COMMENT 'Auftrag',
     `artikel_id`     VARCHAR(30) NOT NULL COMMENT 'Artikel',
     `menge`          DECIMAL(6, 0) NOT NULL DEFAULT '0' COMMENT 'Menge',
     `listpreis`      DECIMAL(10, 2) NOT NULL DEFAULT '0.00' COMMENT
     'Listenpreis',
     `rabatt`         DECIMAL(6, 2) NOT NULL DEFAULT '0.00' COMMENT
     'Rabatt Prozent',
     `preis`          DECIMAL(13, 2) NOT NULL DEFAULT '0.00' COMMENT 'Preis',
     `wl_rab`         DECIMAL(6, 2) DEFAULT NULL COMMENT 'Rabatt',
     `wl_preis`       DECIMAL(13, 2) DEFAULT NULL COMMENT 'Preis',
     `schrott`        DECIMAL(6, 0) NOT NULL DEFAULT '0' COMMENT 'Schrott',
     `maschine`       VARCHAR(5) DEFAULT NULL COMMENT 'Maschine',
     `beschichter`    VARCHAR(5) DEFAULT NULL COMMENT 'Beschichter',
     `beschichtung`   VARCHAR(30) DEFAULT NULL COMMENT 'Beschichtung',
     `sk_kond`        VARCHAR(2) DEFAULT NULL COMMENT 'Schneidkanten NB/VB',
     `status`         SMALLINT(6) NOT NULL DEFAULT '15' COMMENT 'Status',
     `infotext`       TEXT COMMENT 'Infotext',
     `artikeltext`    TEXT COMMENT 'Artikeltext',
     `schleiftext`    TEXT COMMENT 'Schleifanweisung',
     `name_en`        VARCHAR(255) DEFAULT NULL COMMENT 'Artikeltext EN',
     `name_it`        VARCHAR(255) DEFAULT NULL COMMENT 'Artikeltext IT',
     `name_fr`        VARCHAR(255) DEFAULT NULL COMMENT 'Artikeltext FR',
     `c214`           VARCHAR(200) DEFAULT NULL,
     `enddatum`       DATETIME DEFAULT NULL,
     `reklamation`    CHAR(1) NOT NULL DEFAULT 'N' COMMENT 'Reklamation',
     `werkzeugtyp`    ENUM('Standard', 'LohntNichtInfo', 'Sonderwerkzeug',
     'Kundenwerkzeug') NOT NULL DEFAULT 'Standard' COMMENT 'Werkzeugtyp',
     `durchmesser`    VARCHAR(50) DEFAULT NULL COMMENT 'Durchmesser',
     `rueckstandinfo` VARCHAR(255) DEFAULT NULL COMMENT 'Rueckstandinfo',
     PRIMARY KEY (`id`),
     KEY `status` (`status`),
     KEY `auftrag_id` (`auftrag_id`),
     KEY `artikel_id` (`artikel_id`),
     KEY `maschine` (`maschine`),
     KEY `beschichter` (`beschichter`),
     KEY `rueckstandinfo` (`rueckstandinfo`),
     KEY `enddatum` (`enddatum`)
  )
ENGINE=myisam
AUTO_INCREMENT=2518917
DEFAULT CHARSET=utf8

架构protokoll：

CREATE TABLE `protokoll`
  (
     `id`         INT(11) NOT NULL AUTO_INCREMENT COMMENT 'ID',
     `usr_id`     INT(11) NOT NULL COMMENT 'Benutzer',
     `auftrag_id` INT(11) NOT NULL COMMENT 'Auftrag',
     `pos_id`     INT(11) NOT NULL COMMENT 'Position',
     `artikel_id` VARCHAR(255) NOT NULL COMMENT 'Artikel',
     `status_alt` SMALLINT(6) DEFAULT NULL COMMENT 'Status ALT',
     `status_neu` SMALLINT(6) DEFAULT NULL COMMENT 'status NEU',
     `info`       VARCHAR(255) DEFAULT NULL COMMENT 'Infotext',
     `ts`         DATETIME NOT NULL COMMENT 'Zeitstempel',
     PRIMARY KEY (`id`),
     KEY `ts` (`ts`),
     KEY `auftrag_id_2` (`auftrag_id`, `artikel_id`, `status_neu`)
  )
ENGINE=myisam
AUTO_INCREMENT=361183
DEFAULT CHARSET=utf8

Answer 1

这是我要尝试做的...根据提取子查询，我能够将3个元素包装到单个查询/连接结果中。无法申请处理您的限制1个实例，因此将它们保留为内联选择。

SELECT 
      a.id,
      a.debitor,
      a.wnummer,
      a.konto,
      a.datum,
      a.`name`,
      a.name2,
      a.land,
      a.plz,
      a.ort,
      a.`str`,
      a.beschichterdatum,
      a.werkstattdatum,
      a.plandatum,
      a.kommision,
      a.`status`,
      ( SELECT protokoll.ts
           FROM   protokoll
           WHERE  ( protokoll.auftrag_id = a.id )
              AND ( protokoll.status_neu = a.`status` )
           ORDER  BY protokoll.ts DESC
           LIMIT  1) AS status_ts,
      a.tourname,
      a.anlage,
      a.`user`,
      a.vertreter,
      a.druckmodus,
      JoinMaschine.maschine,
      JoinBeschichter.beschichter,
      MengeSchrottRueck.menge,
      MengeSchrottRueck.schrott,
      MengeSchrottRueck.rueckstaendig,

      ( SELECT DISTINCT `position`.rueckstandinfo AS rueckstandinfo
           FROM   `position`
           WHERE  ( `position`.auftrag_id = a.id )
           ORDER BY 
              IF(( Length(`position`.rueckstandinfo) > 0 ), 1, 0) DESC,
              COUNT(*) DESC
           LIMIT  1 ) AS rueckstandinfo,
       ( CASE WHEN ( a.`status` < 41 ) 
                 THEN ( To_days(a.werkstattdatum) - To_days(NOW()) )
              WHEN ( a.`status` < 66 ) 
                 THEN ( To_days(a.beschichterdatum) - To_days( NOW()) )
              WHEN ( a.`status` < 100 ) 
                 THEN (To_days(a.plandatum) - To_days(NOW()) )
         END ) AS kalendertage
FROM   
   auftrag a
      JOIN ( SELECT p.auftrag_id,
                    GROUP_CONCAT(DISTINCT p.maschine 
                                 ORDER BY p.maschine ASC SEPARATOR ', ') AS maschine
               from `position` p
               WHERE p.`status` = 20
                 AND Length(p.maschine) > 0 
               group by p.auftrag_id ) as JoinMaschine
         ON a.ID = JoinMaschine.auftrag_id

      JOIN ( SELECT p.auftrag_id,
                    GROUP_CONCAT(DISTINCT p.beschichter 
                                 ORDER BY p.beschichter ASC SEPARATOR ', ') AS beschichter
                FROM `position` p
               WHERE  ( `position`.auftrag_id = a.id )
                 AND ( `position`.`status` = 50 )
                 AND ( Length(`position`.beschichter) > 0 )
               GROUP BY p.auftrag_id ) AS JoinBeschichter
         ON a.id = JoinBeschichter.auftrag_id

      JOIN ( SELECT p.auftrag_id,
                    COUNT(*) as Menge,
                    SUM( IF( p.schrott = 1, 1, 0 )) as schrott,
                    SUM(p.menge - p.schrott * 
                          IF( ( p.`status` < 41 AND To_days(a.werkstattdatum) - To_days(NOW()) < 0)
                           OR ( p.`status` < 66 AND To_days(a.beschichterdatum) - To_days(NOW()) < 0 )
                           OR ( p.`status` < 100 AND To_days(a.plandatum) - To_days(NOW()) < 0 ), 1, 0 )
                       ) AS rueckstaendig
                FROM `position` p
                group by p.auftrag_id ) as MengeSchrottRueck
         ON a.id = MengeSchrottRueck.auftrag_id

Answer 2

您可以尝试EXPLAIN your_query查看有关查询的更多详细信息。

但是我肯定会尽可能用JOIN和GROUP BY替换select部分中的子查询 - 必须为每一行计算select中的子查询。例如，Menge和Schrott专栏似乎很容易替代我。

Answer 3

如果您的查询在“大部分时间”工作，那么一些外部力量正在影响它。你有什么“其他”的cron工作？查询运行时是否正在运行其他数据库查询？您的插入/查询是否包含在事务中？

所以我的建议：

在PHP脚本中使用事务... PHP + MySQL transactions examples

尝试其中一种非默认锁定模式。 MySQL的默认值是REPEATABLE READ。如果脏读是好的，请尝试READ UNCOMMITTED，或者如果这是不可接受的，请尝试更多的ACID'y READ COMMITTED。 http://www.itecsoftware.com/with-nolock-table-hint-equivalent-for-mysql

Answer 4

我建议您使用解释中显示的所有索引。有时mysql可以认为没有索引更好，结果你得到更长的时间。

复杂的MySQL查询偶尔会运行不终止

4 个答案: