运行之间的查询时间差异太大

时间:2018-11-19 00:14:44

标签: neo4j cypher

我正在尝试neo4j进行我的成绩项目,因此我决定使用LDBC提供的数据源创建一个数据库。 these queries附带了此数据源。

因此,我决定运行那些查询之一并收集其执行时间(我使用cypher-shell命令运行了它们)。所以我收集了这些时间:

    553575, 558724, 556443, 556675, 551304, 555385, 552896

注意:我参加了第一次比赛,因为如post所述,由于冷缓存,第一次似乎是高峰。

一切都很顺利,然后,我决定用新标签创建一些路径,使数据库从1.5GB增长到2.4GB。我决定再次运行期望它们变慢的查询。令我惊讶的是,我看到时间大大缩短了。

    382331, 380566, 405636, 405953, 407277, 391804, 371134, ....

从此post开始。我希望它们能像在同一时间(如第一次运行)一样,并且可能更大,但绝对不会更小。

出于这个问题的目的,我使用的是 AWS计算机,特别是 i3.4xlarge 实例(16个vCPU,122GB RAM和2x 1.9TB NVMe SSD) 。我正在使用Neo4j的当前版本是 3.3.5 ,如上所述,我正在使用cypher-shell

此外,我没有在此计算机上运行其他程序/进程,因此我认为我们可以消除该问题(尽管如此,它应该使它们变慢而不是加快速度)。

谢谢,我只是想问一下是否有人可以在这个问题上给我一些启发:)

编辑:

我所做的更改在:Message节点和:Person的路径之间添加了一些路径。在:Person和:Person之间也有一些,在:TagClass(其他标签)和:Person之间有一些。此外,数据库在:Tag(name)、: Person(name)、: TagClass(name)和:Country(name)上都有索引,但是出于执行查询的目的,删除了这些索引。

原始查询:

PROFILE
MATCH (tag:Tag {name: 'Arnold_Schwarzenegger'})
MATCH (tag)<-[:HAS_TAG]-(message1:Message)-[:HAS_CREATOR]->(person1:Person)
MATCH (tag)<-[:HAS_TAG]-(message2:Message)-[:HAS_CREATOR]->(person1)
OPTIONAL MATCH (message2)<-[:LIKES]-(person2:Person)
OPTIONAL MATCH (person2)<-[:HAS_CREATOR]-(message3:Message)<-[like:LIKES]-(p3:Person)
RETURN
  person1.id,
  count(DISTINCT like) AS authorityScore
ORDER BY
  authorityScore DESC,
  person1.id ASC
LIMIT 100

这些是两次运行的配置文件,原因是空间仅添加了一个计划。我可以向您保证,它们都是完全相同的(即使在数据库命中,扩展路径上,等等):

第一次运行:

| Plan      | Statement   | Version      | Planner | Runtime       | Time   | DbHits | Rows |
| "PROFILE" | "READ_ONLY" | "CYPHER 3.3" | "COST"  | "INTERPRETED" | 556443 | 0      | 100  |

第二:

| Plan      | Statement   | Version      | Planner | Runtime       | Time   | DbHits | Rows |
| "PROFILE" | "READ_ONLY" | "CYPHER 3.3" | "COST"  | "INTERPRETED" | 371134 | 0      | 100  |

计划:

+----------------------+----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| Operator             | Estimated Rows | Rows     | DB Hits   | Cache H/M | Identifiers                                                                                                                   | Other                                                        |
+----------------------+----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| +ProduceResults      |            100 |      100 |         0 |       0/0 | anon[349], anon[355], authorityScore, person1.id                                                                              | 0.0                                                          |
| |                        +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| +Projection          |            100 |      100 |         0 |       0/0 | anon[349], anon[355], authorityScore, person1.id                                                                              | 0.0; {person1.id : , authorityScore : }                      |
| |                        +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| +Top                 |            100 |      100 |         0 |       0/0 | anon[349], anon[355]                                                                                                          | 0.0; 100; anon[355], anon[349]                               |
| |                        +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| +EagerAggregation    |         126593 |     1842 |  49948155 |       0/0 | anon[349], anon[355]                                                                                                          | 0.0; anon[349]                                               |
| |                        +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| +Apply               |    16025884097 | 49948155 |         0 |       0/0 | person2, anon[219], message3, anon[60], tag, anon[136], p3, message1, anon[167], anon[271], message2, anon[91], person1, like | 0.0                                                          |
| |\                       +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| | +Optional          |           7317 | 49948155 |         0 |       0/0 | person2, message3, p3, anon[271], like                                                                                        | 0.0                                                          |
| | |                      +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| | +Filter            |           5037 | 49897917 |  49897917 |       0/0 | person2, message3, p3, anon[271], like                                                                                        | 0.0; p3:Person                                               |
| | |                      +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| | +Expand(All)       |           5037 | 49897917 | 108680609 |       0/0 | person2, message3, p3, anon[271], like                                                                                        | 0.0; (message3)<-[like:LIKES]-(p3)                           |
| | |                      +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| | +Filter            |           7028 | 58782692 |  58782692 |       0/0 | anon[271], message3, person2                                                                                                  | 0.0; message3:Message                                        |
| | |                      +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| | +Expand(All)       |           7028 | 58782692 |  58866898 |       0/0 | anon[271], message3, person2                                                                                                  | 0.0; (person2)<-[anon[271]:HAS_CREATOR]-(message3)           |
| | |                      +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| | +Argument          |           7317 |   130898 |         0 |       0/0 | person2                                                                                                                       | 0.0                                                          |
| |                        +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| +OptionalExpand(All) |           7317 |   130898 |    220191 |       0/0 | person2, anon[219], anon[60], tag, anon[136], message1, anon[167], message2, anon[91], person1                                | 0.0; (message2)<-[anon[219]:LIKES]-(person2); person2:Person |
| |                        +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| +NodeHashJoin        |           7317 |    51779 |         0 |       0/0 | anon[60], tag, anon[136], message1, anon[167], message2, anon[91], person1                                                    | 0.0; tag, person1                                            |
| |\                       +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| | +Filter            |         341165 |     4241 |      4241 |       0/0 | tag, anon[136], anon[167], message2, person1                                                                                  | 0.0; person1:Person                                          |
| | |                      +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| | +Expand(All)       |         341165 |     4241 |      8482 |       0/0 | tag, anon[136], anon[167], message2, person1                                                                                  | 0.0; (message2)-[anon[167]:HAS_CREATOR]->(person1)           |
| | |                      +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| | +Filter            |         341165 |     4241 |      4646 |       0/0 | anon[136], message2, tag                                                                                                      | 0.0; message2:Message                                        |
| | |                      +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| | +Expand(All)       |         372142 |     4646 |      4647 |       0/0 | anon[136], message2, tag                                                                                                      | 0.0; (tag)<-[anon[136]:HAS_TAG]-(message2)                   |
| | |                      +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| | +Filter            |           1608 |        1 |     16080 |       0/0 | tag                                                                                                                           | 0.0; tag.name = {  AUTOSTRING0}                              |
| | |                      +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| | +NodeByLabelScan   |          16080 |    16080 |     16081 |       0/0 | tag                                                                                                                           | 0.0; :Tag                                                    |
| |                        +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| +Filter              |         341165 |     4241 |      4241 |       0/0 | anon[60], tag, message1, anon[91], person1                                                                                    | 0.0; person1:Person                                          |
| |                        +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| +Expand(All)         |         341165 |     4241 |      8482 |       0/0 | anon[60], tag, message1, anon[91], person1                                                                                    | 0.0; (message1)-[anon[91]:HAS_CREATOR]->(person1)            |
| |                        +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| +Filter              |         341165 |     4241 |      4646 |       0/0 | anon[60], message1, tag                                                                                                       | 0.0; message1:Message                                        |
| |                        +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| +Expand(All)         |         372142 |     4646 |      4647 |       0/0 | anon[60], message1, tag                                                                                                       | 0.0; (tag)<-[anon[60]:HAS_TAG]-(message1)                    |
| |                        +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| +Filter              |           1608 |        1 |     16080 |       0/0 | tag                                                                                                                           | 0.0; tag.name = {  AUTOSTRING0}                              |
| |                        +----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+
| +NodeByLabelScan     |          16080 |    16080 |     16081 |       0/0 | tag                                                                                                                           | 0.0; :Tag                                                    |
+----------------------+----------------+----------+-----------+-----------+-------------------------------------------------------------------------------------------------------------------------------+------------------------------    --------------------------------+

... Results ...

100 rows available after 371134 ms, consumed after another 0 ms

注意: SO不允许我写超过30000个字符,因此必须减少文本。

1 个答案:

答案 0 :(得分:0)

Cypher计划器在决定如何执行查询时会考虑数据库的当前特征。对数据库的任何更改都会改变执行计划,这会极大地影响时间。

了解更新数据库后时序差异的最好方法是对查询profile进行前后比较。在不知道您正在使用什么查询的情况下,尤其是在没有前后概要文件信息(均未提供)的情况下,很难不知道为什么更改了时间。

在几乎没有可用信息的情况下,唯一可以做出的猜测是时间已更改,因为数据库特征已以导致计划者更改执行计划的方式发生了改变。