Question

我们的应用程序的一部分提交了数百个BQ查询，以构建汇总的仪表板视图。由于查询数量众多，我们以BATCH优先级提交所有查询。

在过去的24小时内，由于资源过多，我们有很多查询（超过300个）失败。这很奇怪，因为某些失败的查询将结果保存到输出表中是专门为了避免此类问题。我们检查了其他几个项目，并能够在BQ Web UI中正常运行它们。

示例作业ID（无目标表）：job_GPghMyEBvlYFBGBHI5Szv0HMIpMY

查询：

SELECT order_sub.value as value, count(*) as count
FROM
(
  SELECT order_id, drop_ship as value
  FROM (TABLE_QUERY(d34f20170905T212849Zc4abca, 'table_id IN ("order_detail_2018")'))
  WHERE order_date BETWEEN TIMESTAMP("2018-01-01") AND TIMESTAMP("2018-08-31")
) order_sub
JOIN EACH
(
  SELECT customer_id, order_id, sequence
  FROM temp_30_days.query_q6109c34f20180918T151206Ze52003_buyers
) cust_sub
ON order_sub.order_id = cust_sub.order_id
GROUP BY value
ORDER BY value

错误消息：

"status": {
  "state": "DONE",
  "errorResult": {
    "reason": "resourcesExceeded",
    "message": "Resources exceeded during query execution."
  },
  "errors": [
    {
      "reason": "resourcesExceeded",
      "message": "Resources exceeded during query execution."
    }
  ]
}

示例作业ID（带有目标表）：job_2YsCt-cxql5TP0jklb1aRENzINdZ

查询：

SELECT
  customer_id, sequence
FROM
(
  SELECT
    customer_id,
    sequence,
    random,
    MAX(random) OVER (PARTITION BY customer_id) AS max_value
  FROM
  (
    SELECT customer_id, sequence, RAND() as random
    FROM (TABLE_QUERY(d34f20170905T212849Zc4abca, 'table_id IN ("cust_char_2015", "cust_char_2016", "cust_char_2017")'))
    WHERE end_date >= TIMESTAMP("2016-12-31")
    AND effective_date <= TIMESTAMP ("2017-08-31")
    AND (frequency_end = "98")
  )
)
WHERE max_value = random
GROUP EACH BY
  customer_id, sequence

错误消息：

"status": {
  "errorResult": {
    "message": "Resources exceeded during query execution.", 
    "reason": "resourcesExceeded"
  },
  "errors": [
    {
      "message": "Resources exceeded during query execution.",
      "reason": "resourcesExceeded"
    }
  ],
  "state": "DONE"
}

FWIW status.cloud.google.com为BigQuery亮起绿灯，是否还有其他人遇到类似的问题？

BigQuery-BATCH模式查询随机失败，超出资源

0 个答案: