Question

使用DirectPipelineRunner

运行Dataflow时

        DatastoreV1.Query q = DatastoreV1.Query.newBuilder()
        .addKind(DatastoreV1.KindExpression.newBuilder().setName("KIND").build())
        .setFilter(DatastoreHelper.makeFilter(
            DatastoreHelper.makeFilter(
                "date", 
                DatastoreV1.PropertyFilter.Operator.GREATER_THAN_OR_EQUAL, 
                DatastoreHelper.makeValue(date)).build(),
            DatastoreHelper.makeFilter(
                "date", 
                DatastoreV1.PropertyFilter.Operator.LESS_THAN,
                DatastoreHelper.makeValue(next_date)).build()
        ))
        .build()

按预期工作。将作业提交到Dataflow时，出现以下错误：

(4e1bbdfd880a21c1): java.lang.IllegalArgumentException: Query cannot have any inequality filters. at 
com.google.api.services.datastore.client.QuerySplitterImpl.validateFilter(QuerySplitterImpl.java:109) at 
com.google.api.services.datastore.client.QuerySplitterImpl.validateFilter(QuerySplitterImpl.java:105) at 
com.google.api.services.datastore.client.QuerySplitterImpl.validateQuery(QuerySplitterImpl.java:128) at 
com.google.api.services.datastore.client.QuerySplitterImpl.getSplits(QuerySplitterImpl.java:71) at com.google.cloud.dataflow.sdk.io.DatastoreIO$Source.getSplitQueries(DatastoreIO.java:426) at com.google.cloud.dataflow.sdk.io.DatastoreIO$Source.splitIntoBundles(DatastoreIO.java:305) at 
com.google.cloud.dataflow.sdk.runners.dataflow.CustomSources.performSplit(CustomSources.java:305) at 
com.google.cloud.dataflow.sdk.runners.dataflow.CustomSources.performSourceOperation(CustomSources.java:151) at 
com.google.cloud.dataflow.sdk.runners.worker.SourceOperationExecutor.execute(SourceOperationExecutor.java:62) at 
com.google.cloud.dataflow.sdk.runners.worker.DataflowWorker.executeWork(DataflowWorker.java:254) at 
com.google.cloud.dataflow.sdk.runners.worker.DataflowWorker.doWork(DataflowWorker.java:191) at 
com.google.cloud.dataflow.sdk.runners.worker.DataflowWorker.getAndPerformWork(DataflowWorker.java:144) at 
com.google.cloud.dataflow.sdk.runners.worker.DataflowWorkerHarness$WorkerThread.doWork(DataflowWorkerHarness.java:180) at 
com.google.cloud.dataflow.sdk.runners.worker.DataflowWorkerHarness$WorkerThread.call(DataflowWorkerHarness.java:161) at 
com.google.cloud.dataflow.sdk.runners.worker.DataflowWorkerHarness$WorkerThread.call(DataflowWorkerHarness.java:148) at 
java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)

我在本地拥有Dataflow SDK版本：1.3.0。

Answer 1

这是Dataflow SDK for Java 1.4.0及更早版本中的代码问题。我们会将其跟踪为Issue #101 in the GitHub's repository issue tracker。我们会尽快解决这个问题 - 请点击此处进行更新。

我现在想不出任何微不足道的解决方法。对不起！

解决方案是更新到更新版本的Dataflow SDK。

查询不能有任何不等式过滤器

1 个答案: