Question

我有一个网络应用程序，可以执行一系列缓慢的并发工作来计算其结果。而不是让最终用户挂起，我想通过websocket流回进度更新。

我的代码库由Scalaz eithers（/）的组合构成，如：

type ProcessResult = Error \/ Int

def downloadFile(url: String): Future[Error \/ String] = ???
def doSlowProcessing(data1: String, data2: String): Future[ProcessResult] = ???

/* Very simple however doesn't give any progress update */
def execute(): Future[ProcessResult] = {
 val download1 = downloadFile(...)
 val download2 = downloadFile(...)

 val et = for {
   d1 <- download1
   d2 <- download2
   processed <- doSlowProcessing(d1, d2)
 } yield processed   

 et.run 
}

这非常有效但当然整个计算需要在我从未来获得任何东西之前完成。即使我堆叠在Writer monad上进行日志记录，我也只会在完成后获取日志，而不是让我的最终用户更快乐。

我在玩代码运行时使用scalaz-stream Queue将日志作为副作用玩弄，但最终结果非常难看：

def execute(): Process[Task, String \/ ProcessResult] = {
 val (q, src) = async.queue[String \/ ProcessResult]

 val download1 = downloadFile(...)
 val download2 = downloadFile(...)

 val et = for {
   d1 <- q.enqueue("Downloading 1".left); download1
   d2 <- q.enqueue("Downloading 2".left); download2
   processed <- q.enqueue("Doing processing".left); doSlowProcessing(d1, d2)
 } yield processed    

 et.run.onSuccess {
  x =>
   q.enqueue(x.right)
   q.close
 }

 src
}

感觉应该有一种惯用的方法来实现这一目标？如有必要，可以将我的SIP-14 Scala期货转换为任务。

Answer 1

我认为你不需要使用队列，其中一种方法可以是使用wye使用非确定性合并，即

type Result = ???
val download1: Process[Task,File] = ???
val download2: Process[Task,File] = ???


val result: Process[Task,(File,File)] = (download1 yip download2).once 

val processed: Process[Task, Result] = result.flatMap(doSlowProcessing)

// Run asynchronously, 
processed.runLast.runAsync {
  case Some(r) => .... // result computed
  case None => .... //no result, hence download1,2 were empty.
}

//or run synchronously awaiting the result
processed.runLast.run match {
  case Some(r) => .... // result computed
  case None => .... //no result 
}

//to capture the error information while download use 
val withError: Process[Task,Throwable\/File] = download1.attempt

//or to log and recover to other file download
val withError: Process[Task,File] download1 onFailure { err => Log(err); download3 }

这有道理吗？

另请注意，async.queue自0.5.0以来不赞成使用async.unboundedQueue

使用scalaz-stream作为异步计算的实时Writer

1 个答案: