Question

我是Akka的新手并开发了一个样本Akka WebSocket服务器，它使用BroadcastHub将文件的内容流式传输到客户端（基于Akka docs的样本）。

假设客户端的消耗速度与服务器一样快，我该如何衡量吞吐量（消息/秒）？

// file source
val fileSource = FileIO.fromPath(Paths.get(path)

// Akka file source
val theFileSource = fileSource
  .toMat(BroadcastHub.sink)(Keep.right)
  .run
//Akka kafka file source
lazy val kafkaSourceActorStream = {

val (kafkaSourceActorRef, kafkaSource) = Source.actorRef[String](Int.MaxValue, OverflowStrategy.fail)
  .toMat(BroadcastHub.sink)(Keep.both).run()

Consumer.plainSource(consumerSettings, Subscriptions.topics("perf-test-topic"))
  .runForeach(record => kafkaSourceActorRef ! record.value().toString)
}

def logicFlow: Flow[String, String, NotUsed] = Flow.fromSinkAndSource(Sink.ignore, theFileSource)

val websocketFlow: Flow[Message, Message, Any] = {
  Flow[Message]
    .collect {
      case TextMessage.Strict(msg) => Future.successful(msg)
      case _ => println("ignore streamed message")
    }
    .mapAsync(parallelism = 2)(identity)
    .via(logicFlow)
    .map { msg: String => TextMessage.Strict(msg) }
  }

val fileRoute =
  path("file") {
    handleWebSocketMessages(websocketFlow)
  }
}

def startServer(): Unit = {
  bindingFuture = Http().bindAndHandle(wsRoutes, HOST, PORT)
  log.info(s"Server online at http://localhost:9000/")
}

def stopServer(): Unit = {
  bindingFuture
   .flatMap(_.unbind())
   .onComplete{
    _ => system.terminate()
      log.info("terminated")
  }
}
//ws client
def connectToWebSocket(url: String) = {
 println("Connecting to websocket: " + url)

 val (upgradeResponse, closed) = Http().singleWebSocketRequest(WebSocketRequest(url), websocketFlow)

 val connected = upgradeResponse.flatMap{ upgrade =>

   if(upgrade.response.status == StatusCodes.SwitchingProtocols )
  {
    println("Web socket connection success")
    Future.successful(Done)

  }else {
     println("Web socket connection failed with error: {}", upgrade.response.status)
     throw new RuntimeException(s"Web socket connection failed: ${upgrade.response.status}")
   }
}

connected.onComplete { msg =>
    println(msg)
 }         
}
def websocketFlow: Flow[Message, Message, _] = { 
 Flow.fromSinkAndSource(printFlowRate, Source.maybe)
}

lazy val printFlowRate  =
 Flow[Message]    
  .alsoTo(fileSink("output.txt"))
  .via(flowRate(1.seconds))
  .to(Sink.foreach(rate => println(s"$rate")))

def flowRate(sampleTime: FiniteDuration) =
 Flow[Message]
  .conflateWithSeed(_ ⇒ 1){ case (acc, _) ⇒ acc + 1 }
  .zip(Source.tick(sampleTime, sampleTime, NotUsed))
  .map(_._1.toDouble / sampleTime.toUnit(SECONDS))

def fileSink(file: String): Sink[Message, Future[IOResult]] = {
 Flow[Message]
  .map{
    case TextMessage.Strict(msg) => msg
    case TextMessage.Streamed(stream) => stream.runFold("")(_ + _).flatMap(msg => Future.successful(msg))
  }
  .map(s => ByteString(s + "\n"))
  .toMat(FileIO.toFile(new File(file)))(Keep.right)
}

Answer 1

您可以将吞吐量测量流附加到现有流。这是一个受this answer启发的示例，它打印每秒从上游源发出的整数数量：

df = pd.DataFrame([[1,2,3],[324,34,25], [463,23,43]])
out = df.to_csv(index=False) # unicode string (i'm in python 3)

s3 = boto3.resource('s3')
BUCKET = "MyBucketThatDefExistsAndIHaveAccessTo"
bucket = s3.Bucket(BUCKET)
obj = bucket.Object('TEST1')
obj.put(df.to_csv(index=False)) # loads an empty file "TEST1" to my bucket.

# I've also tried, but same result.
obj.put(bytes(df.to_csv(index=False), 'utf8'))

在下面的示例中，我们将上面的接收器附加到发出1到1000万整数的源。为了防止速率测量流干扰主流（在这种情况下，简单地将每个整数转换为字符串并返回作为具体化值的一部分处理的最后一个字符串），我们使用wireTapMat：< / p>

val rateSink = Flow[Int]
  .conflateWithSeed(_ => 0){ case (acc, _) => acc + 1 }
  .zip(Source.tick(1.second, 1.second, NotUsed))
  .map(_._1)
  .toMat(Sink.foreach(i => println(s"$i elements/second")))(Keep.right)

运行上面的示例会打印如下内容：

val (rateFut, mainFut) = Source(1 to 10000000)
  .wireTapMat(rateSink)(Keep.right)
  .map(_.toString)
  .toMat(Sink.last[String])(Keep.both)
  .run() // (Future[Done], Future[String])

rateFut onComplete {
  case Success(x) => println(s"rateFut completed: $x")
  case Failure(_) =>
}

mainFut onComplete {
  case Success(s) => println(s"mainFut completed: $s")
  case Failure(_) =>
}

如果您不需要引用0 elements/second 2597548 elements/second 3279052 elements/second mainFut completed: 10000000 3516141 elements/second 607254 elements/second rateFut completed: Done的具体化值，请使用rateSink代替wireTap。例如，将wireTapMat附加到WebSocket流可能如下所示：

rateSink

val websocketFlow: Flow[Message, Message, Any] = { Flow[Message] .wireTap(rateSink) // <--- .collect { case TextMessage.Strict(msg) => Future.successful(msg) case _ => println("ignore streamed message") } .mapAsync(parallelism = 2)(identity) .via(logicFlow) .map { msg: String => TextMessage.Strict(msg) } }已在wireTap和Source上定义。

Answer 2

我上次工作的地方实施了这种性能的绩效基准。

基本上，它意味着创建一个简单的客户端应用程序，它使用来自websocket的消息并输出一些指标。自然的选择是使用akka-http客户端支持websockets来实现客户端。参见：

https://doc.akka.io/docs/akka-http/current/client-side/websocket-support.html#singlewebsocketrequest

然后我们使用千分尺库向Prometheus公开指标，这是我们报告和制图的首选工具。

https://github.com/micrometer-metrics

https://micrometer.io/docs/concepts#_meters

如何衡量Akka WebSocket流的吞吐量？

2 个答案: