Question

我正在尝试使用bash并行处理文件集。我正在使用命名管道来保持进程数量并从进程中收集输出。

我假设对命名管道的写入是原子的，即不同进程的输出不会混淆。这是一个安全的假设吗？

非常感谢任何建议。我只能使用bash。

以下是代码：

mytask()
{
  wItem=$1
  #dummy func; process workItem
  rt=$RANDOM
  st=$rt;
  let "rt %= 2"
  let "st %= 10"
  sleep $st
  return $rt
}

parallelizeTask()
{
workList=$1
threadCnt=$2
task=$3
threadSyncPipeD=$4
outputSyncPipeD=$5

ti=0
for workItem in $workList; do
  if [ $ti -lt $threadCnt ]; then
    { $task $workItem; if [ $? == 0 ]; then result="success"; else result="failure"; fi; \
      echo "$result:$workItem" >&$outputSyncPipeD; echo "$result:$workItem" >&$threadSyncPipeD; } &
    ((ti++))
    continue;
  fi
  while read n; do
      ((ti--))
      break;
  done <&$threadSyncPipeD
  { $task $workItem; if [ $? == 0 ]; then result="success"; else result="failure"; fi; \
    echo "$result:$workItem" >&$outputSyncPipeD; echo "$result:$workItem" >&$threadSyncPipeD;} &
  ((i++))
done
wait
echo "quit" >&$outputSyncPipeD

while read n; do
 if [[ $n == "quit" ]]; then
    break;
 else
    eval $6="\${$6}\ \$n"
 fi
 done <&$outputSyncPipeD;
}

main()
{
  if [ ! -p threadSyncPipe ]; then
     mkfifo threadSyncPipe
   fi

   if [ ! -p outputSyncPipe ]; then
      mkfifo outputSyncPipe
   fi

   exec 4<>threadSyncPipe
   exec 3<>outputSyncPipe
   gout=
   parallelizeTask "f1 f2 f3 f4 f5 f6" 2 mytask 3 4 gout

   echo "finalOutput: $gout";
   for f in $gout; do
       echo $f
   done

   rm outputSyncPipe
   rm threadSyncPipe
}

main

我在下面找到相关帖子并回答了我的问题。我修改了标题以使其更合适。

Are there repercussions to having many processes write to a single reader on a named pipe in posix?

Answer 1

我在下面给出的相关帖子中找到了答案，根据它，只要写消息小于页面大小4k（页面大小取决于系统配置），对fifo的写入是原子的。

Are there repercussions to having many processes write to a single reader on a named pipe in posix?

谢谢大家的回复和建议。

bash：对命名管道的写入是原子的吗？

1 个答案: