Question

我在cygwin中从github运行这个bash代码。它以每秒1条记录的速度运行。它太慢了，如果我有10000条记录，我需要等待很多。我可以做任何事来加快速度。

来源：https://github.com/lgpasquale/qchextractor

#!/bin/bash

for i in "$@"
do
case $i in
    -h|--help)
        echo "Extract the html documentation contained in a *.qch"
        echo "Usage:"
        echo "    qchextractor.sh <input-qch-file> <output-directory>"
        echo "Options:"
        echo "  -h, --help                  print this help"
        exit 0
    ;;
esac
done

if [ "$#" -ne 2 ]; then
    echo "Illegal number of parameters"
    exit 1
fi
if [ ! -f $1 ]; then
    echo "invalid input file: $1"
    exit 1
fi

output_dir="$2/html"
mkdir -p ${output_dir} || { echo "unable to create output dir $2"; exit 1; }

numfiles=$(sqlite3 "$1" "SELECT Count(*) FROM FileNameTable;")
echo "numero file: ${numfiles}"

maxrow=$(( ${numfiles} - 1 ))
echo -n -e "Extracting files:"
for row in $(seq 1 ${maxrow})
do
    echo -n -e "\rExtracting files: ${row}/${maxrow}"
    file_id=$(sqlite3 "$1" "SELECT FileId FROM FileNameTable LIMIT 1 OFFSET ${row};")
    file_name=$(sqlite3 "$1" "SELECT Name FROM FileNameTable WHERE FileId==${file_id};")
    dir_name=$(dirname "${file_name}")
    mkdir -p "${output_dir}/${dir_name}"
    sqlite3 "$1" "SELECT quote(Data) FROM FileDataTable WHERE Id==${file_id};" \
    | cut -d\' -f2 \
    | sed 's/^........//' \
    | xxd -r -p \
    | ./zpipe -d \
    > "${output_dir}/${file_name}"
#    | zlib-flate -uncompress \
done
echo -n -e "\n"

Answer 1

不要在循环内执行查询。执行单个SQL查询，并将其输出通过管道传递给读取结果的循环。

sqlite3 "$1" "SELECT Name, quote(Data) FROM FileDataTable LIMIT $maxrow" | 
    while read file_name data
    do
        dir_name=$(dirname "$file_name")
        mkdir -p "$output_dir/$dir_name"
        echo "$data" \
            | cut -d\' -f2 \
            | sed 's/^........//' \
            | xxd -r -p \
            | ./zpipe -d \
            > "${output_dir}/${file_name}"
    done

您甚至可能不需要LIMIT $maxrow。看起来您将其设置为表中的行数，并且只需要它，因为循环每次都选择一个特定的行。如果要处理整个表，只需将该子句保留。然后您不需要设置$numfile。

的查询

如何加速这个shell脚本

1 个答案: