我有一个Shell脚本,可以逐个下载网址并检查静态网站中的更新。以下是代码:
#!/bin/bash
input="file.in"
while IFS= read -r line
do
# check if input line is a comment
case "$line" in \#*) continue ;; esac
line2="${line//\//\\}"
if wget -q "$line" -O index.html > /dev/null; then
if [ -f "$line2.html" ]; then
cmp --silent "$line2".html index.html || echo "$line"
else
echo "$line INIT"
fi
else
echo "$line FAIL"
fi
mv index.html "$line2".html
done <"$input"
file.in 是带有网址的列表。示例:
#List of addresses
http://www.google.com
http://www.spotify.com
http://www.flickr.com
http://www.soundcloud.com
https://www.facebook.com
https://en.wikipedia.org/wiki/Linux
我想更改脚本以一次下载所有URL,并使用wget或curl以相同的方式保存它们。谢谢!
答案 0 :(得分:0)
幼稚的方法:
while IFS= read -r line
do
[[ "$line" == \#* ]] && continue
(
line2="${line//\//\\}"
if wget -q "$line" -O index.html > /dev/null; then
if [ -f "$line2.html" ]; then
cmp --silent "$line2".html index.html || echo "$line"
else
echo "$line INIT"
fi
else
echo "$line FAIL"
fi
mv index.html "$line2".html
) &
done <"$input"
wait
echo "all done"