我用PyOsmium制作了OSM文件读取器,结果发现,如果您在主进程(osm.SimpleHandler.apply_file
)中运行文件解析器,然后在线程中运行另一个解析器,则该进程将冻结,并且该进程将留在记忆中。
这是代码,您可以取消注释读取nc.apply_file(input_file)
的第一行,并查看结果。
尝试注释和取消注释这些apply_file调用,然后查看结果。显然,如果在主进程中调用它,则其他任何apply_file
都将冻结。
我做了一个解决方法,将apply_file放入相同的过程。但是问题的原因是什么?是否有可用的修复程序?
import sys
from multiprocessing import Queue, Process
import osmium as osm
input_file = sys.argv[1] # give it an .osm.pbf file
readq = Queue()
class WayCounter(osm.SimpleHandler):
ways_count = 0
def way(self, w):
self.ways_count += 1
nc = WayCounter()
# ========= running this line blocks the reader process =================
#nc.apply_file(input_file)
def reader():
nc.apply_file(input_file) # works if here
print(f'counted ways: {nc.ways_count}')
class FootPathFinder(osm.SimpleHandler):
def way(self, w):
if 'highway' not in w.tags and w.tags.get('railway') != 'platform':
return
print('working')
tags = {t.k: t.v for t in list(w.tags)}
try:
coords = [(n.lon, n.lat) for n in w.nodes]
except osm.InvalidLocationError:
return
refs = [i.ref for i in w.nodes]
readq.put((tags, coords, refs))
pf = FootPathFinder()
pf.apply_file(input_file, locations=True)
readq.put(None)
reader_proc = Process(target=reader)
reader_proc.start()
while True:
data = readq.get()
if data is None:
break
reader_proc.join()