我是Python的新手,但仍然不了解它的全部功能和功能,但是我已经接近要达到的目标。
从本质上讲,我已经有了从网站上抓取我想要的数据的程序,但是当它从“ specs”字符串中打印选定的单词/项目时,它也从字符串中打印了[]和”之类的字符。
该示例是我试图从li的列表中获取“齿轮箱”类型,“燃料”类型和“里程”,我已将它们转换为带有植物的字符串,然后从中选择特定项目串。
我在当前程序中得到的是:
['Manual'] ['Petrol'] ['86,863 miles']
我想要实现的是这样的打印结果:
手动,汽油,86,863英里
在导出到.csv中单独的列时,应在相应标题下的正确列中显示。
我尝试.text删除仅文本,但是显示出来的'list'对象没有属性'text'错误。
import csv
import requests
from bs4 import BeautifulSoup
outfile = open('pistonheads.csv','w', newline='')
writer = csv.writer(outfile)
writer.writerow(["Link", "Make", "Model", "Price", "Image Link",
"Gearbox", "Fuel", "Mileage"])
url = 'https://www.pistonheads.com/classifieds?Category=used- cars&Page=1&ResultsPerPage=100'
get_url = requests.get(url)
get_text = get_url.text
soup = BeautifulSoup(get_text, 'html.parser')
car_link = soup.find_all('div', 'listing-headline', 'price')
for div in car_link:
links = div.findAll('a')
for a in links:
link = ("https://www.pistonheads.com" + a['href'])
make = (a['href'].split('/')[-4])
model = (a['href'].split('/')[-3])
price = a.find('span').text.rstrip()
image_link = a.parent.parent.find('img')['src']
image = ("https:") + image_link
vehicle_details = a.parent.parent.find('ul', class_='specs')
specs = list(vehicle_details.stripped_strings)
gearbox = specs[3:]
fuel = specs[1:2]
mileage = specs[0:1]
writer.writerow([link, make, model, price, image, gearbox, fuel, mileage])
print(link, make, model, price, image, gearbox, fuel, mileage)
outfile.close()
答案 0 :(得分:2)
欢迎使用StackOverflow!
因此,您的脚本有很多改进之处。你到那儿了!
specs = list(vehicle_details.stripped_strings)
是分解为列表的生成器。有效地,您可以通过索引访问所需的内容。例如,mileage
可以简单地是specs[0]
。[
和]
的问题是由于使用切片mileage = specs[0:1]
引起的。在文档中,建立索引会返回一个项目,切片会返回一个新列表。参见lists introduction。mileage, fuel, _, gearbox = specs
mileage = specs[0]
import pdb; pdb.set_trace() # temp set on one line so you can remove it easily after
# now you can interactively inspect your code
(Pdb) specs
祝你好运!享受Python!
答案 1 :(得分:0)
如果您想从列表中获取字符串,也许可以这样做
gearbox = specs[3:][0] if specs[3:] else '-'
fuel = specs[1:2][0] if specs[1:2] else '-'
mileage = specs[0:1][0] if specs[0:1] else '-'
但是这种方式或aldnav答案甚至会引发错误,甚至会引发错误
ValueError:没有足够的值可解包
通常,我将首先提取父容器,而不是选择子容器(a
),然后再选择父容器。
# helper to get dynamic specs element
def getSpec(element, selector):
spec = element.select_one(selector)
return spec.nextSibling.string.strip() if spec else '-'
soup = BeautifulSoup(get_text, 'html.parser')
results = soup.find_all('div', class_="result-contain")
for car in results:
a = car.find('a')
if not a:
continue
link = ("https://www.pistonheads.com" + a['href'])
make = (a['href'].split('/')[-4])
model = (a['href'].split('/')[-3])
price = a.find('span').text.rstrip()
image_link = car.find('img')['src']
image = ("https:") + image_link
if not car.find('ul', class_='specs'):
gearbox = fuel = mileage = '-'
else:
gearbox = getSpec(car, '.location-pin-4')
fuel = getSpec(car, '.gas-1')
mileage = getSpec(car, '.gauge-1')
print(gearbox, fuel, mileage)
writer.writerow([link, make, model, price, image, gearbox, fuel, mileage])
#print(link, make, model, price, image, gearbox, fuel, mileage)
outfile.close()