我无法在主蜘蛛文件中导入items.py文件[scrapy]

时间:2019-04-12 13:18:17

标签: python scrapy

project structure

我正在尝试将我的商品文件导入shop.py文件。我也尝试过[从..items导入ShopcluesItem],但是它不起作用。请告诉我我在做什么错了吗?

import scrapy
from items import ShopcluesItem

class ShopSpider(scrapy.Spider):
    name = 'shop'
    allowed_domains = ['shopclues.com']
    start_urls = ['https://www.shopclues.com/mobiles-smartphones.html?sort_by=bestsellers']

    def parse(self, response):

        items = QuotetutorialItem()

        titles = response.css('img::attr(title)').extract()
        images = response.css('img::attr(data-img)').extract()
        prices = response.css('.p_price::text').extract()
        discounts = response.css('.prd_discount::text').extract()

        for item in zip(titles, prices, images, discounts):
            scraped_info = {
                'title': item[0],
                'price': item[1],
                'image_urls': [item[2]],  # Set's the url for scrapy to download images
                'discount': item[3]
            }

            yield scraped_info

            has_next = response.css('.load-more').extract()
            if has_next:
                next_page = response.meta.get('next_page', 1) + 1
                url = response.urljoin(response.css('script').re_first("'(\?searchId.*page=)'") + str(next_page))
                yield Request(url, meta={'next_page': next_page})

0 个答案:

没有答案
相关问题