我想在商品类中添加商品类

时间:2019-01-18 06:07:48

标签: python scrapy scrapy-spider

最终JSON为:

            "address": ----,
            "state": ----,
            year: {
                "first": ----,
                "second": {
                    "basic": ----,
                    "Information": ----,
                    }
            },

我想创建自己的items.py(仅作为示例):

class Item(scrapy.Item): 
  address = scrapy.Field()
  state = scrapy.Field()
  year = scrapy.Field(first), scrapy.Field(second)

class first(scrapy.Item):
  amounts = scrapy.Field()

class second(scrapy.Item):
  basic = scrapy.Field()
  information = scrapy.Field()

如何实现此功能,已经检查了https://doc.scrapy.org/en/latest/topics/items.html#extending-items

how to implement nested item in scrapy?

但是这个概念没有任何线索……有什么建议吗?

2 个答案:

答案 0 :(得分:1)

class Item(scrapy.Item): 
  address = scrapy.Field()
  state = scrapy.Field()
  year = scrapy.Field(serializer=dict)

class Year(scrapy.Item):
  first = scrapy.Field(serializer=dict)
  second = scrapy.Field(serializer=dict)

class first(scrapy.Item):
  amounts = scrapy.Field()

class second(scrapy.Item):
  basic = scrapy.Field()
  information = scrapy.Field()

通过这种方式您可以这样做:

>>> b = second(basic="hello", information="hello world")
>>> a = first(amounts=3)
>>> year = Year(first=a, second=b)
>>> year
{'first': {'amounts': 3},
 'second': {'basic': 'hello', 'information': 'hello world'}}
>>> item = Item(address='address value', state='state value', year=year)
>>> item
{'address': 'address value',
 'state': 'state value',
 'year': {'first': {'amounts': 3}, 'second': {'basic': 'hello', 'information': 'hello world'}}}

答案 1 :(得分:0)

class Item(scrapy.Item): 
  address = scrapy.Field()
  state = scrapy.Field()
  year = scrapy.Field(first), scrapy.Field(second)    #You dont need to do like this

class first(scrapy.Item):
  amounts = scrapy.Field()     #this work and below

class second(scrapy.Item):    #and yes this work, you can do it in spider level or pipelines, just make your desired data, and pas it to year variable as you want. it will accumulate that
  basic = scrapy.Field()
  information = scrapy.Field()

让我给你举个例子,

first = {'first': first}
second = {'basic': basic, 'info': info}

year = {'first': first, 'second': second}