如何使用Scrapy正确请求“多部分/表单数据”表单?

时间:2018-07-16 16:30:19

标签: python-3.x scrapy

因此,我具有以下创建多部分/表单的代码段:

from requests_toolbelt import MultipartEncoder
from random import random
from sys import maxsize


def gen_boundary():
    alpha_numeric_encoding_map = \
        'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789AB'

    boundary = '----WebKitFormBoundary'

    for i in range(4):
        boundary += alpha_numeric_encoding_map[int(random() * (maxsize + 1)) >> 24 & 0x3F]
        boundary += alpha_numeric_encoding_map[int(random() * (maxsize + 1)) >> 16 & 0x3F]
        boundary += alpha_numeric_encoding_map[int(random() * (maxsize + 1)) >> 8 & 0x3F]
        boundary += alpha_numeric_encoding_map[int(random() * (maxsize + 1)) & 0x3F]

    return boundary


payload = dict(new='order', type='misc')

me = MultipartEncoder(fields=payload, boundary=gen_boundary())
me_body = me.to_string()

headers = {'Content-Type': me.content_type, 'Content-Length': me.len}

为此,我呼叫了请求

yield Request(url='http://example.com/test',body=me_body, headers=headers, callback=self.test)

我的问题是,请求总是返回 500 Internal Server Error ,我该怎么做才能使该请求正常工作?我想念什么吗?

我什至尝试更改lib:

from urllib3 import encode_multipart_formdata
body, headers = encode_multipart_formdata(fields=payload, boundary=self.gen_boundary())

结果是相同的。

1 个答案:

答案 0 :(得分:0)

挖掘一点发现,在提交表单之前,我错过了一个请求!在我提出请求后,表单已成功提交。