如何强制requests
库使用特定的互联网协议版本来获取请求?或者这可以通过Python中的另一种方法更好地实现?我可以,但我不想使用curl
...
澄清目的的例子:
import requests
r = requests.get('https://my-dyn-dns-service.domain/?hostname=my.domain',
auth = ('myUserName', 'my-password'))
答案 0 :(得分:5)
我找到了一个简单的解决方案来强制urrlib3使用ipv4或ipv6。 urrlib3使用此方法为Http和Https创建新连接。您可以在其中指定要使用的任何AF_FAMILY。
import socket
import requests.packages.urllib3.util.connection as urllib3_cn
def allowed_gai_family():
"""
https://github.com/shazow/urllib3/blob/master/urllib3/util/connection.py
"""
family = socket.AF_INET
if urllib3_cn.HAS_IPV6:
family = socket.AF_INET6 # force ipv6 only if it is available
return family
urllib3_cn.allowed_gai_family = allowed_gai_family
答案 1 :(得分:3)
这是一个黑客,但你可以修补getaddrinfo来过滤到只有IPv4地址:
# Monkey patch to force IPv4, since FB seems to hang on IPv6
import socket
old_getaddrinfo = socket.getaddrinfo
def new_getaddrinfo(*args, **kwargs):
responses = old_getaddrinfo(*args, **kwargs)
return [response
for response in responses
if response[0] == socket.AF_INET]
socket.getaddrinfo = new_getaddrinfo
答案 2 :(得分:1)
这是完全未经测试的,可能需要进行一些调整,但结合Using Python “requests” with existing socket connection和how to force python httplib library to use only A requests的答案,看起来你应该能够创建一个仅IPv6的套接字然后请求使用它它的连接池有:
try:
from http.client import HTTPConnection
except ImportError:
from httplib import HTTPConnection
class MyHTTPConnection(HTTPConnection):
def connect(self):
print("This actually called called")
self.sock = socket.socket(socket.AF_INET6)
self.sock.connect((self.host, self.port,0,0))
if self._tunnel_host:
self._tunnel()
requests.packages.urllib3.connectionpool.HTTPConnection = MyHTTPConnection
答案 3 :(得分:1)
阅读上一个答案后,我不得不修改代码以强制IPv4而不是IPv6。请注意,我使用socket.AF_INET而不是socket.AF_INET6,而self.sock.connect()具有2项元组参数。
我还需要覆盖与HTTP连接非常不同的HTTP S 连接,因为requests
包装httplib.HTTPSConnection以在ssl
模块可用时验证证书
import socket
import ssl
try:
from http.client import HTTPConnection
except ImportError:
from httplib import HTTPConnection
from requests.packages.urllib3.connection import VerifiedHTTPSConnection
# HTTP
class MyHTTPConnection(HTTPConnection):
def connect(self):
self.sock = socket.socket(socket.AF_INET)
self.sock.connect((self.host, self.port))
if self._tunnel_host:
self._tunnel()
requests.packages.urllib3.connectionpool.HTTPConnection = MyHTTPConnection
requests.packages.urllib3.connectionpool.HTTPConnectionPool.ConnectionCls = MyHTTPConnection
# HTTPS
class MyHTTPSConnection(VerifiedHTTPSConnection):
def connect(self):
self.sock = socket.socket(socket.AF_INET)
self.sock.connect((self.host, self.port))
if self._tunnel_host:
self._tunnel()
self.sock = ssl.wrap_socket(self.sock, self.key_file, self.cert_file)
requests.packages.urllib3.connectionpool.HTTPSConnection = MyHTTPSConnection
requests.packages.urllib3.connectionpool.VerifiedHTTPSConnection = MyHTTPSConnection
requests.packages.urllib3.connectionpool.HTTPSConnectionPool.ConnectionCls = MyHTTPSConnection
答案 4 :(得分:1)
我对https://stackoverflow.com/a/33046939/5059062采用了类似的方法,但修改了socket
中发出DNS请求的部分,因此仅执行IPv6或IPv4,每个请求,这意味着可以在urllib
中使用此请求,就像在requests
中一样有效。
如果您的程序也使用unix管道和其他类似的东西,这可能会很糟糕,所以我强烈建议使用monkeypatching。
import requests
import socket
from unittest.mock import patch
import re
orig_getaddrinfo = socket.getaddrinfo
def getaddrinfoIPv6(host, port, family=0, type=0, proto=0, flags=0):
return orig_getaddrinfo(host=host, port=port, family=socket.AF_INET6, type=type, proto=proto, flags=flags)
def getaddrinfoIPv4(host, port, family=0, type=0, proto=0, flags=0):
return orig_getaddrinfo(host=host, port=port, family=socket.AF_INET, type=type, proto=proto, flags=flags)
with patch('socket.getaddrinfo', side_effect=getaddrinfoIPv6):
r = requests.get('http://ip6.me')
print('ipv6: '+re.search(r'\+3>(.*?)</',r.content.decode('utf-8')).group(1))
with patch('socket.getaddrinfo', side_effect=getaddrinfoIPv4):
r = requests.get('http://ip6.me')
print('ipv4: '+re.search(r'\+3>(.*?)</',r.content.decode('utf-8')).group(1))
且没有requests
:
import urllib.request
import socket
from unittest.mock import patch
import re
orig_getaddrinfo = socket.getaddrinfo
def getaddrinfoIPv6(host, port, family=0, type=0, proto=0, flags=0):
return orig_getaddrinfo(host=host, port=port, family=socket.AF_INET6, type=type, proto=proto, flags=flags)
def getaddrinfoIPv4(host, port, family=0, type=0, proto=0, flags=0):
return orig_getaddrinfo(host=host, port=port, family=socket.AF_INET, type=type, proto=proto, flags=flags)
with patch('socket.getaddrinfo', side_effect=getaddrinfoIPv6):
r = urllib.request.urlopen('http://ip6.me')
print('ipv6: '+re.search(r'\+3>(.*?)</',r.read().decode('utf-8')).group(1))
with patch('socket.getaddrinfo', side_effect=getaddrinfoIPv4):
r = urllib.request.urlopen('http://ip6.me')
print('ipv4: '+re.search(r'\+3>(.*?)</',r.read().decode('utf-8')).group(1))
在3.5.2中测试
答案 5 :(得分:1)
我为requests
+ urllib3
+ socket
编写了一个运行时修补程序,该修补程序允许根据每个请求可选地传递所需的地址族。
与其他解决方案不同,它不涉及任何猴子修补,而是用修补文件替换了requests
的导入,并且它提供了一个request
兼容的接口,所有子类和修补的所有公开类都“简单” API”功能重新实现。唯一明显的区别应该是存在一个额外的family
参数,您可以使用该参数将名称解析期间使用的地址族限制为socket.AF_INET
或socket.AF_INET6
。然后,使用一系列有点复杂(但大多只是LoC密集型)的战略方法覆盖来将该值一直传递到urllib3
的最底层,在{{1 }}函数调用。
TL; DR 用法如下:
socket.create_connection
到修补程序库的完整链接(〜350 LoC):https://gitlab.com/snippets/1900824