Question

This是我试图抓住的网站。

我可以很轻松地登录网站。但是，我无法检索和重复使用cookie或会话ID来抓取登录页面重定向到的页面之外的页面。我每次都收到403。

这是我尝试过的一个例子：

try
{
    String userAgent = "User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0";

    Connection.Response res = Jsoup.connect("http://www.interpals.net/login.php")
            .data("action", "login")
            .data("login", username)
            .data("password", password)
            .data("auto_login", "1")
            .userAgent(userAgent)
            .method(Connection.Method.POST)
            .followRedirects(false)
            .execute();

    res.parse();
    String sessionID = res.cookie("interpals_sessid");

    Document doc = Jsoup.connect("http://www.interpals.net/friends.php").cookie("interpals_sessid", sessionID).get();

Answer 1

此代码适用于我：

try {    
    String url = "http://www.interpals.net/login.php";
    String userAgent = "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.130 Safari/537.36";

    Connection.Response response = Jsoup.connect(url).userAgent(userAgent)
            .method(Connection.Method.GET)
            .execute();

    response = Jsoup.connect(url)
            .cookies(response.cookies())
            .data("action", "login")
            .data("login", "login")
            .data("password", "password")
            .data("auto_login", "1")
            .userAgent(userAgent)
            .method(Connection.Method.POST)
            .followRedirects(true)
            .execute();            

    Document doc = Jsoup.connect("http://www.interpals.net/friends.php")
            .cookies(response.cookies())
            .userAgent(userAgent)
            .get();

    System.out.println(doc);

} catch (IOException e) {
    e.printStackTrace();
}

使用重定向的Jsoup登录网站，并抓取不是重定向的网页

1 个答案: