Question

这是我抓取文章的功能，现在我正在尝试如何抓取作者的姓名，上传日期和更新日期。我可以采取哪些方法来处理SF编年史上的许多文章？

from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup
# Initializing our title key
title_key = 'title'
dictionary.setdefault(title_key, [])

# Initializing our url key
url_key = 'url'
dictionary.setdefault(url_key, [])

# Initializing our author key
author_key = 'author'
dictionary.setdefault(text_key, [])

# Initializing our date key
author_key = 'date'
dictionary.setdefault(text_key, [])

# Initializing our date updated key
author_key = 'date_updated'
dictionary.setdefault(text_key, []) 

def article_scraper(url):
    # Opening up the connection, grabbing the page
    uClient = uReq(url)
    page_html = uClient.read()
    uClient.close()

    # HTML parsing
    page_soup = soup(page_html, "html.parser")

    dictionary['url'].append(url)
    dictionary['title'].append(page_soup.title.string)
    dictionary['author'] = page_soup.select("author.name")
    return(dictionary)

articles[0] = 'https://www.sfchronicle.com/news/bayarea/heatherknight/article/Special-education-teacher-a-prime-example-of-13560483.php'
article0 = article_scraper(articles[0])

Answer 1

作者姓名：

author = soup.findAll("div", {"class": ""header-authors-name"})

如何获得文章的作者，上载和更新的日期？

1 个答案: