Question

我试图通过“Web Scraping with Python”这本书来练习网页抓取，但是，我得到一个错误“AttributeError：'NoneType'对象没有属性'get_text'”,,,谁能告诉我怎么样我可以解决它，非常感谢,,,这里是代码和错误消息,,,我使用Python3和MySql数据库和Mac OS

from urllib.request import urlopen
from bs4 import BeautifulSoup
import datetime
import random
import pymysql
import re

conn = pymysql.connect(host='127.0.0.1',unix_socket='/tmp/mysql.sock',user='root',passwd=None,db='mysql',charset='utf8')

cur = conn.cursor()
cur.execute("USE scraping")

random.seed(datetime.datetime.now())

def store(title, content):
    cur.execute("INSERT INTO pages (title, content) VALUES (\"%s\",\"%s\")",    (title, content))
    cur.connection.commit()


def getLinks(articleUrl):
    html = urlopen("http://en.wikipedia.org"+articleUrl)
    bsObj = BeautifulSoup(html, "html.parser")
    title = bsObj.find("h1").find("span").get_text()
    content = bsObj.find("div", {"id":"mw-content- text"}).find("p").get_text()
    store(title, content)
    return bsObj.find("div",  {"id":"bodyContent"}).findAll("a",href=re.compile("^(/wiki/)((?!:).)*$"))

links = getLinks("/wiki/Kevin_Bacon")
try:
    while len(links) > 0:
        newArticle = links[random.randint(0, len(links)-1)].attrs["href"]
        print(newArticle)
        links = getLinks(newArticle)
finally:
    cur.close()
    conn.close()

Answer 1

因为h1标记内没有.find('span')个html标记。如果您想在title = bsObj.find("h1").get_text()中获取文字，只需删除content = bsObj.find("div", {"id":"mw-content- text"}).find("p").get_text() ^：

即可

content = bsObj.find("div", {"id":"mw-content-text"}).find("p").get_text()

中有一个空格

long date = System.currentTimeMillis();
int offset = TimeZone.getDefault().getOffset(date);
System.out.printf("%d + %d = %d%n", date, offset, date + offset);

删除它：

11:44:59.135 [ERROR] [org.gradle.BuildExceptionReporter] 
11:44:59.135 [ERROR] [org.gradle.BuildExceptionReporter] FAILURE: Build failed with an exception.
11:44:59.136 [ERROR] [org.gradle.BuildExceptionReporter] 
11:44:59.136 [ERROR] [org.gradle.BuildExceptionReporter] * What went wrong:
11:44:59.136 [ERROR] [org.gradle.BuildExceptionReporter] Execution failed for task ':app:compileDebugJavaWithJavac'.
11:44:59.137 [ERROR] [org.gradle.BuildExceptionReporter] > Compilation failed; see the compiler error output for details.
11:44:59.137 [ERROR] [org.gradle.BuildExceptionReporter] 
11:44:59.137 [ERROR] [org.gradle.BuildExceptionReporter] * Exception is:
11:44:59.138 [ERROR] [org.gradle.BuildExceptionReporter] org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':app:compileDebugJavaWithJavac'.
11:44:59.138 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:69)
11:44:59.139 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46)
11:44:59.139 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:35)
11:44:59.139 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:64)
11:44:59.140 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
11:44:59.140 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:42)
11:44:59.141 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
11:44:59.141 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53)
11:44:59.141 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
11:44:59.142 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.AbstractTask.executeWithoutThrowingTaskFailure(AbstractTask.java:305)
11:44:59.142 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.executeTask(AbstractTaskPlanExecutor.java:79)
11:44:59.142 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:63)
11:44:59.143 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:51)
11:44:59.143 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:23)
11:44:59.143 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:88)
11:44:59.143 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:29)
11:44:59.144 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62)
11:44:59.144 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.DefaultBuildExecuter.access$200(DefaultBuildExecuter.java:23)
11:44:59.144 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.DefaultBuildExecuter$2.proceed(DefaultBuildExecuter.java:68)
11:44:59.145 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
11:44:59.145 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62)
11:44:59.146 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:55)
11:44:59.146 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:149)
11:44:59.146 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:106)
11:44:59.147 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:86)
11:44:59.147 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:80)
11:44:59.147 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.tooling.internal.provider.BuildModelAction.run(BuildModelAction.java:43)
11:44:59.147 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.tooling.internal.provider.BuildModelAction.run(BuildModelAction.java:30)
11:44:59.148 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.tooling.internal.provider.ConfiguringBuildAction.run(ConfiguringBuildAction.java:119)
11:44:59.148 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:36)
11:44:59.148 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:26)
11:44:59.148 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.ExecuteBuild.doBuild(ExecuteBuild.java:47)
11:44:59.149 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:34)
11:44:59.149 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:119)
11:44:59.149 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.WatchForDisconnection.execute(WatchForDisconnection.java:35)
11:44:59.150 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:119)
11:44:59.150 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.ResetDeprecationLogger.execute(ResetDeprecationLogger.java:24)
11:44:59.150 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:119)
11:44:59.150 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.StartStopIfBuildAndStop.execute(StartStopIfBuildAndStop.java:33)
11:44:59.151 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:119)
11:44:59.151 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:71)
11:44:59.151 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:69)
11:44:59.152 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.util.Swapper.swap(Swapper.java:38)
11:44:59.152 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.ForwardClientInput.execute(ForwardClientInput.java:69)
11:44:59.152 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:119)
11:44:59.153 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.LogToClient.doBuild(LogToClient.java:60)
11:44:59.153 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:34)
11:44:59.153 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:119)
11:44:59.154 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment.doBuild(EstablishBuildEnvironment.java:70)
11:44:59.154 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:34)
11:44:59.154 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:119)
11:44:59.154 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.DaemonHygieneAction.execute(DaemonHygieneAction.java:39)
11:44:59.155 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:119)
11:44:59.155 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy$1.run(StartBuildOrRespondWithBusy.java:46)
11:44:59.155 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.launcher.daemon.server.DaemonStateCoordinator$1.run(DaemonStateCoordinator.java:246)
11:44:59.155 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.internal.concurrent.DefaultExecutorFactory$StoppableExecutorImpl$1.run(DefaultExecutorFactory.java:64)
11:44:59.155 [ERROR] [org.gradle.BuildExceptionReporter] Caused by: org.gradle.api.internal.tasks.compile.CompilationFailedException: Compilation failed; see the compiler error output for details.
11:44:59.156 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.compile.JdkJavaCompiler.execute(JdkJavaCompiler.java:44)
11:44:59.156 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.compile.JdkJavaCompiler.execute(JdkJavaCompiler.java:35)
11:44:59.156 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler.delegateAndHandleErrors(NormalizingJavaCompiler.java:97)
11:44:59.157 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler.execute(NormalizingJavaCompiler.java:50)
11:44:59.157 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler.execute(NormalizingJavaCompiler.java:36)
11:44:59.157 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.compile.CleaningJavaCompilerSupport.execute(CleaningJavaCompilerSupport.java:34)
11:44:59.157 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.compile.CleaningJavaCompilerSupport.execute(CleaningJavaCompilerSupport.java:25)
11:44:59.158 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.tasks.compile.JavaCompile.performCompilation(JavaCompile.java:158)
11:44:59.158 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.tasks.compile.JavaCompile.compile(JavaCompile.java:138)
11:44:59.158 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.tasks.compile.JavaCompile.compile(JavaCompile.java:92)
11:44:59.159 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:63)
11:44:59.159 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$IncrementalTaskAction.doExecute(AnnotationProcessingTaskFactory.java:235)
11:44:59.159 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:211)
11:44:59.159 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$IncrementalTaskAction.execute(AnnotationProcessingTaskFactory.java:222)
11:44:59.160 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:200)
11:44:59.160 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:80)
11:44:59.161 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:61)
11:44:59.161 [ERROR] [org.gradle.BuildExceptionReporter]    ... 55 more
11:44:59.161 [ERROR] [org.gradle.BuildExceptionReporter]

然后你的代码可能正常。

AttributeError：'NoneType'对象没有属性'get_text'我该如何解决？

1 个答案: