我正在尝试访问Spark的Scala解释器中的数据库,但是没有成功。
首先,我导入了DriverManager,并使用以下命令将SQL Server JDBC驱动程序添加到类路径中:
scala> import java.sql._
import java.sql._
scala> :cp sqljdbc41.jar
REPL因长转储消息而崩溃:
Added 'C:\spark\sqljdbc41.jar'. Your new classpath is:
";;C:\spark\bin\..\conf;C:\spark\bin\..\lib\spark-assembly-1.1.1-hadoop2.4.0.jar;;C:\spark\bin\..\lib\datanucleus-api-jdo-3.2.1.jar;C:\spark\bin\..\lib\datanucleus-core-3.2.2.jar;C:\spark\bin\..\lib\datanucleus-rdbms-3.2.1.jar;;C:\spark\sqljdbc41.jar"
Replaying: import java.sql._
error:
while compiling: <console>
during phase: jvm
library version: version 2.10.4
compiler version: version 2.10.4
reconstructed args:
last tree to typer: Apply(constructor $read)
symbol: constructor $read in class $read (flags: <method> <triedcooking>)
symbol definition: def <init>(): $line10.$read
tpe: $line10.$read
symbol owners: constructor $read -> class $read -> package $line10
context owners: class iwC -> package $line10
== Enclosing template or block ==
Template( // val <local $iwC>: <notype>, tree.tpe=$line10.iwC
"java.lang.Object", "scala.Serializable" // parents
ValDef(
private
"_"
<tpt>
<empty>
)
...
== Expanded type of tree ==
TypeRef(TypeSymbol(class $read extends Serializable))
uncaught exception during compilation: java.lang.AssertionError
java.lang.AssertionError: assertion failed: Tried to find '$line10' in 'C:\Users\Username\AppData\Local\Temp\spark-28055904-e7d2-4052-9354-ae3769266cb4' but it is not a directory
That entry seems to have slain the compiler. Shall I replay
your session? I can re-run each line except the last one.
我可以使用驱动程序运行Scala程序,一切正常。
如何初始化REPL以允许我通过JDBC从SQL Server访问数据?
答案 0 :(得分:1)
看起来交互式:cp
命令在Windows中不起作用。但是我发现如果我使用以下命令启动spark shell,则会加载JDBC驱动程序并使其可用:
C:\spark> .\bin\spark-shell --jars sqljdbc41.jar
在这种情况下,我将我的jar文件复制到C:\ spark文件夹中。
(此外,还可以使用--help
查看启动时可用的其他命令。)