scala sbt 加载了spark包,DEBUG的时候报错了。
libraryDependencies ++= Seq( "org.apache.spark" % "spark-core_2.12" % "2.4.0", "org.apache.spark" % "spark-sql_2.12" % "2.4.0" )
调试报以下错误:
Exception in thread "main" java.lang.NoSuchMethodError: scala.Product.$init$(Lscala/Product;)V
at org.apache.spark.SparkConf$DeprecatedConfig.<init>(SparkConf.scala:809)
at org.apache.spark.SparkConf$.<init>(SparkConf.scala:642)
at org.apache.spark.SparkConf$.<clinit>(SparkConf.scala)
at org.apache.spark.SparkConf.set(SparkConf.scala:94)
at org.apache.spark.SparkConf.set(SparkConf.scala:83)
at org.apache.spark.SparkConf.setAppName(SparkConf.scala:120)
at com.demo.WordCount$.main(WordCount.scala:11)
at com.demo.WordCount.main(WordCount.scala)
解决办法:
scalaVersion := "2.11.8" libraryDependencies ++= Seq( "org.apache.spark" % "spark-core_2.11" % "2.3.0", "org.apache.spark" % "spark-sql_2.11" % "2.3.0" )
将scala和spark版本都降到2.11,然后好了。