Scala sbt项目解决依赖关系的速度非常慢
我的sbt项目需要超过15分钟Scala sbt项目解决依赖关系的速度非常慢,scala,sbt,Scala,Sbt,我的sbt项目需要超过15分钟 sbt clean compile 我在AWS的一台结实的机器上。我相当肯定这不是cpu或互联网带宽的资源问题。此外,我已经运行了这个命令几次,因此常春藤缓存被填充 这是我所有与构建相关的文件 /build.sbt name := "ProjectX" version := "1.0" scalaVersion := "2.10.5" libraryDependencies += ("org.apache.spark" %% "spark-streamin
sbt clean compile
我在AWS的一台结实的机器上。我相当肯定这不是cpu或互联网带宽的资源问题。此外,我已经运行了这个命令几次,因此常春藤缓存被填充
这是我所有与构建相关的文件
/build.sbt
name := "ProjectX"
version := "1.0"
scalaVersion := "2.10.5"
libraryDependencies += ("org.apache.spark" %% "spark-streaming" % "1.4.1")
.exclude("org.slf4j", "slf4j-log4j12")
.exclude("log4j", "log4j")
.exclude("commons-logging", "commons-logging")
.%("provided")
libraryDependencies += ("org.apache.spark" %% "spark-streaming-kinesis-asl" % "1.4.1")
.exclude("org.slf4j", "slf4j-log4j12")
.exclude("log4j", "log4j")
.exclude("commons-logging", "commons-logging")
libraryDependencies += "org.mongodb" %% "casbah" % "2.8.1"
//test
libraryDependencies += "org.scalatest" %% "scalatest" % "2.2.4" % "test"
//logging
libraryDependencies ++= Seq(
//facade
"org.slf4j" % "slf4j-api" % "1.7.12",
"org.clapper" %% "grizzled-slf4j" % "1.0.2",
//jcl (used by aws sdks)
"org.slf4j" % "jcl-over-slf4j" % "1.7.12",
//log4j1 (spark)
"org.slf4j" % "log4j-over-slf4j" % "1.7.12",
//log4j2
"org.apache.logging.log4j" % "log4j-api" % "2.3",
"org.apache.logging.log4j" % "log4j-core" % "2.3",
"org.apache.logging.log4j" % "log4j-slf4j-impl" % "2.3"
//alternative to log4j2
//"org.slf4j" % "slf4j-simple" % "1.7.5"
)
/project/build.properties
sbt.version = 0.13.8
/project/plugins.sbt
logLevel := Level.Warn
addSbtPlugin("org.scalastyle" %% "scalastyle-sbt-plugin" % "0.7.0")
resolvers += "sonatype-releases" at "https://oss.sonatype.org/content/repositories/releases/"
/project/assembly.sbt
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.13.0")
在日志中,您是否看到以下条目:
[info] [SUCCESSFUL ] org.apache.spark#spark-streaming-kinesis-asl_2.10;1.4.1!spark-streaming-kinesis-asl_2.10.jar (239ms)
这表明您正在下载这些工件。换句话说,您正在启动的AMI没有填充常春藤缓存
在带SSD的笔记本电脑上使用sbt 0.13.12,我可以获得5秒的清洁时间,然后更新
so-31956971> update
[info] Updating {file:/xxx/so-31956971/}app...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[success] Total time: 5 s, completed Aug 25, 2016 4:00:00 AM
您是否运行了'sbt-v'以获得一些详细的sbt输出?它可能会提示您哪些模块需要很长时间才能解析,或者哪些存储库需要解析。注意:artifactoryonline.com上的几个存储库被迁移到bintray.com,最近被关闭。因此,即使工件已经在ivy缓存中,速度也很慢?我们也有同样的问题。使用scala 2.11.7和sbt 13.9。DNS解析很快