捕获故障-'；内存未映射'；R中的错误

我在集群上运行一些R脚本时遇到问题。问题突然出现（所有脚本都工作正常，但有一天它们开始出现

错误）。我不能提供可复制的代码，因为我甚至不能在自己的计算机上复制错误-它只发生在集群上。我还在两组数据中使用相同的代码——一组非常小，运行良好，另一组处理更大的数据帧（约1000万行），并在某些点折叠。我只使用CRAN存储库中的包；R和所有的包都应该是最新的。错误出现在完全不相关的操作中，请参见以下示例：

会话信息：

R version 3.4.3 (2017-11-30)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

将变量写入NetCDF文件

# code snippet
library(ncdf4)
library(reshape2)

input <- read.csv("input_file.csv")
species <- "no2"
dimX <- ncdim_def(name="x", units = "m", vals = unique(input$x), unlim = FALSE)
dimY <- ncdim_def(name="y", units = "m", vals = unique(input$y), unlim = FALSE)
dimTime <- ncdim_def(name = "time", units = "hours", unlim = TRUE)

varOutput <- ncvar_def(name = species, units = "ug/m3",
                dim = list(dimX, dimY, dimTime), missval = -9999, longname = species)

nc_file <- nc_create(filename = "outFile.nc", vars = list(varOutput), force_v4 = T)

ncvar_put(nc = nc_file, varid = species, vals = acast(input, x~y), start = c(1,1,1),
      count = c(length(unique(input$x)), length(unique(input$y)), 1))


具有并行计算的复杂代码

 *** caught segfault ***
address 0x330d39b40, cause 'memory not mapped'

Traceback:
 1: .Call(gstat_fit_variogram, as.integer(fit.method), as.integer(fit.sills),     as.integer(fit.ranges))
 2: fit.variogram(experimental_variogram, model = vgm(psill = psill,     model = model, range = range, nugget = nugget, kappa = kappa),     fit.ranges = c(fit_range), fit.sills = c(fit_nugget, fit_sill),     debug.level = 0)
 3: doTryCatch(return(expr), name, parentenv, handler)
 4: tryCatchOne(expr, names, parentenv, handlers[[1L]])
 5: tryCatchList(expr, classes, parentenv, handlers)
 6: tryCatch(expr, error = function(e) {    call <- conditionCall(e)          if (!is.null(call)) {        if (identical(call[[1L]], quote(doTryCatch)))             call <- sys.call(-4L)        dcall <- deparse(call)[1L]        prefix <- paste("Error in", dcall, ": ")        LONG <- 75L        msg <- conditionMessage(e)        sm <- strsplit(msg, "\n")[[1L]]        w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w")        if (is.na(w))             w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L],                 type = "b")        if (w > LONG)             prefix <- paste0(prefix, "\n  ")    }    else prefix <- "Error : "    msg <- paste0(prefix, conditionMessage(e), "\n")    .Internal(seterrmessage(msg[1L]))    if (!silent && identical(getOption("show.error.messages"),         TRUE)) {        cat(msg, file = outFile)        .Internal(printDeferredWarnings())    }    invisible(structure(msg, class = "try-error", condition = e))})
 7: try(fit.variogram(experimental_variogram, model = vgm(psill = psill,     model = model, range = range, nugget = nugget, kappa = kappa),     fit.ranges = c(fit_range), fit.sills = c(fit_nugget, fit_sill),     debug.level = 0), TRUE)
 8: getModel(initial_sill - initial_nugget, m, initial_range, k,     initial_nugget, fit_range, fit_sill, fit_nugget, verbose = verbose)
 9: autofitVariogram(lmResids ~ 1, obsDf, model = "Mat", kappa = c(0.05,     seq(0.2, 2, 0.1), 3, 5, 10, 15), fix.values = c(NA, NA, NA),     start_vals = c(NA, NA, NA), verbose = F)
10: main_us(obsDf[obsDf$class == "rural" | obsDf$class == "rural-nearcity" |     obsDf$class == "rural-regional" | obsDf$class == "rural-remote",     ], grd_alt, grd_pop, lm_us, fitvar_us, logTransform, plots,     "RuralSt", period, preds)
11: doTryCatch(return(expr), name, parentenv, handler)
12: tryCatchOne(expr, names, parentenv, handlers[[1L]])
13: tryCatchList(expr, classes, parentenv, handlers)
14: tryCatch(main_us(obsDf[obsDf$class == "rural" | obsDf$class ==     "rural-nearcity" | obsDf$class == "rural-regional" | obsDf$class ==     "rural-remote", ], grd_alt, grd_pop, lm_us, fitvar_us, logTransform,     plots, "RuralSt", period, preds), error = function(e) e)
15: eval(.doSnowGlobals$expr, envir = .doSnowGlobals$exportenv)
16: eval(.doSnowGlobals$expr, envir = .doSnowGlobals$exportenv)
17: doTryCatch(return(expr), name, parentenv, handler)
18: tryCatchOne(expr, names, parentenv, handlers[[1L]])
19: tryCatchList(expr, classes, parentenv, handlers)
20: tryCatch(eval(.doSnowGlobals$expr, envir = .doSnowGlobals$exportenv),     error = function(e) e)
21: (function (args) {    lapply(names(args), function(n) assign(n, args[[n]], pos = .doSnowGlobals$exportenv))    tryCatch(eval(.doSnowGlobals$expr, envir = .doSnowGlobals$exportenv),         error = function(e) e)})(quote(list(timeIndex = 255L)))
22: do.call(msg$data$fun, msg$data$args, quote = TRUE)
23: doTryCatch(return(expr), name, parentenv, handler)
24: tryCatchOne(expr, names, parentenv, handlers[[1L]])
25: tryCatchList(expr, classes, parentenv, handlers)
26: tryCatch(do.call(msg$data$fun, msg$data$args, quote = TRUE),     error = handler)
27: doTryCatch(return(expr), name, parentenv, handler)
28: tryCatchOne(expr, names, parentenv, handlers[[1L]])
29: tryCatchList(expr, classes, parentenv, handlers)
30: tryCatch({    msg <- recvData(master)    if (msg$type == "DONE") {        closeNode(master)        break    }    else if (msg$type == "EXEC") {        success <- TRUE        handler <- function(e) {            success <<- FALSE            structure(conditionMessage(e), class = c("snow-try-error",                 "try-error"))        }        t1 <- proc.time()        value <- tryCatch(do.call(msg$data$fun, msg$data$args,             quote = TRUE), error = handler)        t2 <- proc.time()        value <- list(type = "VALUE", value = value, success = success,             time = t2 - t1, tag = msg$data$tag)        msg <- NULL        sendData(master, value)        value <- NULL    }}, interrupt = function(e) NULL)
31: slaveLoop(makeSOCKmaster(master, port, timeout, useXDR))
32: parallel:::.slaveRSOCK()
An irrecoverable exception occurred. R is aborting now ...

我尝试基于卸载和重新安装软件包，但没有帮助。
强烈建议清理工作区，可能是核心问题：

unlink（“.RData”）
这并不是问题的真正解释或令人满意的答案，但我更仔细地检查了代码，发现在第一个示例中，当使用
restrape2
包中的
acast
时会出现问题。在本例中，我删除了它，因为我意识到它实际上并不需要，但可以用
restrape
包中的
restrape
替换它（如中所示）：
restrape（input，idvar=“x”，timevar=“y”，direction=“wide”）[-1]

至于第二个例子，要找到问题的确切原因并不容易，但在我的例子中，作为一种解决办法，我帮助设置了用于并行计算的较少数量的核-集群有48个，我只使用了15个，因为即使在这个问题之前，如果代码使用全部48个核运行，R的内存也会不足。当我将核心数减少到10个时，它突然开始像以前一样工作。
问题是当前安装的共享库与为安装R或软件包而构建的库不匹配

我今天第一次犯了这个错误。见下文。我已经解决了，可以解释情况了

这是一个Ubuntu系统，最近从17.10升级到18.04，运行R-3.4.4。许多C和C++库被替换。但并不是所有的程序都被替换了。我立刻注意到很多程序都出现了分段错误。任何触碰整洁的东西都是失败的。
stringi
包找不到用于编译它的共享库

这里的示例有点有趣，因为它发生在为包运行“R CMD check”时，至少在理论上，这应该是安全的。我发现修复方法是删除包“RCurl”和“url”并重建它们

不管怎么说，症状是这样的

* checking for file ‘kutils.gitex/DESCRIPTION’ ... OK
* preparing ‘kutils’:
* checking DESCRIPTION meta-information ... OK
* installing the package to build vignettes
* creating vignettes ... OK
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
* looking to see if a ‘data/datalist’ file should be added
* re-saving image files
* building ‘kutils_1.40.tar.gz’
Warning: invalid uid value replaced by that for user 'nobody'
Warning: invalid gid value replaced by that for user 'nobody'

Run check: OK? (y or n)y
* using log directory ‘/home/pauljohn/GIT/CRMDA/software/kutils/package/kutils.Rcheck’
* using R version 3.4.4 (2018-03-15)
* using platform: x86_64-pc-linux-gnu (64-bit)
* using session charset: UTF-8
* using option ‘--as-cran’
* checking for file ‘kutils/DESCRIPTION’ ... OK
* checking extension type ... Package
* this is package ‘kutils’ version ‘1.40’
* checking CRAN incoming feasibility ...
 *** caught segfault ***
address 0x68456, cause 'memory not mapped'

Traceback:
 1: curlGetHeaders(u)
 2: doTryCatch(return(expr), name, parentenv, handler)
 3: tryCatchOne(expr, names, parentenv, handlers[[1L]])
 4: tryCatchList(expr, classes, parentenv, handlers)
 5: tryCatch(curlGetHeaders(u), error = identity)
 6: .fetch(u)
 7: .check_http_A(u)
 8: FUN(X[[i]], ...)
 9: lapply(urls[pos], .check_http)
10: do.call(rbind, lapply(urls[pos], .check_http))
11: check_url_db(url_db_from_package_sources(dir), remote = !localOnly)
12: doTryCatch(return(expr), name, parentenv, handler)
13: tryCatchOne(expr, names, parentenv, handlers[[1L]])
14: tryCatchList(expr, classes, parentenv, handlers)
15: tryCatch(check_url_db(url_db_from_package_sources(dir), remote = !localOnly),     error = identity)
16: .check_package_CRAN_incoming(pkgdir, localOnly)
17: check_CRAN_incoming(!check_incoming_remote)
18: tools:::.check_packages()
An irrecoverable exception occurred. R is aborting now ...
Segmentation fault

最好在集群中的每个系统上运行一次内存检查。您能说明我应该在哪一点清理工作区吗？我从不保存我的工作区。我使用
R--no save
从终端运行脚本。您应该可以在终端上执行此操作。抱歉，这根本不起作用。。。但是我想我已经追踪到了软件包的问题，我会更深入地研究它。根据定义，Segfaults是联系软件包维护人员的一个理由。@42-即使问题不可重现？在第二个例子中，我通过减少并行计算的内核数量来解决问题，我应该联系哪个软件包的维护人员？：-）我根据你的建议减少了内核的数量，这是唯一对我有用的东西。谢谢！我们应该怎么做？
Message from syslogd@master1 at Mar  8 13:51:37 ...
 kernel:[Hardware Error]: MC4 Error (node 1): DRAM ECC error detected on the NB.

Message from syslogd@master1 at Mar  8 13:51:37 ...
 kernel:[Hardware Error]: Error Status: Corrected error, no action required.

Message from syslogd@master1 at Mar  8 13:51:37 ...
 kernel:[Hardware Error]: CPU:4 (15:2:0) MC4_STATUS[-|CE|MiscV|-|AddrV|-|-|CECC]: 0x9c08400067080a13

Message from syslogd@master1 at Mar  8 13:51:37 ...
kernel:[Hardware Error]: MC4_ADDR: 0x000000048f32b490

Message from syslogd@master1 at Mar  8 13:51:37 ...
 kernel:[Hardware Error]: cache level: L3/GEN, mem/io: MEM, mem-tx: RD, part-proc: RES (no timeout)

* checking for file ‘kutils.gitex/DESCRIPTION’ ... OK
* preparing ‘kutils’:
* checking DESCRIPTION meta-information ... OK
* installing the package to build vignettes
* creating vignettes ... OK
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
* looking to see if a ‘data/datalist’ file should be added
* re-saving image files
* building ‘kutils_1.40.tar.gz’
Warning: invalid uid value replaced by that for user 'nobody'
Warning: invalid gid value replaced by that for user 'nobody'

Run check: OK? (y or n)y
* using log directory ‘/home/pauljohn/GIT/CRMDA/software/kutils/package/kutils.Rcheck’
* using R version 3.4.4 (2018-03-15)
* using platform: x86_64-pc-linux-gnu (64-bit)
* using session charset: UTF-8
* using option ‘--as-cran’
* checking for file ‘kutils/DESCRIPTION’ ... OK
* checking extension type ... Package
* this is package ‘kutils’ version ‘1.40’
* checking CRAN incoming feasibility ...
 *** caught segfault ***
address 0x68456, cause 'memory not mapped'

Traceback:
 1: curlGetHeaders(u)
 2: doTryCatch(return(expr), name, parentenv, handler)
 3: tryCatchOne(expr, names, parentenv, handlers[[1L]])
 4: tryCatchList(expr, classes, parentenv, handlers)
 5: tryCatch(curlGetHeaders(u), error = identity)
 6: .fetch(u)
 7: .check_http_A(u)
 8: FUN(X[[i]], ...)
 9: lapply(urls[pos], .check_http)
10: do.call(rbind, lapply(urls[pos], .check_http))
11: check_url_db(url_db_from_package_sources(dir), remote = !localOnly)
12: doTryCatch(return(expr), name, parentenv, handler)
13: tryCatchOne(expr, names, parentenv, handlers[[1L]])
14: tryCatchList(expr, classes, parentenv, handlers)
15: tryCatch(check_url_db(url_db_from_package_sources(dir), remote = !localOnly),     error = identity)
16: .check_package_CRAN_incoming(pkgdir, localOnly)
17: check_CRAN_incoming(!check_incoming_remote)
18: tools:::.check_packages()
An irrecoverable exception occurred. R is aborting now ...
Segmentation fault