使用ETL从CSV文件将OPoint数据导入OrientDB 2.2.x
这与我先前的问题有关使用ETL从CSV文件将OPoint数据导入OrientDB 2.2.x,orientdb,orientdb2.2,orientdb-etl,Orientdb,Orientdb2.2,Orientdb Etl,这与我先前的问题有关 (我想出来了) (仍在寻找有用的答案) 作为对(2)的回应,我正在考虑修改我的nazca geoglyph数据集,以使用WKT版本来与更新的功能保持一致 我的输入CSV文件,nazca\u lines\u wkt.CSV如下: Name,Location Hummingbird,POINT(-75.148892 -14.692131) Monkey,POINT(-75.138532 -14.706940) Condor,POINT(-75.126208 -14.697444
nazca\u lines\u wkt.CSV
如下:
Name,Location
Hummingbird,POINT(-75.148892 -14.692131)
Monkey,POINT(-75.138532 -14.706940)
Condor,POINT(-75.126208 -14.697444)
Spider,POINT(-75.122381 -14.694145)
Spiral,POINT(-75.122746 -14.688277)
Hands,POINT(-75.113881 -14.694459)
Tree,POINT(-75.114520 -14.693898)
Astronaut,POINT(-75.079755 -14.745222)
Dog,POINT(-75.130788 -14.706401)
Wing,POINT(-75.100385 -14.680309)
Parrot,POINT(-75.107498 -14.689463)
我创建了一个空的PLOCAL数据库,nazca wkt.orientdb
,并定义了一个GeoGlyphWKT顶点类:
CREATE DATABASE PLOCAL:nazca-wkt.orientdb admin admin plocal graph
CREATE CLASS GeoGlyphWKT EXTENDS V
CREATE PROPERTY GeoGlyphWKT.Name STRING
CREATE PROPERTY GeoGlyphWKT.Location EMBEDDED OPoint
CREATE PROPERTY GeoGlyphWKT.Tag EMBEDDEDSET STRING
我有两个用于oetl脚本的.json文件:
nazca\u lines\u wkt.json
{
"config": {
"log": "info",
"fileDirectory": "./",
"fileName": "nazca_lines_wkt.csv"
}
}
commongeogioglyphwkt.json
{
"begin": [ { "let": { "name": "$filePath", "expression": "$fileDirectory.append($fileName )" } } ],
"config": { "log": "debug" },
"source": { "file": { "path": "$filePath" } },
"extractor":
{
"csv": { "ignoreEmptyLines": true,
"nullValue": "N/A",
"separator": ",",
"columnsOnFirstLine": true,
"dateFormat": "yyyy-MM-dd"
}
},
"transformers": [ { "vertex": { "class": "GeoGlyphWKT" } } ],
"loader": {
"orientdb": {
"dbURL": "plocal:nazca-wkt.orientdb",
"dbType": "graph",
"batchCommit": 1000
}
}
}
{
"source": { "file": { "path": "./nazca_lines_wkt.csv" } },
"extractor": { "csv": {
"separator": ",",
"columns": ["Name:String","Location:String"] } },
"transformers": [
{ "command": { "command": "INSERT INTO GeoGlyphWKT(Name,Location) values('${input.Name}', St_GeomFromText('${input.Location}'))"} }
],
"loader": {
"orientdb": {
"dbURL": "plocal:/home/ivan/OrientDB/db_installati/enterprise/orientdb-enterprise-2.2.13/databases/stack40982509-spatial",
"dbUser": "admin",
"dbPassword": "admin",
"dbType": "graph",
"batchCommit": 1000
}
}
}
我使用以下命令运行oetl:
$ oetl.sh commonGeoGlyphWKT.json nazca_lines_wkt.json
但这会导致以下输出失败:
$ oetl.sh commonGeoGlyphWKT.json nazca_lines_wkt.json
OrientDB etl v.2.2.13 (build 2.2.x@r90d7caa1e4af3fad86594e592c64dc1202558ab1; 2016-11-15 12:04:05+0000) www.orientdb.com
BEGIN ETL PROCESSOR
[file] INFO Reading from file ./nazca_lines_wkt.csv with encoding UTF-8
Started execution with 1 worker threads
Error in Pipeline execution: com.orientechnologies.orient.core.exception.OValidationException: impossible to convert value of field "Location"
DB name="nazca-wkt.orientdb"
ETL process has problem: java.util.concurrent.ExecutionException: com.orientechnologies.orient.core.exception.OValidationException: impossible to convert value of field "Location"
DB name="nazca-wkt.orientdb"
END ETL PROCESSOR
+ extracted 9 rows (0 rows/sec) - 9 rows -> loaded 0 vertices (0 vertices/sec) Total time: 16ms [0 warnings, 1 errors]
我肯定我错过了一些愚蠢的事情。。。是否有人能够使用ETL导入包含点、多边形等WKT字符串的CSV文件
感谢您的帮助 这对我有用: commongeogioglyphwkt.json
{
"begin": [ { "let": { "name": "$filePath", "expression": "$fileDirectory.append($fileName )" } } ],
"config": { "log": "debug" },
"source": { "file": { "path": "$filePath" } },
"extractor":
{
"csv": { "ignoreEmptyLines": true,
"nullValue": "N/A",
"separator": ",",
"columnsOnFirstLine": true,
"dateFormat": "yyyy-MM-dd"
}
},
"transformers": [ { "vertex": { "class": "GeoGlyphWKT" } } ],
"loader": {
"orientdb": {
"dbURL": "plocal:nazca-wkt.orientdb",
"dbType": "graph",
"batchCommit": 1000
}
}
}
{
"source": { "file": { "path": "./nazca_lines_wkt.csv" } },
"extractor": { "csv": {
"separator": ",",
"columns": ["Name:String","Location:String"] } },
"transformers": [
{ "command": { "command": "INSERT INTO GeoGlyphWKT(Name,Location) values('${input.Name}', St_GeomFromText('${input.Location}'))"} }
],
"loader": {
"orientdb": {
"dbURL": "plocal:/home/ivan/OrientDB/db_installati/enterprise/orientdb-enterprise-2.2.13/databases/stack40982509-spatial",
"dbUser": "admin",
"dbPassword": "admin",
"dbType": "graph",
"batchCommit": 1000
}
}
}
nazca\u lines\u wkt.csv
Name,Location
Hummingbird,POINT (-75.148892 -14.692131)
Monkey,POINT (-75.138532 -14.706940)
Condor,POINT(-75.126208 -14.697444)
Spider,POINT(-75.122381 -14.694145)
Spiral,POINT(-75.122746 -14.688277)
Hands,POINT(-75.113881 -14.694459)
Tree,POINT(-75.114520 -14.693898)
Astronaut,POINT(-75.079755 -14.745222)
Dog,POINT(-75.130788 -14.706401)
Wing,POINT(-75.100385 -14.680309)
Parrot,POINT(-75.107498 -14.689463)
这对我有用: commongeogioglyphwkt.json
{
"begin": [ { "let": { "name": "$filePath", "expression": "$fileDirectory.append($fileName )" } } ],
"config": { "log": "debug" },
"source": { "file": { "path": "$filePath" } },
"extractor":
{
"csv": { "ignoreEmptyLines": true,
"nullValue": "N/A",
"separator": ",",
"columnsOnFirstLine": true,
"dateFormat": "yyyy-MM-dd"
}
},
"transformers": [ { "vertex": { "class": "GeoGlyphWKT" } } ],
"loader": {
"orientdb": {
"dbURL": "plocal:nazca-wkt.orientdb",
"dbType": "graph",
"batchCommit": 1000
}
}
}
{
"source": { "file": { "path": "./nazca_lines_wkt.csv" } },
"extractor": { "csv": {
"separator": ",",
"columns": ["Name:String","Location:String"] } },
"transformers": [
{ "command": { "command": "INSERT INTO GeoGlyphWKT(Name,Location) values('${input.Name}', St_GeomFromText('${input.Location}'))"} }
],
"loader": {
"orientdb": {
"dbURL": "plocal:/home/ivan/OrientDB/db_installati/enterprise/orientdb-enterprise-2.2.13/databases/stack40982509-spatial",
"dbUser": "admin",
"dbPassword": "admin",
"dbType": "graph",
"batchCommit": 1000
}
}
}
nazca\u lines\u wkt.csv
Name,Location
Hummingbird,POINT (-75.148892 -14.692131)
Monkey,POINT (-75.138532 -14.706940)
Condor,POINT(-75.126208 -14.697444)
Spider,POINT(-75.122381 -14.694145)
Spiral,POINT(-75.122746 -14.688277)
Hands,POINT(-75.113881 -14.694459)
Tree,POINT(-75.114520 -14.693898)
Astronaut,POINT(-75.079755 -14.745222)
Dog,POINT(-75.130788 -14.706401)
Wing,POINT(-75.100385 -14.680309)
Parrot,POINT(-75.107498 -14.689463)
感谢您的报告,作为ETL模块的主要维护者,我从未导入过地理空间数据。让我试试,我会回来回答的谢谢你看!感谢您的报告,作为ETL模块的主要维护者,我从未导入过地理空间数据。让我试试,我会回来回答的谢谢你看!