Scala IgniteRDD不返回数据帧中的行
我正在IgniteRDD上执行sql查询。对于RDD转换和操作,它运行正常,但是当我在IgniteRDD上调用sql时,它不会返回任何结果。这是我写的确切代码Scala IgniteRDD不返回数据帧中的行,scala,ignite,gridgain,Scala,Ignite,Gridgain,我正在IgniteRDD上执行sql查询。对于RDD转换和操作,它运行正常,但是当我在IgniteRDD上调用sql时,它不会返回任何结果。这是我写的确切代码 private val conf = new SparkConf() .setAppName("IgniteRDDExample") .setMaster("local") .set("spark.executor.instances", "2") /** Spark context */ val sparkContext =
private val conf = new SparkConf()
.setAppName("IgniteRDDExample")
.setMaster("local")
.set("spark.executor.instances", "2")
/** Spark context */
val sparkContext = new SparkContext(conf)
/** Defines spring cache Configuration path */
private val CONFIG = "examples/config/example-shared-rdd.xml"
/** Creates Ignite context with above configuration configuration */
val igniteContext = new IgniteContext(sparkContext, CONFIG, false)
/** Creates an Ignite RDD of Type (Int,Int) Integer Pair */
val sharedRDD: IgniteRDD[Int, Int] = igniteContext.fromCache[Int, Int]("sharedRDD")
/** Fill IgniteRDD with Int pairs */
sharedRDD.savePairs(sparkContext.parallelize(1 to 1000, 10).map(i => (i, i)))
/** Transforming Pairs to contain their Squared value */
sharedRDD.mapValues(x => (x * x))
/** Retrieve sharedRDD back from the Cache */
val transformedValues: IgniteRDD[Int, Int] = igniteContext.fromCache[Int, Int]("sharedRDD")
transformedValues.take(5).foreach(println)
/** Performing SQL query on existing cache and
* collect result into a Spark Dataframe
* */
val rs = transformedValues.sql("select * from Integer where number > ? and number < ? ", 10, 100)
/** Show DataFrame results */
println("The count is::::::::::: "+rs.count)
这里的action.take5在转换后的值上返回结果并打印它。但当我在上面运行sql方法时,它返回0作为行计数。我不知道。请帮忙
以下是我的缓存配置设置:
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="cacheConfiguration">
<!-- SharedRDD cache example configuration (Atomic mode). -->
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<!-- Set a cache name. -->
<property name="name" value="sharedRDD"/>
<!-- Set cache mode. -->
<property name="cacheMode" value="PARTITIONED"/>
<!-- set atomicity -->
<property name="atomicityMode" value="ATOMIC"/>
<!-- Number of backup nodes. -->
<property name="backups" value="1"/>
<property name="queryEntities">
<list>
<bean class="org.apache.ignite.cache.QueryEntity">
<!-- Setting indexed type's key class -->
<property name="keyType" value="Integer"></property>
<!-- Setting indexed type's value class -->
<property name="valueType" value="Integer"></property>
<property name="fields">
<map>
<entry key="number" value="Integer"/>
</map>
</property>
<!--Enable Index on the field -->
<property name="indexes">
<list>
<bean class="org.apache.ignite.cache.QueryIndex">
<constructor-arg value="number"/>
</bean>
</list>
</property>
</bean>
</list>
</property>
</bean>
</property>
<!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<!--
Ignite provides several options for automatic discovery that can be used
instead os static IP based discovery. For information on all options refer
to our documentation: http://apacheignite.readme.io/docs/cluster-config
-->
<!-- Uncomment static IP finder to enable static-based discovery of initial nodes. -->
<!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">-->
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
<property name="addresses">
<list>
<!-- In distributed environment, replace with actual host IP address. -->
<value>127.0.0.1:47500..47509</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
</bean>
</beans>
Integer类中没有数字字段,因此从SQL的角度来看它总是空的。将基本体用作键和/或值时,可以使用预定义的_key和_val字段。因此,查询可以如下所示:
select * from Integer where _val > ? and _val < ?
在这种情况下,也没有太多理由使用queryEntities。setIndexedTypes足够简单了。我将查询更改为从整数中选择*,结果数据框中仍然有0条记录。缓存的配置是否仍有错误?