Expected a closure; got ... IndexedRDD$MultiputZipper

[tharpa]

While running your example from the README on a recent Spark (2.0-SNAPSHOT) compiled for 2.11.7, I'm seeing the following exception:

16/03/01 19:07:45 WARN ClosureCleaner: Expected a closure; got edu.berkeley.cs.amplab.spark.indexedrdd.IndexedRDD$MultiputZipper
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.SparkContext.runJob(Lorg/apache/spark/rdd/RDD;Lscala/Function2;Lscala/collection/Seq;ZLscala/reflect/ClassTag;)Ljava/lang/Object;
    at edu.berkeley.cs.amplab.spark.indexedrdd.IndexedRDD.multiget(IndexedRDD.scala:83)
    at edu.berkeley.cs.amplab.spark.indexedrdd.IndexedRDD.get(IndexedRDD.scala:76)

I've compiled your package and its dependency (PART) setting scalaVersion to 2.11.7 and published it to the local Ivy repo. Do you think the problem I'm experiencing is related to Scala or Spark version I'm using (I'd like to stay on Scala 2.11 for my project)? If so, what would your roadmap look like for releasing the Scala 2.11 version of your package (see issue #12)? Thanks

[lawben]

I am getting the same warning all over the place:

171433 [JobGenerator] WARN  org.apache.spark.util.ClosureCleaner  - Expected a closure; got edu.berkeley.cs.amplab.spark.indexedrdd.IndexedRDD$CreateUsingIndexZipper
171434 [JobGenerator] WARN  org.apache.spark.util.ClosureCleaner  - Expected a closure; got edu.berkeley.cs.amplab.spark.indexedrdd.IndexedRDD$LeftJoinZipper 

I am working with Scala 2.11.7 from release 0.3.
However, I haven't quite figured out if this an actual problem or if I can just ignore it.

[cfregly]

+1

16/05/24 04:24:30 WARN ClosureCleaner: Expected a closure; got edu.berkeley.cs.amplab.spark.indexedrdd.IndexedRDD$LeftJoinZipper

Scala 2.10, IndexedRdd 0.3 from within a Spark 1.6.1 Streaming Job

my guess is - as the error says - we're referencing something outside of the RDD closure. it's not killing my job, so i'm happy for now, but the WARN is scary!

[ParkL]

+1

16-07-15 18:23:08 WARN Expected a closure; got edu.berkeley.cs.amplab.spark.indexedrdd.IndexedRDD$OtherFullOuterJoinZipper

if it helps: someone filed this issue on the spark JIRA, so technically the warning might be false alarm

https://issues.apache.org/jira/browse/SPARK-11630

Using Scala 2.11, indexedrdd 0.3 and spark 1.5.1

[ycheng1947]

+1
16/11/22 13:34:05 WARN ClosureCleaner: Expected a closure; got edu.berkeley.cs.amplab.spark.indexedrdd.IndexedRDD$MultiputZipper
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.SparkContext.runJob(Lorg/apache/spark/rdd/RDD;Lscala/Function2;Lscala/collection/Seq;ZLscala/reflect/ClassTag;)Ljava/lang/Object;

using scala 2.11.7,indexedRDD 0.3 and spark2.0.0

How to fix the problem ?

[Yishun99]

+1

[linmingze]

+1

[leslieyuan]

+1
17/03/22 15:20:28 WARN ClosureCleaner: Expected a closure; got KafkaStream$$$Lambda$11/1133617880
17/03/22 15:20:28 WARN ClosureCleaner: Expected a closure; got KafkaStream$$$Lambda$12/1742798650
Exception in thread "main" java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: scala/runtime/java8/JFunction2$mcIII$sp

Anyone answer?

[jialing3]

+1

Expected a closure; got edu.berkeley.cs.amplab.spark.indexedrdd.IndexedRDD$FullOuterJoinZipper

[brandonSc]

+1

[Schm1tz1]

+1