Home » Java » java – Convert JavaRDD<List<SomeClass>> to JavaRDD<SomeClass>-Exceptionshub

java – Convert JavaRDD<List<SomeClass>> to JavaRDD<SomeClass>-Exceptionshub

Posted by: admin February 25, 2020 Leave a comment

Questions:

I have a JavaRDD of a list of class objects. I want to flatten it to a JaveRDD of class objects such that
JavaRDD<{1, 2, 3, 4}, {5, 6, 7}> goes to JavaRDD<1, 2, 3, 4, 5, 6, 7>

in this post
Convert RDD List to RDD of individual element in spark
one solution is

val newrdd = rdd.flatmap(line => line)

however, line=>line is scala (I think)
I tried

rdd.flatmap(line-> line)

and it gives and error

“no instance(s) of type variable(s) U exist so that List conforms to Iterator “

How to&Answers:

The function you pass to flatMap should return a java.util.Iterator<T>, not a List<T>. It should suffice to do this:

JavaRDD<SomeClass> newRdd = rdd.flatMap(List::iterator);