Developer API

org.apache.spark

Class NarrowDependency<T>

java.lang.Object

org.apache.spark.Dependency<T>

org.apache.spark.NarrowDependency<T>

All Implemented Interfaces:

java.io.Serializable

Direct Known Subclasses:

OneToOneDependency, RangeDependency

public abstract class NarrowDependency<T>
extends Dependency<T>

Base class for dependencies where each partition of the child RDD depends on a small number of partitions of the parent RDD. Narrow dependencies allow for pipelined execution.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
NarrowDependency(RDD<T> _rdd)

Method Summary

Methods

Modifier and Type	Method and Description
abstract scala.collection.Seq<java.lang.Object>	getParents(int partitionId) Get the parent partitions for a child partition.
RDD<T>	rdd()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

NarrowDependency

public NarrowDependency(RDD<T> _rdd)

Method Detail

getParents

public abstract scala.collection.Seq<java.lang.Object> getParents(int partitionId)

Get the parent partitions for a child partition.

Parameters:

partitionId - a partition of the child RDD

Returns:

the partitions of the parent RDD that the child partition depends upon

rdd

public RDD<T> rdd()

Specified by:

rdd in class Dependency<T>