class MapperRowCounter extends AccumulatorV2[Long, List[(Integer, Long)]]
An AccumulatorV2 counter for collecting a list of (mapper index, row count).
- Source
- MapperRowCounter.scala
- Since
3.4.0
- Alphabetic
- By Inheritance
- MapperRowCounter
- AccumulatorV2
- Serializable
- Serializable
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Instance Constructors
- new MapperRowCounter()
Value Members
-
def
add(v: Long): Unit
Takes the inputs and accumulates.
Takes the inputs and accumulates.
- Definition Classes
- MapperRowCounter → AccumulatorV2
-
def
copy(): MapperRowCounter
Creates a new copy of this accumulator.
Creates a new copy of this accumulator.
- Definition Classes
- MapperRowCounter → AccumulatorV2
-
def
copyAndReset(): MapperRowCounter
Creates a new copy of this accumulator, which is zero value.
Creates a new copy of this accumulator, which is zero value. i.e. call
isZero
on the copy must return true.- Definition Classes
- MapperRowCounter → AccumulatorV2
-
final
def
id: Long
Returns the id of this accumulator, can only be called after registration.
Returns the id of this accumulator, can only be called after registration.
- Definition Classes
- AccumulatorV2
-
final
def
isRegistered: Boolean
Returns true if this accumulator has been registered.
Returns true if this accumulator has been registered.
- Definition Classes
- AccumulatorV2
- Note
All accumulators must be registered before use, or it will throw exception.
-
def
isZero: Boolean
Returns false if this accumulator has had any values added to it or the sum is non-zero.
Returns false if this accumulator has had any values added to it or the sum is non-zero.
- Definition Classes
- MapperRowCounter → AccumulatorV2
-
def
merge(other: AccumulatorV2[Long, List[(Integer, Long)]]): Unit
Merges another same-type accumulator into this one and update its state, i.e.
Merges another same-type accumulator into this one and update its state, i.e. this should be merge-in-place.
- Definition Classes
- MapperRowCounter → AccumulatorV2
-
final
def
name: Option[String]
Returns the name of this accumulator, can only be called after registration.
Returns the name of this accumulator, can only be called after registration.
- Definition Classes
- AccumulatorV2
-
def
reset(): Unit
Resets this accumulator, which is zero value.
Resets this accumulator, which is zero value. i.e. call
isZero
must return true.- Definition Classes
- MapperRowCounter → AccumulatorV2
- def setPartitionId(id: Integer): Unit
-
def
toString(): String
- Definition Classes
- AccumulatorV2 → AnyRef → Any
-
def
value: List[(Integer, Long)]
Defines the current value of this accumulator
Defines the current value of this accumulator
- Definition Classes
- MapperRowCounter → AccumulatorV2