Class

ksb.csle.component.pipe.stream.operator

SelectColumnsPipeOperator

Related Doc: package operator

Permalink

class SelectColumnsPipeOperator extends BasePipeOperator[DataFrame, StreamPipeOperatorInfo, SparkSession]

:: ApplicationDeveloperApi ::

Operator that selects the columns that user wants to choose.

Linear Supertypes
BasePipeOperator[DataFrame, StreamPipeOperatorInfo, SparkSession], BaseGenericPipeOperator[(Any) ⇒ DataFrame, DataFrame, DataFrame, StreamPipeOperatorInfo, SparkSession], BaseGenericMutantOperator[StreamPipeOperatorInfo, (Any) ⇒ DataFrame, (DataFrame) ⇒ DataFrame], BaseDoer, Logging, Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SelectColumnsPipeOperator
  2. BasePipeOperator
  3. BaseGenericPipeOperator
  4. BaseGenericMutantOperator
  5. BaseDoer
  6. Logging
  7. Serializable
  8. Serializable
  9. AnyRef
  10. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new SelectColumnsPipeOperator(o: StreamPipeOperatorInfo, session: SparkSession)

    Permalink

    o

    Object that contains message ksb.csle.common.proto.StreamOperatorProto.SelectColumnsPipeInfo SelectColumnsPipeInfo contains attributes as follows:

    • col: Column ids to be selected (repeated)
    • colName: Column names to be selected (repeated)
    • isRename: set "true" if you want to change the column names to be selected (optional)
    • newName: Column names to be changed (repeated)

    SelectColumnsInfo

    message SelectColumnsPipeInfo {
    repeated int32 col = 1;
    repeated string colName = 2;
    optional bool isRename = 3 [default = false];
    repeated string newName = 4;
    }

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  6. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  10. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  11. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  12. val isRename: Boolean

    Permalink
  13. val logger: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  14. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  15. val newCols: ProtocolStringList

    Permalink
  16. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  17. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  18. def operate(df: (Any) ⇒ DataFrame): (DataFrame) ⇒ DataFrame

    Permalink

    Selects specific columns.

    Selects specific columns.

    df

    Input function pipe

    returns

    output data pipe

    Definition Classes
    SelectColumnsPipeOperator → BasePipeOperator → BaseGenericPipeOperator → BaseGenericMutantOperator
  19. val p: SelectColumnsPipeInfo

    Permalink
  20. val selectedCols: Array[String]

    Permalink
  21. def stop: Unit

    Permalink
    Definition Classes
    BasePipeOperator → BaseGenericPipeOperator → BaseGenericMutantOperator
    Annotations
    @DeveloperApi()
  22. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  23. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  24. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  25. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  26. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from BasePipeOperator[DataFrame, StreamPipeOperatorInfo, SparkSession]

Inherited from BaseGenericPipeOperator[(Any) ⇒ DataFrame, DataFrame, DataFrame, StreamPipeOperatorInfo, SparkSession]

Inherited from BaseGenericMutantOperator[StreamPipeOperatorInfo, (Any) ⇒ DataFrame, (DataFrame) ⇒ DataFrame]

Inherited from BaseDoer

Inherited from Logging

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped