Class

ksb.csle.component.operator.transformation

TimeSynchronizeOperator

Related Doc: package transformation

Permalink

class TimeSynchronizeOperator extends BaseGenericOperator[StreamOperatorInfo, DataFrame]

:: ApplicationDeveloperApi ::

Operator that joins two dataframes according to timestamp. The dataframe to be joined (second dataframe) is loaded by csv file format.

Linear Supertypes
BaseGenericOperator[StreamOperatorInfo, DataFrame], BaseGenericMutantOperator[StreamOperatorInfo, DataFrame, DataFrame], BaseDoer, Logging, Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. TimeSynchronizeOperator
  2. BaseGenericOperator
  3. BaseGenericMutantOperator
  4. BaseDoer
  5. Logging
  6. Serializable
  7. Serializable
  8. AnyRef
  9. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new TimeSynchronizeOperator(o: StreamOperatorInfo)

    Permalink

    o

    Object that contains message ksb.csle.common.proto.StreamOperatorProto.TimeSynchronizationInfo TimeSynchronizationInfo contains attributes as follows:

    • hasTimeDelay: Parameter to define whether second dataframe has a time delay (required)
    • isAutoMode: If this parameter is true, time delay is calculated automatically (optional) (as smallest difference between two dataframes). If this parameter is false, next parameter is used.
    • timeDelay: Paremeter to specify timeDelay if above parameter is false (optional)
    • interpolationInfo: interpolation operator of first dataframe (required)
    • isTimeStampColumn: Parameter to define whether selected column is timestamp type (required) If this parameter is false, next parameter is used.
    • userTimeStampPattern: Parameter to specify the timestamp pattern of selected column if above parameter is false (optional)
    • joinDataInfo: Parameter to specify the parameters of second dataframe for reading and interpolation. (required) ksb.csle.common.proto.StreamOperatorProto.TimeSyncDataFrameInfo TimeSyncDataFrameInfo contains attributes as follows:
    • filePath: File path of second dataframe (required)
    • hasHeader: Parameter to define whether csv file has header (optional)
    • separator: separator of csv file (optional)
    • interpolationInfo: interpolation operator of second dataframe (required)
    • isTimeStampColumn: Parameter to define whether selected column is timestamp type (required) If this parameter is false, next parameter is used.
    • userTimeStampPattern: Parameter to specify the timestamp pattern of selected column if above parameter is false (optional) ksb.csle.common.proto.StreamOperatorProto.InterpolationInfo InterpolationInfo contains attributes as follows:
    • indexColumnName: Column name of timestamp column (required)
    • stepSize: Intervals of time range, unit is second (optional)
    • methodOption: Interpolation method (required)
    • keepColumnType: Parameter to define whether to keep the originals column type (optional) After interpolation option, the column type is changed to double type.

    InterpolationInfo

    message InterpolationInfo {
    required string indexColumnName = 1;
    required int32 stepSize = 2 [default = 1];
    required InterpolationMethodOption methodOption = 3 [default = LINEAR];
    optional bool keepColumnType = 4 [default = false];
    }

    TimeSyncDataFrameInfo

    message TimeSyncDataFrameInfo {
    required string filePath = 1;
    optional bool hasHeader = 2 [default = true];
    optional string separator = 3 [default = ","];
    required InterpolationInfo interpolationInfo = 4;
    required bool isTimeStampColumn = 5 [default = true];
    optional string userTimeStampPattern = 6 [default = "yyyy-mm-dd hh:mm:ss"];
    }

    TimeSynchronizationInfo

    message TimeSynchronizationInfo {
    required bool hasTimeDelay = 1 [default = false];
    optional bool isAutoMode = 2 [default = true];
    optional int32 timeDelay = 3;
    required InterpolationInfo interpolationInfo = 4;
    required bool isTimeStampColumn = 5 [default = true];
    optional string userTimeStampPattern = 6 [default = "yyyy-mm-dd hh:mm:ss"];
    required TimeSyncDataFrameInfo joinDataInfo = 7;
    }

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  6. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  10. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  11. val interpolationOp_df1: StreamOperatorInfo

    Permalink
  12. val interpolationOp_df2: StreamOperatorInfo

    Permalink
  13. val interpolation_df1: InterpolateOperator

    Permalink
  14. val interpolation_df2: InterpolateOperator

    Permalink
  15. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  16. val logger: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  17. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  18. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  19. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  20. def operate(df: DataFrame): DataFrame

    Permalink
    Definition Classes
    TimeSynchronizeOperator → BaseGenericOperator → BaseGenericMutantOperator
  21. val p: TimeSynchronizationInfo

    Permalink
  22. def stop: Unit

    Permalink
    Definition Classes
    TimeSynchronizeOperator → BaseGenericOperator → BaseGenericMutantOperator
  23. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  24. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  25. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  26. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  27. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from BaseGenericOperator[StreamOperatorInfo, DataFrame]

Inherited from BaseGenericMutantOperator[StreamOperatorInfo, DataFrame, DataFrame]

Inherited from BaseDoer

Inherited from Logging

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped