Class

ksb.csle.component.controller

WindowedSparkSessionOrStreamController

Related Doc: package controller

Permalink

final class WindowedSparkSessionOrStreamController[T, P, R, SI, SO] extends BaseGenericController[T, StreamControllerInfo, R, SI, SO]

:: ApplicationDeveloperApi ::

Controller that conduct windowing control to queuing data. This controller passes queued data to a consecutive operators. This can be run on streaming engine, and work as a streaming processing controller.

T

Data type class to be passes through data pipeline from reader to operators

P

Message type class

R

Data type for the final data format

SI

Session type class for reader object

SO

Session type class for writer object

Linear Supertypes
BaseGenericController[T, StreamControllerInfo, R, SI, SO], BaseController[T, BaseResult], BaseDoer, Logging, Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. WindowedSparkSessionOrStreamController
  2. BaseGenericController
  3. BaseController
  4. BaseDoer
  5. Logging
  6. Serializable
  7. Serializable
  8. AnyRef
  9. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new WindowedSparkSessionOrStreamController(runner: BaseRunner[SI, _, _], i: BaseReader[T, _, SI], o: BaseWriter[R, _, SO], p: StreamControllerInfo, ops: List[BaseGenericOperator[_, R]])

    Permalink

    runner

    Runner object that runs spark session

    i

    Reader object that reads data from data source

    o

    Writer object that writes pipelined data from operators

    p

    Message object that has one of messages object containing attributes related to controlling process pipelining

    ops

    a list of BaseGenericOperator objects that processes data one by one.

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. var bufferArray: Array[Row]

    Permalink
  6. var bufferCount: Int

    Permalink
  7. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  8. var df: DataFrame

    Permalink
  9. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  10. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  11. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  12. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  13. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  14. val i: BaseReader[T, _, SI]

    Permalink

    Reader object that reads data from data source

    Reader object that reads data from data source

    Definition Classes
    WindowedSparkSessionOrStreamController → BaseGenericController
  15. def init(): Any

    Permalink
    Definition Classes
    BaseController
    Annotations
    @DeveloperApi()
  16. var inputDf: DataFrame

    Permalink
  17. val inputQueSize: Int

    Permalink
  18. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  19. val logger: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  20. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  21. var newRows: Array[Row]

    Permalink
  22. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  23. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  24. val o: BaseWriter[R, _, SO]

    Permalink

    Writer object that writes pipelined data from operators

    Writer object that writes pipelined data from operators

    Definition Classes
    WindowedSparkSessionOrStreamController → BaseGenericController
  25. val ops: List[BaseGenericOperator[_, R]]

    Permalink

    a list of BaseGenericOperator objects that processes data one by one.

    a list of BaseGenericOperator objects that processes data one by one.

    Definition Classes
    WindowedSparkSessionOrStreamController → BaseGenericController
  26. val p: StreamControllerInfo

    Permalink

    Message object that has one of messages object containing attributes related to controlling process pipelining

    Message object that has one of messages object containing attributes related to controlling process pipelining

    Definition Classes
    WindowedSparkSessionOrStreamController → BaseGenericController
  27. final def pipeOperators(runner: BaseRunner[_, _, _], ops: List[BaseGenericOperator[_, R]], r: R): R

    Permalink
    Attributes
    protected
    Definition Classes
    BaseGenericController
    Annotations
    @tailrec()
  28. def process(): BaseResult

    Permalink
    Definition Classes
    WindowedSparkSessionOrStreamController → BaseGenericController
  29. def progress(): BaseResult

    Permalink
    Definition Classes
    BaseGenericController → BaseController
  30. var resultDf: DataFrame

    Permalink
  31. var rowCountToSend: Int

    Permalink
  32. val runner: BaseRunner[SI, _, _]

    Permalink

    Runner object that runs spark session

    Runner object that runs spark session

    Definition Classes
    WindowedSparkSessionOrStreamController → BaseGenericController
  33. val ssIn: SparkSession

    Permalink
  34. val ssOut: SparkSession

    Permalink
  35. val sscIn: StreamingContext

    Permalink
  36. val sscOut: StreamingContext

    Permalink
  37. def stop(): Any

    Permalink
    Definition Classes
    WindowedSparkSessionOrStreamController → BaseGenericController → BaseController
  38. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  39. var tempArray4Engine: Array[Row]

    Permalink
  40. var tempArray4Predict: Array[Row]

    Permalink
  41. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  42. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  43. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  44. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  45. val windowSize: Int

    Permalink
  46. val x: WindowedStreamControllerInfo

    Permalink

Inherited from BaseGenericController[T, StreamControllerInfo, R, SI, SO]

Inherited from BaseController[T, BaseResult]

Inherited from BaseDoer

Inherited from Logging

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped