Object that contains message ksb.csle.common.proto.RunnerProto.SparkRunnerInfo PySparkRunnerInfo containing attributes are as follows:
message SparkRunnerInfo { required string pyEntryPath = 1 [default = "/pyML/autosparkml/bridge/call_trainer.py"]; optional SparkArgs sparkArgs = 1; optional bool inJson = 2 [default = false]; }
message SparkArgs { optional string appName = 1; optional string master = 2 [default = "local[*]"]; optional string deployMode = 3; optional string mainClass = 4; optional string conf = 5; optional string executorCores = 7; optional string memory = 8; optional string driverMemory = 9; optional string executerMemory = 10; optional string numExecutors = 11; optional string totalCores = 12; optional string extraArgs = 13; repeated string confOptions = 14; optional string jars = 15; optional string pyFiles = 16; optional string mainJar = 17; optional string sparkVersion = 18 [default = "2.1.1"]; }
:: ApplicationDeveloperApi ::
Runner that gets session which derived from Spark context created by YARN.