last day (20 days later) » 

6:09 PM
1
A: Can Flink attach multiple jobs to Stream local envirnoment with Web Ui by java code?

Till RohrmannYou should be able to re-use the created StreamExecutionEnvironment which you have stored in env. StreamExecutionEnvironment.createLocalEnvironmentWithWebUI won't set the respective context environment factory. Therefore, StreamExecutionEnvironment.getExecutionEnvironment won't return the create...

 
yes, I did try to resue it, but I get error, saying that the address bind is already in use, I think env try to use the same jobmanager address for some reason. I will update my questoin with the code and error message soon. Thanks for help.
 
You're right. The problem is that the underlying LocalStreamEnvironment creates for each execute call a new LocalFlinkMiniCluster using the same ports. This might cause the problems. Are you trying to concurrently run Flink jobs on the same local cluster?
 
Yes I am trying to run multiple jobs in one local local cluster.
 
Concurrently or one after each other?
 
In fact, at first, I was running multiple LocalFlinkMiniCluster on different prot, each of them has only one job. However, it consume a lot of RAM spend, I am think to run all the job in one LocalFlinkMiniCluster.
I try to run them concurrently , not one after each other
I think maybe I should use JobClient to submit multiple jobs?
 
6:09 PM
Running them concurrently won't work because the the LocalFlinkMiniCluster will only be started with as many slots as are needed for the first job to run. What you can do though, is to start a LocalFlinkMiniCluster and then use a RemoteStreamEnvironment to submit jobs to this cluster. However, be aware that you have to start the cluster with as many slots as you need for all your jobs to be run concurrently. Alternatively you can also combine all jobs into one. Then the operators can share slots.
 
Thank you Till Rohmann, I will try the RemoteStreamEnviroment solution first.
I tried to use RemoteStreamEnvironment to submit the task but I can not connect the jobManager of the LocalFlinkMiniCluster, I got this Trying to connect to localhost/127.0.0.1:29011 2017-05-11 14:06:48.308 INFO 9496 --- [cTaskExecutor-3] o.a.flink.runtime.net.ConnectionUtils : Failed to connect from address 'tong-iQware/127.0.1.1': Connection refused (Connection refused) 2017-05-11 14:06:48.309 INFO 9496 --- [cTaskExecutor-3] o.a.flink.runtime.net.ConnectionUtils : Failed to connect from address '/127.0.0.1': Connection refused (Connection refused)
        config.setInteger(ConfigConstants.JOB_MANAGER_WEB_PORT_KEY, 29010;
        config.setInteger(ConfigConstants.JOB_MANAGER_IPC_PORT_KEY,  29010 + 1);
this is my local Mini Cluster
 

  last day (20 days later) »