提交12万个作业失败,抛出Can't create unique job ID异常

问题描述

提交120000个作业,但抛出了异常:Can't create unique job ID. Please make the job id prefix more specific.

集群三个数据节点。

代码如下,其中作业ID为finaloneyearwrite。symbols共有10000个symbol。

for(j in 0..11){          
for(i in 0..(symbols.size()-1)){
rpc(nodes[i%(nodes.size())],submitJob,"finaloneyearwrite","yearWrite",writeData,col_names,col_types,total_range[j],factor_names,symbols[i])
}
}
请先 登录 后评论