一段脚本任务如下:
ret=`/data/dolphinscheduler/loadOraTable.sh "yonbip/yonbip@10.16.10.69:1521/orcl" "select t.bondcontractno,t.olcunissuemny from yonbip.bond_contract t " "/dmp/biz" "bip" "2025-08-13"`
echo "$ret"
findCols=`echo $ret |grep "new line cols:"`
echo "findCols: findCols"
colsNum=$(echo $findCols | awk -F 'new line cols:{' '{print $2}' | awk -F '}' '{print $1}')
echo "#{setValue(newLineColNums=${colsNum})}"
echo "newLineColNums: $colsNum"
运行时出现如下错误:
[INFO] 2025-08-14 05:52:33.633 +0000 - raw script : ret=`/data/dolphinscheduler/loadOraTable.sh "yonbip/yonbip@10.16.10.69:1521/orcl" "select t.bondcontractno,t.olcunissuemny from yonbip.bond_contract t " "/dmp/biz" "bip" "2025-08-13"`
echo "$ret"
findCols=`echo $ret |grep "new line cols:"`
if [ -n $findCols ]
then
echo "findCols: findCols"
colsNum=$(echo $findCols | awk -F 'new line cols:{' '{print $2}' | awk -F '}' '{print $1}')
fi
echo "#{setValue(newLineColNums=${colsNum})}"
echo "newLineColNums: $colsNum"
[INFO] 2025-08-14 05:52:33.633 +0000 - task execute path : /tmp/dolphinscheduler/exec/process/hive/18584734972800/18632548623621_8/632/2403
[INFO] 2025-08-14 05:52:33.633 +0000 - Begin to create command file:/tmp/dolphinscheduler/exec/process/hive/18584734972800/18632548623621_8/632/2403/632_2403.command
[INFO] 2025-08-14 05:52:33.633 +0000 - Success create command file, command: #!/bin/bash
BASEDIR=$(cd `dirname $0`; pwd)
cd $BASEDIR
source /opt/datasophon/dolphinscheduler-3.1.8/worker-server/conf/dolphinscheduler_env.sh
/tmp/dolphinscheduler/exec/process/hive/18584734972800/18632548623621_8/632/2403/632_2403_node.sh
[INFO] 2025-08-14 05:52:33.634 +0000 - task run command: sudo -u hive -E bash /tmp/dolphinscheduler/exec/process/hive/18584734972800/18632548623621_8/632/2403/632_2403.command
[INFO] 2025-08-14 05:52:33.635 +0000 - process start, process id is: 74084
[INFO] 2025-08-14 05:52:39.030 +0000 - process has exited. execute path:/tmp/dolphinscheduler/exec/process/hive/18584734972800/18632548623621_8/632/2403, processId:74084 ,exitStatusCode:0 ,processWaitForStatus:true ,processExitValue:0
[ERROR] 2025-08-14 05:52:39.031 +0000 - shell task error
java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.dolphinscheduler.plugin.task.api.parameters.AbstractParameters.getMapByString(AbstractParameters.java:191)
at org.apache.dolphinscheduler.plugin.task.api.parameters.AbstractParameters.dealOutParam(AbstractParameters.java:144)
at org.apache.dolphinscheduler.plugin.task.shell.ShellTask.handle(ShellTask.java:99)
at org.apache.dolphinscheduler.server.worker.runner.DefaultWorkerDelayTaskExecuteRunnable.executeTask(DefaultWorkerDelayTaskExecuteRunnable.java:49)
at org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable.run(WorkerTaskExecuteRunnable.java:174)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
[ERROR] 2025-08-14 05:52:39.031 +0000 - Task execute failed, due to meet an exception
org.apache.dolphinscheduler.plugin.task.api.TaskException: Execute shell task error
at org.apache.dolphinscheduler.plugin.task.shell.ShellTask.handle(ShellTask.java:108)
at org.apache.dolphinscheduler.server.worker.runner.DefaultWorkerDelayTaskExecuteRunnable.executeTask(DefaultWorkerDelayTaskExecuteRunnable.java:49)
at org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable.run(WorkerTaskExecuteRunnable.java:174)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.dolphinscheduler.plugin.task.api.parameters.AbstractParameters.getMapByString(AbstractParameters.java:191)
at org.apache.dolphinscheduler.plugin.task.api.parameters.AbstractParameters.dealOutParam(AbstractParameters.java:144)
at org.apache.dolphinscheduler.plugin.task.shell.ShellTask.handle(ShellTask.java:99)
... 9 common frames omitted
脚本本身运行没出现任何错误,但dolphinscheduler提示错误。后来多次研究发现,如果设置变量值为空时会出现这个异常,就是下面这行用于向dolphinscheduler输出变量的脚本:
echo "#{setValue(newLineColNums=${colsNum})}"
如果colsNum为空,就会出现错误。
于是,加一个判断就能解决问题了:
ret=`/data/dolphinscheduler/loadOraTable.sh "${dbConnect}" "select ${slctColums} from ${SRC_DB}.${tableName} t " "/dmp/${DMP_DB}" "${srcSystem}" "${bizDate}"`
colsNum=$(echo $ret |grep "new line cols:" | awk -F 'new line cols:{' '{print $2}' | awk -F '}' '{print $1}')
if (( -n $colsNum ))
then
echo "#{setValue(newLineColNums=${colsNum})}"
fi
echo "newLineColNums: $colsNum"