Debian11之DolphinScheduler使用

登录

默认用户名和密码 admin/dolphinscheduler123

http://192.168.111.180:12345/dolphinscheduler/ui

基础配置

1、创建Worker【admin用户下】

• 创建项目的时候会指定Worker，这个配置决定了项目中的任务在哪个服务器执行
  

2、创建环境【admin用户下】

- 如果涉及到三方应用的协助工作，例如：通过datax做数据同步，则需要在工作节点安装python、datax，安装成功后需要在相关环境下配置应用的路径

export JAVA_HOME=/usr/local/jdk/jdk1.8.0_141
export PYTHON_LAUNCHER=/usr/local/python2.7/bin/python2.7
export DATAX_LAUNCHER=/mydata/soft/datax/bin/datax.py
export PATH=$PYTHON_LAUNCHER:$JAVA_HOME/bin:$DATAX_LAUNCHER:$PATH

3、创建队列【admin用户下】

4、创建租户[admin用户下]

• 创建租户其实就是在工作节点服务器下创建一个用户，用于为执行项目分配linux权限
  

5、创建用户[admin用户下]

创建项目[普通用户下]

为项目授权【admin用户下】

为项目创建WorkerGroup【admin用户下】

案例一：MySql数据同步【多表+单数据】

自定义数据

demo1数据库

CREATE TABLE `oa_order` (
  `id` int NOT NULL AUTO_INCREMENT,
  `user_key` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci DEFAULT NULL,
  `show_order_no` varchar(255) COLLATE utf8mb4_general_ci DEFAULT NULL,
  `erp_order_no` varchar(255) COLLATE utf8mb4_general_ci DEFAULT NULL,
  `remark` varchar(255) COLLATE utf8mb4_general_ci DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci;

INSERT INTO `demo1`.`oa_order` (`id`, `user_key`, `show_order_no`, `erp_order_no`, `remark`) VALUES (1, '310110', '显示订单号1', 'erp订单号1', '订单备注1');
INSERT INTO `demo1`.`oa_order` (`id`, `user_key`, `show_order_no`, `erp_order_no`, `remark`) VALUES (2, '310110', '显示订单号2', 'erp订单号2', '订单备注2');

demo2数据库

CREATE TABLE `oa_order` (
  `id` int NOT NULL AUTO_INCREMENT,
  `user_key` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci DEFAULT NULL,
  `show_order_no` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci DEFAULT NULL,
  `erp_order_no` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci DEFAULT NULL,
  `remark` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci;

创建工作流

查询oa_order

添加oa_order

保存工作流

启动工作流

• 工作流启动成功后会创建一个工作流实例，分别运行 添加oa_order 和 查询oa_order任务
  
  

工作流日志

案例二：MySql数据同步【多表+多数据】

前提条件

• 基于DataX三方插件实现，需要提前在工作节点上安装
• DataX依赖于Python，需要提前在工作节点上安装
• 当前项目所属的环境对象下需要指定三方插件信息

export dolphinscheduler=lixing
export JAVA_HOME=/usr/local/java/jdk1.8.0_301
export PYTHON_LAUNCHER=/usr/local/bin/python
export DATAX_LAUNCHER=/usr/local/datax/datax/bin/datax.py
export PATH=$PYTHON_LAUNCHER:$JAVA_HOME/bin:$DATAX_LAUNCHER:$PATH

自定义数据

demo1数据库

CREATE TABLE `oa_order` (
  `id` int NOT NULL AUTO_INCREMENT,
  `user_key` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci DEFAULT NULL,
  `show_order_no` varchar(255) COLLATE utf8mb4_general_ci DEFAULT NULL,
  `erp_order_no` varchar(255) COLLATE utf8mb4_general_ci DEFAULT NULL,
  `remark` varchar(255) COLLATE utf8mb4_general_ci DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci;

INSERT INTO `demo1`.`oa_order` (`id`, `user_key`, `show_order_no`, `erp_order_no`, `remark`) VALUES (1, '310110', '显示订单号1', 'erp订单号1', '订单备注1');
INSERT INTO `demo1`.`oa_order` (`id`, `user_key`, `show_order_no`, `erp_order_no`, `remark`) VALUES (2, '310110', '显示订单号2', 'erp订单号2', '订单备注2');

CREATE TABLE `oa_order_address` (
  `id` int NOT NULL AUTO_INCREMENT,
  `user_key` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci DEFAULT NULL,
  `erp_order_no` varchar(255) COLLATE utf8mb4_general_ci DEFAULT NULL,
  `name` varchar(255) COLLATE utf8mb4_general_ci DEFAULT NULL,
  `address` varchar(255) COLLATE utf8mb4_general_ci DEFAULT NULL,
  `address2` varchar(255) COLLATE utf8mb4_general_ci DEFAULT NULL,
  `phone` varchar(255) COLLATE utf8mb4_general_ci DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci;

INSERT INTO `demo1`.`oa_order_address` (`id`, `user_key`, `erp_order_no`, `name`, `address`, `address2`, `phone`) VALUES (1, '310110', 'erp订单号1', '张三', '妮敦道', '19号', '18013896707');

demo2数据库

CREATE TABLE `oa_order_info` (
  `id` int NOT NULL AUTO_INCREMENT,
  `user_key` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci DEFAULT NULL,
  `show_order_no` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci DEFAULT NULL,
  `erp_order_no` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci DEFAULT NULL,
  `remark` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci DEFAULT NULL,
  `name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci DEFAULT NULL,
  `address` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci DEFAULT NULL,
  `address2` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci DEFAULT NULL,
  `phone` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci;

创建工作流

关联查询oa_order和oa_order_address

保存工作流

启动工作流

工作流日志