FLINKCDC连接oracle导致归档日志暴增

发布于:2024-07-29 ⋅ 阅读:(111) ⋅ 点赞:(0)

前言

前段时间再用flinkcdc连接oracle的时候,oracle的归档日志疯狂的飙升,我经常去到归档目录下查看占用的内存情况。。

情况

在使用flinkcdc连接oracle的时候,发现归档日志飙升,查看了很多文档,该配置了都配置了,

# 连接状态的时候,回一直飙升
root@1b612c91b33b:/u01/app/oracle/archive# du -h
674M    .

flinkcdc程序

        Properties properties = new Properties();
        properties.setProperty("debezium.database.tablename.case.insensitive", "false");
        properties.setProperty("debezium.log.mining.strategy", "online_catalog");
//        properties.setProperty("database.tablename.case.insensitive", "false");
//        properties.setProperty("log.mining.strategy", "online_catalog");
        //持续挖掘日志
        properties.setProperty("debezium.log.mining.continuous.mine", "true");
        SourceFunction<String> sourceFunction = OracleSource.<String>builder()
                .hostname("192.168.1.9")
                .port(1521)
                .database("XE") // monitor XE database
                .schemaList("flinkuser") // monitor inventory schema
                .tableList("flinkuser.SYS_USER") // monitor products table
                .username("flinkuser")
                .password("flinkpw")
                .startupOptions(StartupOptions.initial())
                .debeziumProperties(properties)
                .deserializer(new JsonDebeziumDeserializationSchema()) // converts SourceRecord to JSON String
                .build();
        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
        DataStreamSource<String> sourceStream = env.addSource(sourceFunction).setParallelism(1);// use parallelism 1 for sink to keep message ordering
        SingleOutputStreamOperator<String> streamOperator = sourceStream
                .filter(StringUtils::isNotEmpty);
        streamOperator.setParallelism(1).print();
         env.execute();

问题解决

问题是怎么解决的呢,将flinkcdc的代码修改一下,,

        properties.setProperty("debezium.database.tablename.case.insensitive", "false");
        properties.setProperty("debezium.log.mining.strategy", "online_catalog");
		换成下面两句。。
        properties.setProperty("database.tablename.case.insensitive", "false");
         properties.setProperty("log.mining.strategy", "online_catalog");

一些文章中,都会在配置文件上带上debezium的。。。而这种时候是不需要这个前缀的


网站公告

今日签到

点亮在社区的每一天
去签到