Using Spring for Apache Pulsar:Publishing and Consuming Partitioned Topics

发布于:2025-07-15 ⋅ 阅读:(11) ⋅ 点赞:(0)

在下面的示例中,我们发布了一个名为hello pulser participated的主题。这是一个被分区的主题,对于这个示例,我们假设该主题已经创建了三个分区。

@SpringBootApplication
public class PulsarBootPartitioned {

	public static void main(String[] args) {
		SpringApplication.run(PulsarBootPartitioned.class, "--spring.pulsar.producer.message-routing-mode=CustomPartition");
	}

	@Bean
	public ApplicationRunner runner(PulsarTemplate<String> pulsarTemplate) {
		pulsarTemplate.setDefaultTopicName("hello-pulsar-partitioned");
		return args -> {
			for (int i = 0; i < 10; i++) {
				pulsarTemplate.sendAsync("hello john doe 0 ", new FooRouter());
				pulsarTemplate.sendAsync("hello alice doe 1", new BarRouter());
				pulsarTemplate.sendAsync("hello buzz doe 2", new BuzzRouter());
			}
		};
	}

	@PulsarListener(subscriptionName = "hello-pulsar-partitioned-subscription", topics = "hello-pulsar-partitioned")
	public void listen(String message) {
		System.out.println("Message Received: " + message);
	}

    static class FooRouter implements MessageRouter {

		@Override
		public int choosePartition(Message<?> msg, TopicMetadata metadata) {
			return 0;
		}
	}

	static class BarRouter implements MessageRouter {

		@Override
		public int choosePartition(Message<?> msg, TopicMetadata metadata) {
			return 1;
		}
	}

	static class BuzzRouter implements MessageRouter {

		@Override
		public int choosePartition(Message<?> msg, TopicMetadata metadata) {
			return 2;
		}
	}

}

在前面的示例中,我们发布到一个分区的主题,我们想将一些数据段发布到特定的分区。如果您将其保留为Pulsar的默认值,它将遵循分区分配的轮转模式,我们希望覆盖该模式。为此,我们提供了一个带有send方法的消息路由器对象。考虑实现的三个消息路由器。FooRouter始终将数据发送到分区0,BarRouter发送到分区1,BuzzRouter发送给分区2。还要注意,我们现在使用PulsarTemplate的sendAsync方法,该方法返回CompletableFuture。运行应用程序时,我们还需要将生产者上的messageRoutingMode设置为CustomPartition(spring.pulsinger.producer.message路由模式)。

在消费者端,我们使用具有独占订阅类型的PulsarListener。这意味着来自所有分区的数据最终都在同一个消费者中,并且没有订购保证。

如果我们希望每个分区由一个不同的消费者使用,我们该怎么办?我们可以切换到故障转移订阅模式,并添加三个单独的消费者:

@PulsarListener(subscriptionName = "hello-pulsar-partitioned-subscription", topics = "hello-pulsar-partitioned", subscriptionType = SubscriptionType.Failover)
public void listen1(String foo) {
    System.out.println("Message Received 1: " + foo);
}

@PulsarListener(subscriptionName = "hello-pulsar-partitioned-subscription", topics = "hello-pulsar-partitioned", subscriptionType = SubscriptionType.Failover)
public void listen2(String foo) {
    System.out.println("Message Received 2: " + foo);
}

@PulsarListener(subscriptionName = "hello-pulsar-partitioned-subscription",  topics = "hello-pulsar-partitioned", subscriptionType = SubscriptionType.Failover)
public void listen3(String foo) {
    System.out.println("Message Received 3: " + foo);
}

当你遵循这种方法时,一个分区总是被一个专用的消费者占用。

同样,如果你想使用Pulsar的共享消费者类型,你可以使用共享订阅类型。但是,当您使用共享模式时,您将失去任何排序保证,因为单个消费者可能会在另一个消费者有机会之前收到来自所有分区的消息。

考虑以下示例:

@PulsarListener(subscriptionName = "hello-pulsar-shared-subscription", topics = "hello-pulsar-partitioned", subscriptionType = SubscriptionType.Shared)
public void listen1(String foo) {
    System.out.println("Message Received 1: " + foo);
}

@PulsarListener(subscriptionName = "hello-pulsar-shared-subscription", topics = "hello-pulsar-partitioned", subscriptionType = SubscriptionType.Shared)
public void listen2(String foo) {
    System.out.println("Message Received 2: " + foo);
}


网站公告

今日签到

点亮在社区的每一天
去签到