一年一度由世界知名科技媒体 InfoWorld 评选的 Bossie Awards 于 2018 年 9 月 26 日公布,其中,曾连续两年入选最佳开源数据库与数据分析平台奖的 Kafka 意外遭遇滑铁卢落选,取而代之的是新兴项目 Pulsar。
Bossie Awards 中对 Pulsar 点评如下:
Pulsar 旨在取代 Apache Kafka 多年的主宰地位。 Pulsar 在很多情况下提供了比 Kafka 更快的吞吐量和更低的延迟,并为开发人员提供了一组兼容的 API,让他们可以很轻松地从 Kafka 切换到 Pulsar。Pulsar 的最大优点在于它提供了比 Apache Kafka 更简单明了、更健壮的一系列操作功能,特别在解决可观察性、地域复制和多租户方面的问题。在运行大型 Kafka 集群方面感觉有困难的企业可以考虑转向使用 Pulsar。
那么,Apache Pulsar 和 Apache Kafka 到底有什么不同?在下面这篇文章中,Jack Vanlightly 通过测试数据,直观地呈现出 Pulsar 的运行状况,并与 Kafka 进行了对比说明。
作者:Jack Vanlightly
原文:
https://jack-vanlightly.com/blog/2019/9/4/a-look-at-multi-topic-subscriptions-with-apache-pulsar
本文是另一篇《关于 Apache Kafka 的多主题订阅》的姊妹篇,移步 CloudKarafka 博客可以阅读这篇文章。在开始讨论 Pulsar 多主题订阅之前,我会先对 Kafka 多主题订阅的测试结果进行简短摘要,因为针对这两项技术的测试方法,是完全相同的。
我们希望可以通过测试,更好地了解我们可以在多主题订阅的场景下得到怎样的结果,着重测试了消息排序。消息排序(Message Ordering)是消息系统的基础,尽管 Pulsar 或 Kafka 都无法确保跨主题场景下的消息有序,但探索它能达到的最好程度,依然是非常有趣且有益的过程。
Apache Kafka 测试摘要
在针对 Apache Pulsar 实施的测试中,我注意到,只要 Kafka 的 consumer 可以“实时”地消费消息,就可以完美地按顺序传递消息。这里提到的“实时”,是指当消息发布到 topic 上,consumer 可以第一时间完成消息消费,即即时消费。处理消息的过程中,consumer 发生任何抖动,或者在发布消息的过程中,producer 速度过快,都会导致消息失去排序。就好像从 2 个 topic 的起始位置读取消息,producer 根本是无序可言的。
理论上讲,Kafka 的 consumer 从订阅的 topic 获取消息时,遵从既定排序。这似乎适用于 consumer 的整个处理过程,例如,当 consumer 调用 poll() 函数时,既可能从 topic1 获得响应,也可能从 topic2 获得响应。
当 consumer 消费消息的速度与 producer 发送消息的速度基本保持同步,且未消费的消息总数小于 max.poll.records 时,我们可以在跨主题场景下看到非常良好的消息排序,甚至可以说是完美排序。
不过一旦你失去了严格意义上的实时消费,比如,未消费的消息总数大于 max.poll.records 时,我们将看到大规模的延伸,此时,consumer 只从 1 个 topic 获取消息。
例如,consumer 提取消息的顺序是:
-
topic2, partition 0
-
topic1, partition 0
假设每个 topic 包含 700 条消息,max.poll.records 设置为 500,并且从时间偏移最小的 topic 开始,那么消息获取的顺序为:
-
poll() - 来自 topic2, partition 0 的 500 条消息
-
poll() - 来自 topic2, partition 0 的 200 条消息;以及来自 topic1, partition 0 的 300 条消息
-
poll() - 来自 topic1, partition 0 的 400 条消息
这就意味着,如果出现以下任何一种情况,就不要指望在跨主题场景下,消息仍保持正确排序:
-
producer 发送消息的速度,远远大于 consumer 消费消息的速度;
-
从最新的 topic 消费消息;
-
重新设置偏移时间;
-
存在 1 个被不时唤醒的进程,从 2 个或多个 topic 进行消息消费。
我们并不是在批评 Kafka,单纯是对其行为规律进行研究。可以看出,在跨分区的场景下,Kafka 无法保证正确的消息排序。
深入研究 Apache Pulsar
那么 Apache Pulsar 的情况又是怎样的呢?总体说来,Pulsar 在跨主题场景下进行消息有序传递的情况要优于 Kafka,但是当实时消费无法完美实现时,也会出现消息乱序的情况。不过,Pulsar 缓存消息的方式,对于处理过程中可能出现的抖动具有更高的宽容度,最大限度地保证了消息传递的正确排序。
与 Apache Kafka 的测试相同,我们使用 1 个 producer 向 2 个 topic 发送带有 2 个计数器的消息队列:
-
全局计数器:每发送 1 条消息,计数器递增;
-
topic 计数器:topic 自有计数器,当有消息发送至 topic 时,计数器递增。
全局计数器用于创建全局消息序列,显示跨主题场景下的消息排序状况;topic 计数器用于显示指定 topic 是否以正确的顺序传递消息。与 Kafka 的测试结果相同,我们得到的 topic 排序应该总是完美的,但跨主题的全局排序则不一定。
事实也正如所料,所有测试都得到了完美的 topic 排序。
例如,producer 按以下顺序发送消息:
-
Topic 1 -> { “GlobalCounter”: 1, “TopicCounter”: 1 }
-
Topic 2 -> { “GlobalCounter”: 2, “TopicCounter”: 1 }
-
Topic 1 -> { “GlobalCounter”: 3, “TopicCounter”: 2 }
-
Topic 2 -> { “GlobalCounter”: 4, “TopicCounter”: 2 }
-
Topic 1 -> { “GlobalCounter”: 5, “TopicCounter”: 3 }
-
Topic 2 -> { “GlobalCounter”: 6, “TopicCounter”: 3 }
-
Topic 1 -> { “GlobalCounter”: 7, “TopicCounter”: 4 }
-
Topic 2 -> { “GlobalCounter”: 8, “TopicCounter”: 4 }
-
Topic 1 -> { “GlobalCounter”: 9, “TopicCounter”: 5 }
-
Topic 2 -> { “GlobalCounter”: 10, “TopicCounter”: 5 }
topic 1 和 topic 2 如下图所示:
图1: 以轮询方式发送至 topic 的消息
测试策略
首先说明一下测试中用到的术语:
-
尾随 topic 和 时间旅行:
尾随 topic(Tailing the topic)是指 consumer在消息到达 topic 时,第一时间消费信息。由于实时性的扰动会对消息排序产生影响,因此尾随 topic 场景可细分为很多种情况。时间旅行(Time-travel)是指从 topic 的起始位置开始消费消息,本质上是读取过去的消息。
-
消息等比到达/消息不等比到达:
消息等比到达(Equal message arrival rate)是指 producer 以轮询方式向 topic 发送消息,每个 topic 以相同速度接收消息。消息不等比到达(Unequal message arrival rate)是指不同 topic接收消息的速度不同。例如,当 topic 1 接收到 1 条消息时,topic2 已接收到 10 条消息。
-
0 处理时长/恒定时长/抖动时长:
0 处理时长(Zero processing time)、恒定时长(Constant time)和抖动时长(Jitter)是指consumer 处理消息所需时长。
-
快速 producer:
快速 producer(Fast producer)是指发送消息速度超过 consumer 消费消息速度的 producer。也就是说,在快速 producer 场景下,consumer 无法进行严格的实时消费。
经过几天的测试和思考,分别对下列场景进行观察:
-
尾随 topic,消息等比到达,0 处理时长
-
尾随 topic,消息等比到达,短恒定时长
-
尾随 topic,消息等比到达,短抖动时长
-
尾随 topic,消息等比到达,快速 producer
-
尾随 topic,消息不等比到达,0 处理时长
-
尾随 topic,消息不等比到达,短恒定时长
-
尾随 topic,消息不等比到达,短抖动时长
-
尾随 topic,消息不等比到达,快速 producer
-
时间旅行,消息等比到达
-
时间旅行,消息不等比到达
测试 1:尾随 topic,消息等比到达,0 处理时长
在测试 1 的场景下,producer 以尽可能快的速度,将消息按照等比轮询的方式,同步发送到 2 个 topic,确保消息按顺序到达。consumer 只需对消息排序进行检查,实际处理时长为 0。
20:31:25.230Z : Consumer : Subscribing
20:31:25.691Z : Consumer : Consuming
20:31:25.967Z : Producer : Topic: topic1, TC: 1, GC: 1
20:31:25.972Z : Consumer : Topic: topic1, TC: 1 OK, GC: 1 OK
20:31:25.974Z : Producer : Topic: topic2, TC: 1, GC: 2
20:31:25.975Z : Consumer : Topic: topic2, TC: 1 OK, GC: 2 OK
20:31:25.979Z : Producer : Topic: topic1, TC: 2, GC: 3
20:31:25.981Z : Consumer : Topic: topic1, TC: 2 OK, GC: 3 OK
20:31:25.985Z : Producer : Topic: topic2, TC: 2, GC: 4
20:31:25.987Z : Consumer : Topic: topic2, TC: 2 OK, GC: 4 OK
20:31:25.991Z : Producer : Topic: topic1, TC: 3, GC: 5
20:31:25.992Z : Consumer : Topic: topic1, TC: 3 OK, GC: 5 OK
20:31:25.997Z : Producer : Topic: topic2, TC: 3, GC: 6
20:31:25.998Z : Consumer : Topic: topic2, TC: 3 OK, GC: 6 OK
20:31:26.002Z : Producer : Topic: topic1, TC: 4, GC: 7
20:31:26.003Z : Consumer : Topic: topic1, TC: 4 OK, GC: 7 OK
20:31:26.007Z : Producer : Topic: topic2, TC: 4, GC: 8
20:31:26.009Z : Consumer : Topic: topic2, TC: 4 OK, GC: 8 OK
20:31:26.013Z : Producer : Topic: topic1, TC: 5, GC: 9
20:31:26.014Z : Consumer : Topic: topic1, TC: 5 OK, GC: 9 OK
20:31:26.018Z : Producer : Topic: topic2, TC: 5, GC: 10
20:31:26.020Z : Consumer : Topic: topic2, TC: 5 OK, GC: 10 OK
20:31:26.023Z : Producer : Topic: topic1, TC: 6, GC: 11
20:31:26.025Z : Consumer : Topic: topic1, TC: 6 OK, GC: 11 OK
20:31:26.027Z : Producer : Topic: topic2, TC: 6, GC: 12
20:31:26.031Z : Consumer : Topic: topic2, TC: 6 OK, GC: 12 OK
20:31:26.033Z : Producer : Topic: topic1, TC: 7, GC: 13
20:31:26.036Z : Consumer : Topic: topic1, TC: 7 OK, GC: 13 OK
20:31:26.039Z : Producer : Topic: topic2, TC: 7, GC: 14
20:31:26.042Z : Consumer : Topic: topic2, TC: 7 OK, GC: 14 OK
20:31:26.044Z : Producer : Topic: topic1, TC: 8, GC: 15
20:31:26.047Z : Consumer : Topic: topic1, TC: 8 OK, GC: 15 OK
20:31:26.049Z : Producer : Topic: topic2, TC: 8, GC: 16
20:31:26.054Z : Consumer : Topic: topic2, TC: 8 OK, GC: 16 OK
20:31:26.054Z : Producer : Topic: topic1, TC: 9, GC: 17
20:31:26.058Z : Consumer : Topic: topic1, TC: 9 OK, GC: 17 OK
20:31:26.060Z : Producer : Topic: topic2, TC: 9, GC: 18
20:31:26.065Z : Consumer : Topic: topic2, TC: 9 OK, GC: 18 OK
20:31:26.065Z : Producer : Topic: topic1, TC: 10, GC: 19
20:31:26.069Z : Consumer : Topic: topic1, TC: 10 OK, GC: 19 OK
20:31:26.071Z : Producer : Topic: topic2, TC: 10, GC: 20
20:31:26.076Z : Consumer : Topic: topic2, TC: 10 OK, GC: 20 OK
20:31:26.076Z : Producer : Topic: topic1, TC: 11, GC: 21
20:31:26.080Z : Consumer : Topic: topic1, TC: 11 OK, GC: 21 OK
20:31:26.082Z : Producer : Topic: topic2, TC: 11, GC: 22
20:31:26.087Z : Consumer : Topic: topic2, TC: 11 OK, GC: 22 OK
20:31:26.087Z : Producer : Topic: topic1, TC: 12, GC: 23
20:31:26.090Z : Consumer : Topic: topic1, TC: 12 OK, GC: 23 OK
20:31:26.092Z : Producer : Topic: topic2, TC: 12, GC: 24
20:31:26.097Z : Producer : Topic: topic1, TC: 13, GC: 25
20:31:26.098Z : Consumer : Topic: topic2, TC: 12 OK, GC: 24 OK
20:31:26.102Z : Consumer : Topic: topic1, TC: 13 OK, GC: 25 OK
20:31:26.103Z : Producer : Topic: topic2, TC: 13, GC: 26
20:31:26.107Z : Producer : Topic: topic1, TC: 14, GC: 27
20:31:26.109Z : Consumer : Topic: topic2, TC: 13 OK, GC: 26 OK
20:31:26.112Z : Producer : Topic: topic2, TC: 14, GC: 28
20:31:26.113Z : Consumer : Topic: topic1, TC: 14 OK, GC: 27 OK
20:31:26.117Z : Producer : Topic: topic1, TC: 15, GC: 29
20:31:26.120Z : Consumer : Topic: topic2, TC: 14 OK, GC: 28 OK
20:31:26.122Z : Producer : Topic: topic2, TC: 15, GC: 30
20:31:26.123Z : Consumer : Topic: topic1, TC: 15 OK, GC: 29 OK
20:31:26.127Z : Producer : Topic: topic1, TC: 16, GC: 31
20:31:26.131Z : Consumer : Topic: topic2, TC: 15 OK, GC: 30 OK
20:31:26.132Z : Producer : Topic: topic2, TC: 16, GC: 32
20:31:26.135Z : Consumer : Topic: topic1, TC: 16 OK, GC: 31 OK
20:31:26.137Z : Producer : Topic: topic1, TC: 17, GC: 33
20:31:26.142Z : Consumer : Topic: topic2, TC: 16 OK, GC: 32 OK
20:31:26.142Z : Producer : Topic: topic2, TC: 17, GC: 34
20:31:26.145Z : Consumer : Topic: topic1, TC: 17 OK, GC: 33 OK
20:31:26.148Z : Producer : Topic: topic1, TC: 18, GC: 35
20:31:26.153Z : Producer : Topic: topic2, TC: 18, GC: 36
20:31:26.154Z : Consumer : Topic: topic2, TC: 17 OK, GC: 34 OK
20:31:26.154Z : Consumer : Topic: topic2, TC: 18 OK, GC: 36 JUMP FORWARD 1
20:31:26.157Z : Consumer : Topic: topic1, TC: 18 OK, GC: 35 JUMP BACKWARDS 1
20:31:26.158Z : Producer : Topic: topic1, TC: 19, GC: 37
20:31:26.163Z : Producer : Topic: topic2, TC: 19, GC: 38
20:31:26.165Z : Consumer : Topic: topic2, TC: 19 OK, GC: 38 JUMP FORWARD 2
20:31:26.168Z : Consumer : Topic: topic1, TC: 19 OK, GC: 37 JUMP BACKWARDS 1
20:31:26.168Z : Producer : Topic: topic1, TC: 20, GC: 39
20:31:26.173Z : Producer : Topic: topic2, TC: 20, GC: 40
20:31:26.176Z : Consumer : Topic: topic2, TC: 20 OK, GC: 40 JUMP FORWARD 2
20:31:26.178Z : Producer : Topic: topic1, TC: 21, GC: 41
20:31:26.178Z : Consumer : Topic: topic1, TC: 20 OK, GC: 39 JUMP BACKWARDS 1
20:31:26.179Z : Consumer : Topic: topic1, TC: 21 OK, GC: 41 JUMP FORWARD 1
20:31:26.185Z : Producer : Topic: topic2, TC: 21, GC: 42
20:31:26.187Z : Consumer : Topic: topic2, TC: 21 OK, GC: 42 OK
20:31:26.190Z : Producer : Topic: topic1, TC: 22, GC: 43
20:31:26.191Z : Consumer : Topic: topic1, TC: 22 OK, GC: 43 OK
20:31:26.196Z : Producer : Topic: topic2, TC: 22, GC: 44
20:31:26.198Z : Consumer : Topic: topic2, TC: 22 OK, GC: 44 OK
20:31:26.201Z : Producer : Topic: topic1, TC: 23, GC: 45
20:31:26.203Z : Consumer : Topic: topic1, TC: 23 OK, GC: 45 OK
20:31:26.206Z : Producer : Topic: topic2, TC: 23, GC: 46
20:31:26.209Z : Consumer : Topic: topic2, TC: 23 OK, GC: 46 OK
20:31:26.211Z : Producer : Topic: topic1, TC: 24, GC: 47
20:31:26.214Z : Consumer : Topic: topic1, TC: 24 OK, GC: 47 OK
20:31:26.216Z : Producer : Topic: topic2, TC: 24, GC: 48
20:31:26.221Z : Consumer : Topic: topic2, TC: 24 OK, GC:48 OK
20:31:26.221Z :Producer :Topic: topic1, TC:25, GC:49
20:31:26.225Z :Consumer :Topic: topic1, TC:25 OK, GC:49 OK
20:31:26.227Z :Producer :Topic: topic2, TC:25, GC:50
20:31:26.232Z :Producer :Topic: topic1, TC:26, GC:51
20:31:26.232Z :Consumer :Topic: topic2, TC:25 OK, GC:50 OK
20:31:26.236Z :Consumer :Topic: topic1, TC:26 OK, GC:51 OK
20:31:26.238Z :Producer :Topic: topic2, TC:26, GC:52
20:31:26.243Z :Consumer :Topic: topic2, TC:26 OK, GC:52 OK
20:31:26.243Z :Producer :Topic: topic1, TC:27, GC:53
20:31:26.247Z :Consumer :Topic: topic1, TC:27 OK, GC:53 OK
20:31:26.249Z :Producer :Topic: topic2, TC:27, GC:54
20:31:26.254Z :Consumer :Topic: topic2, TC:27 OK, GC:54 OK
20:31:26.255Z :Producer :Topic: topic1, TC:28, GC:55
20:31:26.257Z :Consumer :Topic: topic1, TC:28 OK, GC:55 OK
20:31:26.260Z :Producer :Topic: topic2, TC:28, GC:56
20:31:26.265Z :Producer :Topic: topic1, TC:29, GC:57
20:31:26.265Z :Consumer :Topic: topic2, TC:28 OK, GC:56 OK
20:31:26.269Z :Consumer :Topic: topic1, TC:29 OK, GC:57 OK
20:31:26.272Z :Producer :Topic: topic2, TC:29, GC:58
20:31:26.276Z :Consumer :Topic: topic2, TC:29 OK, GC:58 OK
20:31:26.276Z :Producer :Topic: topic1, TC:30, GC:59
20:31:26.280Z :Consumer :Topic: topic1, TC:30 OK, GC:59 OK
20:31:26.282Z :Producer :Topic: topic2, TC:30, GC:60
20:31:26.287Z :Producer :Topic: topic1, TC:31, GC:61
20:31:26.287Z :Consumer :Topic: topic2, TC:30 OK, GC:60 OK
20:31:26.291Z :Consumer :Topic: topic1, TC:31 OK, GC:61 OK
20:31:26.293Z :Producer :Topic: topic2, TC:31, GC:62
20:31:26.298Z :Consumer :Topic: topic2, TC:31 OK, GC:62 OK
20:31:26.299Z :Producer :Topic: topic1, TC:32, GC:63
20:31:26.302Z :Consumer :Topic: topic1, TC:32 OK, GC:63 OK
20:31:26.305Z :Producer :Topic: topic2, TC:32, GC:64
20:31:26.310Z :Consumer :Topic: topic2, TC:32 OK, GC:64 OK
20:31:26.314Z :Producer :Topic: topic1, TC:33, GC:65
20:31:26.315Z :Consumer :Topic: topic1, TC:33 OK, GC:65 OK
20:31:26.320Z :Producer :Topic: topic2, TC:33, GC:66
20:31:26.321Z :Consumer :Topic: topic2, TC:33 OK, GC:66 OK
20:31:26.325Z :Producer :Topic: topic1, TC:34, GC:67
20:31:26.326Z :Consumer :Topic: topic1, TC:34 OK, GC:67 OK
20:31:26.331Z :Producer :Topic: topic2, TC:34, GC:68
20:31:26.332Z :Consumer :Topic: topic2, TC:34 OK, GC:68 OK
20:31:26.336Z :Producer :Topic: topic1, TC:35, GC:69
20:31:26.337Z :Consumer :Topic: topic1, TC:35 OK, GC:69 OK
20:31:26.341Z :Producer :Topic: topic2, TC:35, GC:70
20:31:26.342Z :Consumer :Topic: topic2, TC:35 OK, GC:70 OK
20:31:26.348Z :Producer :Topic: topic1, TC:36, GC:71
20:31:26.348Z :Consumer :Topic: topic1, TC:36 OK, GC:71 OK
20:31:26.354Z :Producer :Topic: topic2, TC:36, GC:72
20:31:26.354Z :Consumer :Topic: topic2, TC:36 OK, GC:72 OK
20:31:26.359Z :Producer :Topic: topic1, TC:37, GC:73
20:31:26.359Z :Consumer :Topic: topic1, TC:37 OK, GC:73 OK
20:31:26.364Z :Producer :Topic: topic2, TC:37, GC:74
20:31:26.365Z :Consumer :Topic: topic2, TC:37 OK, GC:74 OK
20:31:26.370Z :Producer :Topic: topic1, TC:38, GC:75
20:31:26.370Z :Consumer :Topic: topic1, TC:38 OK, GC:75 OK
20:31:26.375Z :Producer :Topic: topic2, TC:38, GC:76
20:31:26.376Z :Consumer :Topic: topic2, TC:38 OK, GC:76 OK
20:31:26.399Z :Producer :Topic: topic1, TC:39, GC:77
20:31:26.399Z :Consumer :Topic: topic1, TC:39 OK, GC:77 OK
20:31:26.406Z :Producer :Topic: topic2, TC:39, GC:78
20:31:26.407Z :Consumer :Topic: topic2, TC:39 OK, GC:78 OK
20:31:26.412Z :Producer :Topic: topic1, TC:40, GC:79
20:31:26.412Z :Consumer :Topic: topic1, TC:40 OK, GC:79 OK
20:31:26.417Z :Producer :Topic: topic2, TC:40, GC:80
20:31:26.418Z :Consumer :Topic: topic2, TC:40 OK, GC:80 OK
20:31:26.422Z :Producer :Topic: topic1, TC:41, GC:81
20:31:26.423Z :Consumer :Topic: topic1, TC:41 OK, GC:81 OK
20:31:26.427Z :Producer :Topic: topic2, TC:41, GC:82
20:31:26.429Z :Consumer :Topic: topic2, TC:41 OK, GC:82 OK
20:31:26.432Z :Producer :Topic: topic1, TC:42, GC:83
20:31:26.434Z :Consumer :Topic: topic1, TC:42 OK, GC:83 OK
20:31:26.437Z :Producer :Topic: topic2, TC:42, GC:84
20:31:26.440Z :Consumer :Topic: topic2, TC:42 OK, GC:84 OK
20:31:26.443Z :Producer :Topic: topic1, TC:43, GC:85
20:31:26.445Z :Consumer :Topic: topic1, TC:43 OK, GC:85 OK
20:31:26.449Z :Producer :Topic: topic2, TC:43, GC:86
20:31:26.451Z :Consumer :Topic: topic2, TC:43 OK, GC:86 OK
20:31:26.455Z :Producer :Topic: topic1, TC:44, GC:87
20:31:26.455Z :Consumer :Topic: topic1, TC:44 OK, GC:87 OK
20:31:26.460Z :Producer :Topic: topic2, TC:44, GC:88
20:31:26.462Z :Consumer :Topic: topic2, TC:44 OK, GC:88 OK
20:31:26.465Z :Producer :Topic: topic1, TC:45, GC:89
20:31:26.467Z :Consumer :Topic: topic1, TC:45 OK, GC:89 OK
20:31:26.470Z :Producer :Topic: topic2, TC:45, GC:90
20:31:26.473Z :Consumer :Topic: topic2, TC:45 OK, GC:90 OK
20:31:26.475Z :Producer :Topic: topic1, TC:46, GC:91
20:31:26.478Z :Consumer :Topic: topic1, TC:46 OK, GC:91 OK
20:31:26.479Z :Producer :Topic: topic2, TC:46, GC:92
20:31:26.484Z :Producer :Topic: topic1, TC:47, GC:93
20:31:26.484Z :Consumer :Topic: topic2, TC:46 OK, GC:92 OK
20:31:26.489Z :Producer :Topic: topic2, TC:47, GC:94
20:31:26.489Z :Consumer :Topic: topic1, TC:47 OK, GC:93 OK
20:31:26.494Z :Producer :Topic: topic1, TC:48, GC:95
20:31:26.495Z :Consumer :Topic: topic2, TC:47 OK, GC:94 OK
20:31:26.499Z :Producer :Topic: topic2, TC:48, GC:96
20:31:26.500Z :Consumer :Topic: topic1, TC:48 OK, GC:95 OK
20:31:26.504Z :Producer :Topic: topic1, TC:49, GC:97
20:31:26.506Z :Consumer :Topic: topic2, TC:48 OK, GC:96 OK
20:31:26.509Z :Producer :Topic: topic2, TC:49, GC:98
20:31:26.511Z :Consumer :Topic: topic1, TC:49 OK, GC:97 OK
20:31:26.513Z :Producer :Topic: topic1, TC:50, GC:99
20:31:26.517Z :Consumer :Topic: topic2, TC:49 OK, GC:98 OK
20:31:26.519Z :Producer :Topic: topic2, TC:50, GC:100
20:31:26.522Z :Consumer :Topic: topic1, TC:50 OK, GC:99 OK
20:31:26.528Z :Consumer :Topic: topic2, TC:50 OK, GC:100 OK
测试结果显示,在 100 条被 consumer 消费的消息中,有 93 条消息的全局计数器显示为“OK”,另有 7 条消息为乱序。虽然本轮测试没有得到完美的消息排序,但已非常接近完美状态,在全局序列中没有出现大规模前后跳跃的情况。
测试 2:尾随 topic,消息等比到达,短恒定时长
在测试 1 中,producer 以尽可能快的速度向 2 个 topic 同步发送消息,而 consumer 则尽可能快地消费消息。在本轮测试中,我们分别给 producer 和 consumer 插入 10 ms 的短暂间隔。
20:38:35.722Z :Consumer C1 :Subscribing
20:38:36.183Z :Consumer C1 :Consuming
20:38:36.463Z :Producer :Topic: topic1, TC:1, GC:1
20:38:36.468Z :Consumer C1 :Topic: topic1, TC:1 OK, GC:1 OK
20:38:36.481Z :Producer :Topic: topic2, TC:1, GC:2
20:38:36.483Z :Consumer C1 :Topic: topic2, TC:1 OK, GC:2 OK
20:38:36.498Z :Producer :Topic: topic1, TC:2, GC:3
20:38:36.500Z :Consumer C1 :Topic: topic1, TC:2 OK, GC:3 OK
20:38:36.515Z :Producer :Topic: topic2, TC:2, GC:4
20:38:36.517Z :Consumer C1 :Topic: topic2, TC:2 OK, GC:4 OK
20:38:36.531Z :Producer :Topic: topic1, TC:3, GC:5
20:38:36.531Z :Consumer C1 :Topic: topic1, TC:3 OK, GC:5 OK
20:38:36.547Z :Producer :Topic: topic2, TC:3, GC:6
20:38:36.548Z :Consumer C1 :Topic: topic2, TC:3 OK, GC:6 OK
20:38:36.563Z :Producer :Topic: topic1, TC:4, GC:7
20:38:36.564Z :Consumer C1 :Topic: topic1, TC:4 OK, GC:7 OK
20:38:36.579Z :Producer :Topic: topic2, TC:4, GC:8
20:38:36.580Z :Consumer C1 :Topic: topic2, TC:4 OK, GC:8 OK
20:38:36.595Z :Producer :Topic: topic1, TC:5, GC:9
20:38:36.596Z :Consumer C1 :Topic: topic1, TC:5 OK, GC:9 OK
20:38:36.611Z :Producer :Topic: topic2, TC:5, GC:10
20:38:36.612Z :Consumer C1 :Topic: topic2, TC:5 OK, GC:10 OK
20:38:36.627Z :Producer :Topic: topic1, TC:6, GC:11
20:38:36.628Z :Consumer C1 :Topic: topic1, TC:6 OK, GC:11 OK
20:38:36.643Z :Producer :Topic: topic2, TC:6, GC:12
20:38:36.643Z :Consumer C1 :Topic: topic2, TC:6 OK, GC:12 OK
20:38:36.659Z :Producer :Topic: topic1, TC:7, GC:13
20:38:36.660Z :Consumer C1 :Topic: topic1, TC:7 OK, GC:13 OK
20:38:36.676Z :Producer :Topic: topic2, TC:7, GC:14
20:38:36.676Z :Consumer C1 :Topic: topic2, TC:7 OK, GC:14 OK
20:38:36.692Z :Producer :Topic: topic1, TC:8, GC:15
20:38:36.693Z :Consumer C1 :Topic: topic1, TC:8 OK, GC:15 OK
20:38:36.709Z :Producer :Topic: topic2, TC:8, GC:16
20:38:36.709Z :Consumer C1 :Topic: topic2, TC:8 OK, GC:16 OK
20:38:36.725Z :Producer :Topic: topic1, TC:9, GC:17
20:38:36.725Z :Consumer C1 :Topic: topic1, TC:9 OK, GC:17 OK
20:38:36.742Z :Producer :Topic: topic2, TC:9, GC:18
20:38:36.743Z :Consumer C1 :Topic: topic2, TC:9 OK, GC:18 OK
20:38:36.757Z :Producer :Topic: topic1, TC:10, GC:19
20:38:36.758Z :Consumer C1 :Topic: topic1, TC:10 OK, GC:19 OK
20:38:36.774Z :Producer :Topic: topic2, TC:10, GC:20
20:38:36.775Z :Consumer C1 :Topic: topic2, TC:10 OK, GC:20 OK
20:38:36.790Z :Producer :Topic: topic1, TC:11, GC:21
20:38:36.791Z :Consumer C1 :Topic: topic1, TC:11 OK, GC:21 OK
20:38:36.806Z :Producer :Topic: topic2, TC:11, GC:22
20:38:36.807Z :Consumer C1 :Topic: topic2, TC:11 OK, GC:22 OK
20:38:36.822Z :Producer :Topic: topic1, TC:12, GC:23
20:38:36.822Z :Consumer C1 :Topic: topic1, TC:12 OK, GC:23 OK
20:38:36.838Z :Producer :Topic: topic2, TC:12, GC:24
20:38:36.839Z :Consumer C1 :Topic: topic2, TC:12 OK, GC:24 OK
20:38:36.853Z :Producer :Topic: topic1, TC:13, GC:25
20:38:36.854Z :Consumer C1 :Topic: topic1, TC:13 OK, GC:25 OK
20:38:36.869Z :Producer :Topic: topic2, TC:13, GC:26
20:38:36.869Z :Consumer C1 :Topic: topic2, TC:13 OK, GC:26 OK
20:38:36.885Z :Producer :Topic: topic1, TC:14, GC:27
20:38:36.886Z :Consumer C1 :Topic: topic1, TC:14 OK, GC:27 OK
20:38:36.901Z :Producer :Topic: topic2, TC:14, GC:28
20:38:36.902Z :Consumer C1 :Topic: topic2, TC:14 OK, GC:28 OK
20:38:36.917Z :Producer :Topic: topic1, TC:15, GC:29
20:38:36.918Z :Consumer C1 :Topic: topic1, TC:15 OK, GC:29 OK
20:38:36.933Z :Producer :Topic: topic2, TC:15, GC:30
20:38:36.933Z :Consumer C1 :Topic: topic2, TC:15 OK, GC:30 OK
20:38:36.949Z :Producer :Topic: topic1, TC:16, GC:31
20:38:36.950Z :Consumer C1 :Topic: topic1, TC:16 OK, GC:31 OK
20:38:36.965Z :Producer :Topic: topic2, TC:16, GC:32
20:38:36.966Z :Consumer C1 :Topic: topic2, TC:16 OK, GC:32 OK
20:38:36.981Z :Producer :Topic: topic1, TC:17, GC:33
20:38:36.982Z :Consumer C1 :Topic: topic1, TC:17 OK, GC:33 OK
20:38:36.997Z :Producer :Topic: topic2, TC:17, GC:34
20:38:36.998Z :Consumer C1 :Topic: topic2, TC:17 OK, GC:34 OK
20:38:37.013Z :Producer :Topic: topic1, TC:18, GC:35
20:38:37.014Z :Consumer C1 :Topic: topic1, TC:18 OK, GC:35 OK
20:38:37.029Z :Producer :Topic: topic2, TC:18, GC:36
20:38:37.030Z :Consumer C1 :Topic: topic2, TC:18 OK, GC:36 OK
20:38:37.045Z :Producer :Topic: topic1, TC:19, GC:37
20:38:37.046Z :Consumer C1 :Topic: topic1, TC:19 OK, GC:37 OK
20:38:37.061Z :Producer :Topic: topic2, TC:19, GC:38
20:38:37.062Z :Consumer C1 :Topic: topic2, TC:19 OK, GC:38 OK
20:38:37.077Z :Producer :Topic: topic1, TC:20, GC:39
20:38:37.078Z :Consumer C1 :Topic: topic1, TC:20 OK, GC:39 OK
20:38:37.093Z :Producer :Topic: topic2, TC:20, GC:40
20:38:37.094Z :Consumer C1 :Topic: topic2, TC:20 OK, GC:40 OK
20:38:37.109Z :Producer :Topic: topic1, TC:21, GC:41
20:38:37.110Z :Consumer C1 :Topic: topic1, TC:21 OK, GC:41 OK
20:38:37.126Z :Producer :Topic: topic2, TC:21, GC:42
20:38:37.127Z :Consumer C1 :Topic: topic2, TC:21 OK, GC:42 OK
20:38:37.142Z :Producer :Topic: topic1, TC:22, GC:43
20:38:37.143Z :Consumer C1 :Topic: topic1, TC:22 OK, GC:43 OK
20:38:37.158Z :Producer :Topic: topic2, TC:22, GC:44
20:38:37.159Z :Consumer C1 :Topic: topic2, TC:22 OK, GC:44 OK
20:38:37.174Z :Producer :Topic: topic1, TC:23, GC:45
20:38:37.175Z :Consumer C1 :Topic: topic1, TC:23 OK, GC:45 OK
20:38:37.190Z :Producer :Topic: topic2, TC:23, GC:46
20:38:37.191Z :Consumer C1 :Topic: topic2, TC:23 OK, GC:46 OK
20:38:37.206Z :Producer :Topic: topic1, TC:24, GC:47
20:38:37.207Z :Consumer C1 :Topic: topic1, TC:24 OK, GC:47 OK
20:38:37.222Z :Producer :Topic: topic2, TC:24, GC:48
20:38:37.223Z :Consumer C1 :Topic: topic2, TC:24 OK, GC:48 OK
20:38:37.238Z :Producer :Topic: topic1, TC:25, GC:49
20:38:37.239Z :Consumer C1 :Topic: topic1, TC:25 OK, GC:49 OK
20:38:37.254Z :Producer :Topic: topic2, TC:25, GC:50
20:38:37.255Z :Consumer C1 :Topic: topic2, TC:25 OK, GC:50 OK
20:38:37.270Z :Producer :Topic: topic1, TC:26, GC:51
20:38:37.271Z :Consumer C1 :Topic: topic1, TC:26 OK, GC:51 OK
20:38:37.286Z :Producer :Topic: topic2, TC:26, GC:52
20:38:37.287Z :Consumer C1 :Topic: topic2, TC:26 OK, GC:52 OK
20:38:37.301Z :Producer :Topic: topic1, TC:27, GC:53
20:38:37.302Z :Consumer C1 :Topic: topic1, TC:27 OK, GC:53 OK
20:38:37.318Z :Producer :Topic: topic2, TC:27, GC:54
20:38:37.319Z :Consumer C1 :Topic: topic2, TC:27 OK, GC:54 OK
20:38:37.334Z :Producer :Topic: topic1, TC:28, GC:55
20:38:37.335Z :Consumer C1 :Topic: topic1, TC:28 OK, GC:55 OK
20:38:37.350Z :Producer :Topic: topic2, TC:28, GC:56
20:38:37.350Z :Consumer C1 :Topic: topic2, TC:28 OK, GC:56 OK
20:38:37.366Z :Producer :Topic: topic1, TC:29, GC:57
20:38:37.366Z :Consumer C1 :Topic: topic1, TC:29 OK, GC:57 OK
20:38:37.381Z :Producer :Topic: topic2, TC:29, GC:58
20:38:37.382Z :Consumer C1 :Topic: topic2, TC:29 OK, GC:58 OK
20:38:37.398Z :Producer :Topic: topic1, TC:30, GC:59
20:38:37.398Z :Consumer C1 :Topic: topic1, TC:30 OK, GC:59 OK
20:38:37.414Z :Producer :Topic: topic2, TC:30, GC:60
20:38:37.414Z :Consumer C1 :Topic: topic2, TC:30 OK, GC:60 OK
20:38:37.430Z :Producer :Topic: topic1, TC:31, GC:61
20:38:37.431Z :Consumer C1 :Topic: topic1, TC:31 OK, GC:61 OK
20:38:37.447Z :Producer :Topic: topic2, TC:31, GC:62
20:38:37.447Z :Consumer C1 :Topic: topic2, TC:31 OK, GC:62 OK
20:38:37.462Z :Producer :Topic: topic1, TC:32, GC:63
20:38:37.463Z :Consumer C1 :Topic: topic1, TC:32 OK, GC:63 OK
20:38:37.478Z :Producer :Topic: topic2, TC:32, GC:64
20:38:37.480Z :Consumer C1 :Topic: topic2, TC:32 OK, GC:64 OK
20:38:37.493Z :Producer :Topic: topic1, TC:33, GC:65
20:38:37.494Z :Consumer C1 :Topic: topic1, TC:33 OK, GC:65 OK
20:38:37.510Z :Producer :Topic: topic2, TC:33, GC:66
20:38:37.511Z :Consumer C1 :Topic: topic2, TC:33 OK, GC:66 OK
20:38:37.526Z :Producer :Topic: topic1, TC:34, GC:67
20:38:37.527Z :Consumer C1 :Topic: topic1, TC:34 OK, GC:67 OK
20:38:37.541Z :Producer :Topic: topic2, TC:34, GC:68
20:38:37.542Z :Consumer C1 :Topic: topic2, TC:34 OK, GC:68 OK
20:38:37.556Z :Producer :Topic: topic1, TC:35, GC:69
20:38:37.557Z :Consumer C1 :Topic: topic1, TC:35 OK, GC:69 OK
20:38:37.571Z :Producer :Topic: topic2, TC:35, GC:70
20:38:37.572Z :Consumer C1 :Topic: topic2, TC:35 OK, GC:70 OK
20:38:37.586Z :Producer :Topic: topic1, TC:36, GC:71
20:38:37.587Z :Consumer C1 :Topic: topic1, TC:36 OK, GC:71 OK
20:38:37.603Z :Producer :Topic: topic2, TC:36, GC:72
20:38:37.603Z :Consumer C1 :Topic: topic2, TC:36 OK, GC:72 OK
20:38:37.619Z :Producer :Topic: topic1, TC:37, GC:73
20:38:37.619Z :Consumer C1 :Topic: topic1, TC:37 OK, GC:73 OK
20:38:37.634Z :Producer :Topic: topic2, TC:37, GC:74
20:38:37.635Z :Consumer C1 :Topic: topic2, TC:37 OK, GC:74 OK
20:38:37.650Z :Producer :Topic: topic1, TC:38, GC:75
20:38:37.651Z :Consumer C1 :Topic: topic1, TC:38 OK, GC:75 OK
20:38:37.666Z :Producer :Topic: topic2, TC:38, GC:76
20:38:37.666Z :Consumer C1 :Topic: topic2, TC:38 OK, GC:76 OK
20:38:37.682Z :Producer :Topic: topic1, TC:39, GC:77
20:38:37.682Z :Consumer C1 :Topic: topic1, TC:39 OK, GC:77 OK
20:38:37.698Z :Producer :Topic: topic2, TC:39, GC:78
20:38:37.698Z :Consumer C1 :Topic: topic2, TC:39 OK, GC:78 OK
20:38:37.714Z :Producer :Topic: topic1, TC:40, GC:79
20:38:37.715Z :Consumer C1 :Topic: topic1, TC:40 OK, GC:79 OK
20:38:37.730Z :Producer :Topic: topic2, TC:40, GC:80
20:38:37.730Z :Consumer C1 :Topic: topic2, TC:40 OK, GC:80 OK
20:38:37.746Z :Producer :Topic: topic1, TC:41, GC:81
20:38:37.747Z :Consumer C1 :Topic: topic1, TC:41 OK, GC:81 OK
20:38:37.762Z :Producer :Topic: topic2, TC:41, GC:82
20:38:37.763Z :Consumer C1 :Topic: topic2, TC:41 OK, GC:82 OK
20:38:37.778Z :Producer :Topic: topic1, TC:42, GC:83
20:38:37.778Z :Consumer C1 :Topic: topic1, TC:42 OK, GC:83 OK
20:38:37.794Z :Producer :Topic: topic2, TC:42, GC:84
20:38:37.795Z :Consumer C1 :Topic: topic2, TC:42 OK, GC:84 OK
20:38:37.810Z :Producer :Topic: topic1, TC:43, GC:85
20:38:37.811Z :Consumer C1 :Topic: topic1, TC:43 OK, GC:85 OK
20:38:37.826Z :Producer :Topic: topic2, TC:43, GC:86
20:38:37.827Z :Consumer C1 :Topic: topic2, TC:43 OK, GC:86 OK
20:38:37.842Z :Producer :Topic: topic1, TC:44, GC:87
20:38:37.842Z :Consumer C1 :Topic: topic1, TC:44 OK, GC:87 OK
20:38:37.859Z :Producer :Topic: topic2, TC:44, GC:88
20:38:37.860Z :Consumer C1 :Topic: topic2, TC:44 OK, GC:88 OK
20:38:37.875Z :Producer :Topic: topic1, TC:45, GC:89
20:38:37.876Z :Consumer C1 :Topic: topic1, TC:45 OK, GC:89 OK
20:38:37.891Z :Producer :Topic: topic2, TC:45, GC:90
20:38:37.891Z :Consumer C1 :Topic: topic2, TC:45 OK, GC:90 OK
20:38:37.906Z :Producer :Topic: topic1, TC:46, GC:91
20:38:37.907Z :Consumer C1 :Topic: topic1, TC:46 OK, GC:91 OK
20:38:37.922Z :Producer :Topic: topic2, TC:46, GC:92
20:38:37.923Z :Consumer C1 :Topic: topic2, TC:46 OK, GC:92 OK
20:38:37.937Z :Producer :Topic: topic1, TC:47, GC:93
20:38:37.938Z :Consumer C1 :Topic: topic1, TC:47 OK, GC:93 OK
20:38:37.953Z :Producer :Topic: topic2, TC:47, GC:94
20:38:37.953Z :Consumer C1 :Topic: topic2, TC:47 OK, GC:94 OK
20:38:37.969Z :Producer :Topic: topic1, TC:48, GC:95
20:38:37.970Z :Consumer C1 :Topic: topic1, TC:48 OK, GC:95 OK
20:38:37.985Z :Producer :Topic: topic2, TC:48, GC:96
20:38:37.985Z :Consumer C1 :Topic: topic2, TC:48 OK, GC:96 OK
20:38:38Z :Producer :Topic: topic1, TC:49, GC:97
20:38:38.001Z :Consumer C1 :Topic: topic1, TC:49 OK, GC:97 OK
20:38:38.016Z :Producer :Topic: topic2, TC:49, GC:98
20:38:38.016Z :Consumer C1 :Topic: topic2, TC:49 OK, GC:98 OK
20:38:38.032Z :Producer :Topic: topic1, TC:50, GC:99
20:38:38.032Z :Consumer C1 :Topic: topic1, TC:50 OK, GC:99 OK
20:38:38.047Z :Producer :Topic: topic2, TC:50, GC:100
20:38:38.047Z :Consumer C1 :Topic: topic2, TC:50 OK, GC:100 OK
测试结果表明,在跨主题场景下,Pulsar 至少可以在测试环境中,按顺序处理消息,尽管扩展到实际应用场景中,仍有待进一步观察。在本轮测试中,我们得到了完美的消息交互:发送-消费-发送-消费,即为严格的实时消费。
测试 3:尾随 topic,消息等比到达,短抖动时长
在测试 2 中,我们看到,增加处理时间可以得到完美的消息排序。
如果我们给 consumer 增加随机的抖动处理时长,结果又会如何?我们在测试 2 中看到的“发送-消费”式消息交错应该不会再现。
假设 producer 发送消息的处理时间为 25 ms;consumer 消费消息的处理时间为 1 ms 到 50 ms,峰值可达 200 ms。在同等情况下,Kafka 的 consumer 会造成部分消息乱序。到目前为止,Kafka 已经从完美的消息排序,变为乱序频出,下面我们来看看 Pulsar 的情况如何:
20:33:12.808Z :Consumer :Subscribing
20:33:13.264Z :Consumer :Consuming
20:33:13.538Z :Producer :Topic: topic1, TC:1, GC:1
20:33:13.541Z :Consumer :Topic: topic1, TC:1 OK, GC:1 OK
20:33:13.570Z :Producer :Topic: topic2, TC:1, GC:2
20:33:13.592Z :Consumer :Topic: topic2, TC:1 OK, GC:2 OK
20:33:13.602Z :Producer :Topic: topic1, TC:2, GC:3
20:33:13.632Z :Consumer :Topic: topic1, TC:2 OK, GC:3 OK
20:33:13.632Z :Producer :Topic: topic2, TC:2, GC:4
20:33:13.637Z :Consumer :Topic: topic2, TC:2 OK, GC:4 OK
20:33:13.664Z :Producer :Topic: topic1, TC:3, GC:5
20:33:13.678Z :Consumer :Topic: topic1, TC:3 OK, GC:5 OK
20:33:13.696Z :Producer :Topic: topic2, TC:3, GC:6
20:33:13.698Z :Consumer :Topic: topic2, TC:3 OK, GC:6 OK
20:33:13.727Z :Producer :Topic: topic1, TC:4, GC:7
20:33:13.728Z :Consumer :Topic: topic1, TC:4 OK, GC:7 OK
20:33:13.758Z :Producer :Topic: topic2, TC:4, GC:8
20:33:13.775Z :Consumer :Topic: topic2, TC:4 OK, GC:8 OK
20:33:13.790Z :Producer :Topic: topic1, TC:5, GC:9
20:33:13.797Z :Consumer :Topic: topic1, TC:5 OK, GC:9 OK
20:33:13.822Z :Producer :Topic: topic2, TC:5, GC:10
20:33:13.823Z :Consumer :Topic: topic2, TC:5 OK, GC:10 OK
20:33:13.853Z :Producer :Topic: topic1, TC:6, GC:11
20:33:13.854Z :Consumer :Topic: topic1, TC:6 OK, GC:11 OK
20:33:13.884Z :Producer :Topic: topic2, TC:6, GC:12
20:33:13.885Z :Consumer :Topic: topic2, TC:6 OK, GC:12 OK
20:33:13.916Z :Producer :Topic: topic1, TC:7, GC:13
20:33:13.931Z :Consumer :Topic: topic1, TC:7 OK, GC:13 OK
20:33:13.947Z :Producer :Topic: topic2, TC:7, GC:14
20:33:13.959Z :Consumer :Topic: topic2, TC:7 OK, GC:14 OK
20:33:13.979Z :Producer :Topic: topic1, TC:8, GC:15
20:33:14.009Z :Consumer :Topic: topic1, TC:8 OK, GC:15 OK
20:33:14.012Z :Producer :Topic: topic2, TC:8, GC:16
20:33:14.042Z :Producer :Topic: topic1, TC:9, GC:17
20:33:14.042Z :Consumer :Topic: topic2, TC:8 OK, GC:16 OK
20:33:14.045Z :Consumer :Topic: topic1, TC:9 OK, GC:17 OK
20:33:14.074Z :Producer :Topic: topic2, TC:9, GC:18
20:33:14.091Z :Consumer :Topic: topic2, TC:9 OK, GC:18 OK
20:33:14.106Z :Producer :Topic: topic1, TC:10, GC:19
20:33:14.120Z :Consumer :Topic: topic1, TC:10 OK, GC:19 OK
20:33:14.137Z :Producer :Topic: topic2, TC:10, GC:20
20:33:14.138Z :Consumer :Topic: topic2, TC:10 OK, GC:20 OK
20:33:14.169Z :Producer :Topic: topic1, TC:11, GC:21
20:33:14.181Z :Consumer :Topic: topic1, TC:11 OK, GC:21 OK
20:33:14.201Z :Producer :Topic: topic2, TC:11, GC:22
20:33:14.201Z :Consumer :Topic: topic2, TC:11 OK, GC:22 OK
20:33:14.231Z :Producer :Topic: topic1, TC:12, GC:23
20:33:14.235Z :Consumer :Topic: topic1, TC:12 OK, GC:23 OK
20:33:14.263Z :Producer :Topic: topic2, TC:12, GC:24
20:33:14.264Z :Consumer :Topic: topic2, TC:12 OK, GC:24 OK
20:33:14.294Z :Producer :Topic: topic1, TC:13, GC:25
20:33:14.300Z :Consumer :Topic: topic1, TC:13 OK, GC:25 OK
20:33:14.325Z :Producer :Topic: topic2, TC:13, GC:26
20:33:14.347Z :Consumer :Topic: topic2, TC:13 OK, GC:26 OK
20:33:14.357Z :Producer :Topic: topic1, TC:14, GC:27
20:33:14.380Z :Consumer :Topic: topic1, TC:14 OK, GC:27 OK
20:33:14.388Z :Producer :Topic: topic2, TC:14, GC:28
20:33:14.419Z :Producer :Topic: topic1, TC:15, GC:29
20:33:14.422Z :Consumer :Topic: topic2, TC:14 OK, GC:28 OK
20:33:14.447Z :Consumer :Topic: topic1, TC:15 OK, GC:29 OK
20:33:14.449Z :Producer :Topic: topic2, TC:15, GC:30
20:33:14.481Z :Producer :Topic: topic1, TC:16, GC:31
20:33:14.485Z :Consumer :Topic: topic2, TC:15 OK, GC:30 OK
20:33:14.495Z :Consumer :Topic: topic1, TC:16 OK, GC:31 OK
20:33:14.513Z :Producer :Topic: topic2, TC:16, GC:32
20:33:14.542Z :Producer :Topic: topic1, TC:17, GC:33
20:33:14.575Z :Producer :Topic: topic2, TC:17, GC:34
20:33:14.606Z :Producer :Topic: topic1, TC:18, GC:35
20:33:14.637Z :Producer :Topic: topic2, TC:18, GC:36
20:33:14.668Z :Producer :Topic: topic1, TC:19, GC:37
20:33:14.698Z :Producer :Topic: topic2, TC:19, GC:38
20:33:14.729Z :Producer :Topic: topic1, TC:20, GC:39
20:33:14.741Z :Consumer :Topic: topic2, TC:16 OK, GC:32 OK
20:33:14.745Z :Consumer :Topic: topic1, TC:17 OK, GC:33 OK
20:33:14.760Z :Producer :Topic: topic2, TC:20, GC:40
20:33:14.784Z :Consumer :Topic: topic2, TC:17 OK, GC:34 OK
20:33:14.791Z :Consumer :Topic: topic1, TC:18 OK, GC:35 OK
20:33:14.792Z :Producer :Topic: topic1, TC:21, GC:41
20:33:14.797Z :Consumer :Topic: topic2, TC:18 OK, GC:36 OK
20:33:14.817Z :Consumer :Topic: topic1, TC:19 OK, GC:37 OK
20:33:14.823Z :Producer :Topic: topic2, TC:21, GC:42
20:33:14.839Z :Consumer :Topic: topic2, TC:19 OK, GC:38 OK
20:33:14.855Z :Producer :Topic: topic1, TC:22, GC:43
20:33:14.867Z :Consumer :Topic: topic1, TC:20 OK, GC:39 OK
20:33:14.886Z :Producer :Topic: topic2, TC:22, GC:44
20:33:14.894Z :Consumer :Topic: topic2, TC:20 OK, GC:40 OK
20:33:14.917Z :Producer :Topic: topic1, TC:23, GC:45
20:33:14.925Z :Consumer :Topic: topic1, TC:21 OK, GC:41 OK
20:33:14.930Z :Consumer :Topic: topic2, TC:21 OK, GC:42 OK
20:33:14.946Z :Consumer :Topic: topic1, TC:22 OK, GC:43 OK
20:33:14.948Z :Producer :Topic: topic2, TC:23, GC:46
20:33:14.975Z :Consumer :Topic: topic2, TC:22 OK, GC:44 OK
20:33:14.979Z :Producer :Topic: topic1, TC:24, GC:47
20:33:14.983Z :Consumer :Topic: topic1, TC:23 OK, GC:45 OK
20:33:14.993Z :Consumer :Topic: topic2, TC:23 OK, GC:46 OK
20:33:15.011Z :Producer :Topic: topic2, TC:24, GC:48
20:33:15.017Z :Consumer :Topic: topic1, TC:24 OK, GC:47 OK
20:33:15.042Z :Producer :Topic: topic1, TC:25, GC:49
20:33:15.044Z :Consumer :Topic: topic2, TC:24 OK, GC:48 OK
20:33:15.070Z :Consumer :Topic: topic1, TC:25 OK, GC:49 OK
20:33:15.073Z :Producer :Topic: topic2, TC:25, GC:50
20:33:15.095Z :Consumer :Topic: topic2, TC:25 OK, GC:50 OK
20:33:15.104Z :Producer :Topic: topic1, TC:26, GC:51
20:33:15.135Z :Producer :Topic: topic2, TC:26, GC:52
20:33:15.167Z :Producer :Topic: topic1, TC:27, GC:53
20:33:15.199Z :Producer :Topic: topic2, TC:27, GC:54
20:33:15.230Z :Producer :Topic: topic1, TC:28, GC:55
20:33:15.262Z :Producer :Topic: topic2, TC:28, GC:56
20:33:15.293Z :Producer :Topic: topic1, TC:29, GC:57
20:33:15.324Z :Producer :Topic: topic2, TC:29, GC:58
20:33:15.338Z :Consumer :Topic: topic1, TC:26 OK, GC:51 OK
20:33:15.355Z :Producer :Topic: topic1, TC:30, GC:59
20:33:15.365Z :Consumer :Topic: topic2, TC:26 OK, GC:52 OK
20:33:15.386Z :Producer :Topic: topic2, TC:30, GC:60
20:33:15.414Z :Consumer :Topic: topic1, TC:27 OK, GC:53 OK
20:33:15.419Z :Producer :Topic: topic1, TC:31, GC:61
20:33:15.449Z :Producer :Topic: topic2, TC:31, GC:62
20:33:15.456Z :Consumer :Topic: topic2, TC:27 OK, GC:54 OK
20:33:15.480Z :Producer :Topic: topic1, TC:32, GC:63
20:33:15.503Z :Consumer :Topic: topic1, TC:28 OK, GC:55 OK
20:33:15.511Z :Producer :Topic: topic2, TC:32, GC:64
20:33:15.528Z :Consumer :Topic: topic2, TC:28 OK, GC:56 OK
20:33:15.539Z :Consumer :Topic: topic1, TC:29 OK, GC:57 OK
20:33:15.542Z :Producer :Topic: topic1, TC:33, GC:65
20:33:15.574Z :Producer :Topic: topic2, TC:33, GC:66
20:33:15.582Z :Consumer :Topic: topic2, TC:29 OK, GC:58 OK
20:33:15.605Z :Producer :Topic: topic1, TC:34, GC:67
20:33:15.627Z :Consumer :Topic: topic1, TC:30 OK, GC:59 OK
20:33:15.636Z :Consumer :Topic: topic2, TC:30 OK, GC:60 OK
20:33:15.636Z :Producer :Topic: topic2, TC:34, GC:68
20:33:15.642Z :Consumer :Topic: topic1, TC:31 OK, GC:61 OK
20:33:15.649Z :Consumer :Topic: topic2, TC:31 OK, GC:62 OK
20:33:15.668Z :Producer :Topic: topic1, TC:35, GC:69
20:33:15.691Z :Consumer :Topic: topic1, TC:32 OK, GC:63 OK
20:33:15.699Z :Producer :Topic: topic2, TC:35, GC:70
20:33:15.700Z :Consumer :Topic: topic2, TC:32 OK, GC:64 OK
20:33:15.729Z :Producer :Topic: topic1, TC:36, GC:71
20:33:15.746Z :Consumer :Topic: topic1, TC:33 OK, GC:65 OK
20:33:15.760Z :Producer :Topic: topic2, TC:36, GC:72
20:33:15.782Z :Consumer :Topic: topic2, TC:33 OK, GC:66 OK
20:33:15.792Z :Producer :Topic: topic1, TC:37, GC:73
20:33:15.794Z :Consumer :Topic: topic1, TC:34 OK, GC:67 OK
20:33:15.823Z :Producer :Topic: topic2, TC:37, GC:74
20:33:15.828Z :Consumer :Topic: topic2, TC:34 OK, GC:68 OK
20:33:15.854Z :Producer :Topic: topic1, TC:38, GC:75
20:33:15.861Z :Consumer :Topic: topic1, TC:35 OK, GC:69 OK
20:33:15.870Z :Consumer :Topic: topic2, TC:35 OK, GC:70 OK
20:33:15.885Z :Producer :Topic: topic2, TC:38, GC:76
20:33:15.889Z :Consumer :Topic: topic1, TC:36 OK, GC:71 OK
20:33:15.895Z :Consumer :Topic: topic2, TC:36 OK, GC:72 OK
20:33:15.916Z :Producer :Topic: topic1, TC:39, GC:77
20:33:15.936Z :Consumer :Topic: topic1, TC:37 OK, GC:73 OK
20:33:15.947Z :Producer :Topic: topic2, TC:39, GC:78
20:33:15.971Z :Consumer :Topic: topic2, TC:37 OK, GC:74 OK
20:33:15.978Z :Producer :Topic: topic1, TC:40, GC:79
20:33:16.010Z :Producer :Topic: topic2, TC:40, GC:80
20:33:16.010Z :Consumer :Topic: topic1, TC:38 OK, GC:75 OK
20:33:16.024Z :Consumer :Topic: topic2, TC:38 OK, GC:76 OK
20:33:16.041Z :Producer :Topic: topic1, TC:41, GC:81
20:33:16.058Z :Consumer :Topic: topic1, TC:39 OK, GC:77 OK
20:33:16.071Z :Producer :Topic: topic2, TC:41, GC:82
20:33:16.085Z :Consumer :Topic: topic2, TC:39 OK, GC:78 OK
20:33:16.103Z :Producer :Topic: topic1, TC:42, GC:83
20:33:16.133Z :Producer :Topic: topic2, TC:42, GC:84
20:33:16.166Z :Producer :Topic: topic1, TC:43, GC:85
20:33:16.197Z :Producer :Topic: topic2, TC:43, GC:86
20:33:16.229Z :Producer :Topic: topic1, TC:44, GC:87
20:33:16.259Z :Producer :Topic: topic2, TC:44, GC:88
20:33:16.290Z :Producer :Topic: topic1, TC:45, GC:89
20:33:16.307Z :Consumer :Topic: topic1, TC:40 OK, GC:79 OK
20:33:16.321Z :Producer :Topic: topic2, TC:45, GC:90
20:33:16.352Z :Producer :Topic: topic1, TC:46, GC:91
20:33:16.354Z :Consumer :Topic: topic2, TC:40 OK, GC:80 OK
20:33:16.378Z :Consumer :Topic: topic1, TC:41 OK, GC:81 OK
20:33:16.383Z :Producer :Topic: topic2, TC:46, GC:92
20:33:16.412Z :Consumer :Topic: topic2, TC:41 OK, GC:82 OK
20:33:16.413Z :Consumer :Topic: topic1, TC:42 OK, GC:83 OK
20:33:16.414Z :Producer :Topic: topic1, TC:47, GC:93
20:33:16.444Z :Producer :Topic: topic2, TC:47, GC:94
20:33:16.453Z :Consumer :Topic: topic2, TC:42 OK, GC:84 OK
20:33:16.476Z :Producer :Topic: topic1, TC:48, GC:95
20:33:16.506Z :Producer :Topic: topic2, TC:48, GC:96
20:33:16.538Z :Producer :Topic: topic1, TC:49, GC:97
20:33:16.570Z :Producer :Topic: topic2, TC:49, GC:98
20:33:16.601Z :Producer :Topic: topic1, TC:50, GC:99
20:33:16.632Z :Producer :Topic: topic2, TC:50, GC:100
20:33:16.656Z :Consumer :Topic: topic1, TC:43 OK, GC:85 OK
20:33:16.678Z :Consumer :Topic: topic2, TC:43 OK, GC:86 OK
20:33:16.718Z :Consumer :Topic: topic1, TC:44 OK, GC:87 OK
20:33:16.733Z :Consumer :Topic: topic2, TC:44 OK, GC:88 OK
20:33:16.768Z :Consumer :Topic: topic1, TC:45 OK, GC:89 OK
20:33:16.783Z :Consumer :Topic: topic2, TC:45 OK, GC:90 OK
20:33:16.802Z :Consumer :Topic: topic1, TC:46 OK, GC:91 OK
20:33:16.834Z :Consumer :Topic: topic2, TC:46 OK, GC:92 OK
20:33:16.852Z :Consumer :Topic: topic1, TC:47 OK, GC:93 OK
20:33:16.888Z :Consumer :Topic: topic2, TC:47 OK, GC:94 OK
20:33:16.898Z :Consumer :Topic: topic1, TC:48 OK, GC:95 OK
20:33:17.118Z :Consumer :Topic: topic2, TC:48 OK, GC:96 OK
20:33:17.155Z :Consumer :Topic: topic1, TC:49 OK, GC:97 OK
20:33:17.164Z :Consumer :Topic: topic2, TC:49 OK, GC:98 OK
20:33:17.168Z :Consumer :Topic: topic1, TC:50 OK, GC:99 OK
20:33:17.176Z :Consumer :Topic: topic2, TC:50 OK, GC:100 OK
测试结果显示,虽然没有严格的“发送-消费”式消息交错,我们依然得到了完美的消息排序。但是请不要忘记,内部缓存默认可容纳 1000 条消息。Pulsar 可能是真正的 producer 尾随者,只不过,从应用程序代码的角度,并没有看到。
测试 4:尾随 topic,消息等比到达,快速 producer
如果消息到达 topic 的速度,高于 consumer 可以应付的水平,这种场景下,消息排序又会如何?
在本轮测试中涉及 1000 条消息,且 producer 向 2 个 topic 发送消息的速度,大于 consumer 消费消息的速度,这种情况下,我们将看到什么?为了避免 Pulsar 通过内部缓存调节进度,我们将 maxReceiverQueue 设置为 10。
在 Kafka 的等效测试中,我们也将 max.poll.records 设置为 10。此时,Kafka 已明显出现消息乱序的现象。但是在 Pulsar 的测试中,我们看到:
20:34:12.056Z :Consumer :Subscribing
20:34:12.523Z :Consumer :Consuming
20:34:17.692Z :Producer :Topic: topic1, TC:1, GC:1
20:34:17.695Z :Consumer :Topic: topic1, TC:1 OK, GC:1 OK
20:34:17.699Z :Producer :Topic: topic2, TC:1, GC:2
20:34:17.707Z :Producer :Topic: topic1, TC:2, GC:3
20:34:17.714Z :Producer :Topic: topic2, TC:2, GC:4
20:34:17.722Z :Producer :Topic: topic1, TC:3, GC:5
20:34:17.728Z :Producer :Topic: topic2, TC:3, GC:6
20:34:17.735Z :Producer :Topic: topic1, TC:4, GC:7
20:34:17.742Z :Producer :Topic: topic2, TC:4, GC:8
20:34:17.750Z :Producer :Topic: topic1, TC:5, GC:9
20:34:17.757Z :Producer :Topic: topic2, TC:5, GC:10
20:34:17.763Z :Producer :Topic: topic1, TC:6, GC:11
20:34:17.770Z :Producer :Topic: topic2, TC:6, GC:12
20:34:17.779Z :Producer :Topic: topic1, TC:7, GC:13
20:34:17.785Z :Producer :Topic: topic2, TC:7, GC:14
20:34:17.792Z :Producer :Topic: topic1, TC:8, GC:15
20:34:17.797Z :Consumer :Topic: topic2, TC:1 OK, GC:2 OK
20:34:17.798Z :Producer :Topic: topic2, TC:8, GC:16
20:34:17.806Z :Producer :Topic: topic1, TC:9, GC:17
20:34:17.813Z :Producer :Topic: topic2, TC:9, GC:18
20:34:17.819Z :Producer :Topic: topic1, TC:10, GC:19
20:34:17.827Z :Producer :Topic: topic2, TC:10, GC:20
20:34:17.833Z :Producer :Topic: topic1, TC:11, GC:21
20:34:17.841Z :Producer :Topic: topic2, TC:11, GC:22
20:34:17.847Z :Producer :Topic: topic1, TC:12, GC:23
20:34:17.855Z :Producer :Topic: topic2, TC:12, GC:24
20:34:17.862Z :Producer :Topic: topic1, TC:13, GC:25
20:34:17.868Z :Producer :Topic: topic2, TC:13, GC:26
20:34:17.876Z :Producer :Topic: topic1, TC:14, GC:27
20:34:17.882Z :Producer :Topic: topic2, TC:14, GC:28
20:34:17.890Z :Producer :Topic: topic1, TC:15, GC:29
20:34:17.896Z :Producer :Topic: topic2, TC:15, GC:30
20:34:17.898Z :Consumer :Topic: topic1, TC:2 OK, GC:3 OK
20:34:17.904Z :Producer :Topic: topic1, TC:16, GC:31
20:34:17.911Z :Producer :Topic: topic2, TC:16, GC:32
20:34:17.917Z :Producer :Topic: topic1, TC:17, GC:33
20:34:17.924Z :Producer :Topic: topic2, TC:17, GC:34
20:34:17.932Z :Producer :Topic: topic1, TC:18, GC:35
20:34:17.938Z :Producer :Topic: topic2, TC:18, GC:36
20:34:17.946Z :Producer :Topic: topic1, TC:19, GC:37
20:34:17.953Z :Producer :Topic: topic2, TC:19, GC:38
20:34:17.960Z :Producer :Topic: topic1, TC:20, GC:39
20:34:17.967Z :Producer :Topic: topic2, TC:20, GC:40
...[omitted 950 messages for brevity]
20:34:24.118Z :Producer :Topic: topic2, TC:495, GC:990
20:34:24.124Z :Producer :Topic: topic1, TC:496, GC:991
20:34:24.131Z :Producer :Topic: topic2, TC:496, GC:992
20:34:24.137Z :Producer :Topic: topic1, TC:497, GC:993
20:34:24.143Z :Producer :Topic: topic2, TC:497, GC:994
20:34:24.148Z :Producer :Topic: topic1, TC:498, GC:995
20:34:24.150Z :Consumer :Topic: topic1, TC:33 OK, GC:65 OK
20:34:24.155Z :Producer :Topic: topic2, TC:498, GC:996
20:34:24.162Z :Producer :Topic: topic1, TC:499, GC:997
20:34:24.168Z :Producer :Topic: topic2, TC:499, GC:998
20:34:24.174Z :Producer :Topic: topic1, TC:500, GC:999
20:34:24.181Z :Producer :Topic: topic2, TC:500, GC:1,000
20:34:24.251Z :Consumer :Topic: topic2, TC:33 OK, GC:66 OK
20:34:24.351Z :Consumer :Topic: topic1, TC:34 OK, GC:67 OK
20:34:24.452Z :Consumer :Topic: topic2, TC:34 OK, GC:68 OK
20:34:24.553Z :Consumer :Topic: topic1, TC:35 OK, GC:69 OK
20:34:24.654Z :Consumer :Topic: topic2, TC:35 OK, GC:70 OK
20:34:24.754Z :Consumer :Topic: topic1, TC:36 OK, GC:71 OK
20:34:24.856Z :Consumer :Topic: topic2, TC:36 OK, GC:72 OK
20:34:24.957Z :Consumer :Topic: topic1, TC:37 OK, GC:73 OK
20:34:25.058Z :Consumer :Topic: topic2, TC:37 OK, GC:74 OK
20:34:25.159Z :Consumer :Topic: topic1, TC:38 OK, GC:75 OK
20:34:25.260Z :Consumer :Topic: topic2, TC:38 OK, GC:76 OK
20:34:25.361Z :Consumer :Topic: topic1, TC:39 OK, GC:77 OK
20:34:25.462Z :Consumer :Topic: topic2, TC:39 OK, GC:78 OK
20:34:25.563Z :Consumer :Topic: topic1, TC:40 OK, GC:79 OK
20:34:25.664Z :Consumer :Topic: topic2, TC:40 OK, GC:80 OK
20:34:25.765Z :Consumer :Topic: topic1, TC:41 OK, GC:81 OK
20:34:25.866Z :Consumer :Topic: topic2, TC:41 OK, GC:82 OK
20:34:25.967Z :Consumer :Topic: topic1, TC:42 OK, GC:83 OK
20:34:26.068Z :Consumer :Topic: topic2, TC:42 OK, GC:84 OK
20:34:26.169Z :Consumer :Topic: topic1, TC:43 OK, GC:85 OK
20:34:26.270Z :Consumer :Topic: topic2, TC:43 OK, GC:86 OK
20:34:26.371Z :Consumer :Topic: topic1, TC:44 OK, GC:87 OK
20:34:26.472Z :Consumer :Topic: topic2, TC:44 OK, GC:88 OK
20:34:26.573Z :Consumer :Topic: topic1, TC:45 OK, GC:89 OK
20:34:26.674Z :Consumer :Topic: topic2, TC:45 OK, GC:90 OK
20:34:26.775Z :Consumer :Topic: topic1, TC:46 OK, GC:91 OK
20:34:26.876Z :Consumer :Topic: topic2, TC:46 OK, GC:92 OK
20:34:26.978Z :Consumer :Topic: topic1, TC:47 OK, GC:93 OK
20:34:27.079Z :Consumer :Topic: topic2, TC:47 OK, GC:94 OK
20:34:27.180Z :Consumer :Topic: topic1, TC:48 OK, GC:95 OK
20:34:27.281Z :Consumer :Topic: topic2, TC:48 OK, GC:96 OK
20:34:27.382Z :Consumer :Topic: topic1, TC:49 OK, GC:97 OK
20:34:27.483Z :Consumer :Topic: topic2, TC:49 OK, GC:98 OK
20:34:27.584Z :Consumer :Topic: topic1, TC:50 OK, GC:99 OK
20:34:27.685Z :Consumer :Topic: topic2, TC:50 OK, GC:100 OK
...[Omitted 850 messages for brevity, trust, they're all in order]
20:35:53.468Z :Consumer :Topic: topic2, TC:475 OK, GC:950 OK
20:35:53.570Z :Consumer :Topic: topic1, TC:476 OK, GC:951 OK
20:35:53.671Z :Consumer :Topic: topic2, TC:476 OK, GC:952 OK
20:35:53.772Z :Consumer :Topic: topic1, TC:477 OK, GC:953 OK
20:35:53.872Z :Consumer :Topic: topic2, TC:477 OK, GC:954 OK
20:35:53.973Z :Consumer :Topic: topic1, TC:478 OK, GC:955 OK
20:35:54.074Z :Consumer :Topic: topic2, TC:478 OK, GC:956 OK
20:35:54.176Z :Consumer :Topic: topic1, TC:479 OK, GC:957 OK
20:35:54.276Z :Consumer :Topic: topic2, TC:479 OK, GC:958 OK
20:35:54.378Z :Consumer :Topic: topic1, TC:480 OK, GC:959 OK
20:35:54.479Z :Consumer :Topic: topic2, TC:480 OK, GC:960 OK
20:35:54.580Z :Consumer :Topic: topic1, TC:481 OK, GC:961 OK
20:35:54.681Z :Consumer :Topic: topic2, TC:481 OK, GC:962 OK
20:35:54.782Z :Consumer :Topic: topic1, TC:482 OK, GC:963 OK
20:35:54.883Z :Consumer :Topic: topic2, TC:482 OK, GC:964 OK
20:35:54.984Z :Consumer :Topic: topic1, TC:483 OK, GC:965 OK
20:35:55.085Z :Consumer :Topic: topic2, TC:483 OK, GC:966 OK
20:35:55.186Z :Consumer :Topic: topic1, TC:484 OK, GC:967 OK
20:35:55.286Z :Consumer :Topic: topic2, TC:484 OK, GC:968 OK
20:35:55.387Z :Consumer :Topic: topic1, TC:485 OK, GC:969 OK
20:35:55.488Z :Consumer :Topic: topic2, TC:485 OK, GC:970 OK
20:35:55.590Z :Consumer :Topic: topic1, TC:486 OK, GC:971 OK
20:35:55.690Z :Consumer :Topic: topic2, TC:486 OK, GC:972 OK
20:35:55.791Z :Consumer :Topic: topic1, TC:487 OK, GC:973 OK
20:35:55.892Z :Consumer :Topic: topic2, TC:487 OK, GC:974 OK
20:35:55.993Z :Consumer :Topic: topic1, TC:488 OK, GC:975 OK
20:35:56.094Z :Consumer :Topic: topic2, TC:488 OK, GC:976 OK
20:35:56.195Z :Consumer :Topic: topic1, TC:489 OK, GC:977 OK
20:35:56.296Z :Consumer :Topic: topic2, TC:489 OK, GC:978 OK
20:35:56.397Z :Consumer :Topic: topic1, TC:490 OK, GC:979 OK
20:35:56.498Z :Consumer :Topic: topic2, TC:490 OK, GC:980 OK
20:35:56.599Z :Consumer :Topic: topic1, TC:491 OK, GC:981 OK
20:35:56.699Z :Consumer :Topic: topic2, TC:491 OK, GC:982 OK
20:35:56.800Z :Consumer :Topic: topic1, TC:492 OK, GC:983 OK
20:35:56.901Z :Consumer :Topic: topic2, TC:492 OK, GC:984 OK
20:35:57.002Z :Consumer :Topic: topic1, TC:493 OK, GC:985 OK
20:35:57.102Z :Consumer :Topic: topic2, TC:493 OK, GC:986 OK
20:35:57.203Z :Consumer :Topic: topic1, TC:494 OK, GC:987 OK
20:35:57.304Z :Consumer :Topic: topic2, TC:494 OK, GC:988 OK
20:35:57.405Z :Consumer :Topic: topic1, TC:495 OK, GC:989 OK
20:35:57.506Z :Consumer :Topic: topic2, TC:495 OK, GC:990 OK
20:35:57.607Z :Consumer :Topic: topic1, TC:496 OK, GC:991 OK
20:35:57.708Z :Consumer :Topic: topic2, TC:496 OK, GC:992 OK
20:35:57.809Z :Consumer :Topic: topic1, TC:497 OK, GC:993 OK
20:35:57.910Z :Consumer :Topic: topic2, TC:497 OK, GC:994 OK
20:35:58.011Z :Consumer :Topic: topic1, TC:498 OK, GC:995 OK
20:35:58.112Z :Consumer :Topic: topic2, TC:498 OK, GC:996 OK
20:35:58.213Z :Consumer :Topic: topic1, TC:499 OK, GC:997 OK
20:35:58.313Z :Consumer :Topic: topic2, TC:499 OK, GC:998 OK
20:35:58.414Z :Consumer :Topic: topic1, TC:500 OK, GC:999 OK
20:35:58.514Z :Consumer :Topic: topic2, TC:500 OK, GC:1,000 OK
测试结果显示,我们再一次得到了完美的消息排序。这实在是个令人惊喜的结果,因为根据我们的语气,减小缓存大小将导致消息无法按顺序传递。
到目前为止,我们都是采用轮询的方式,使消息等比地到达 2 个 topic,这种情况在实际应用环境中并不常见。也许这也正说明了,测试结果非常有趣,但是与真实世界缺乏关联,因为消息并不会按照全局序列等比地到达 2 个 topic。
测试 5:尾随 topic,消息不等比到达,0 处理时长
到目前为止,我们已经看过了 consumer 从 2 个消息等比到达的 topic 上消费消息的情况。但是,当 1 个 topic 的消息到达速度是另外 1 个 topic 的 10 倍时,会发生什么情况呢?下面的测试结果显示当 consumer 的处理时间为 0 时的情况:
21:02:15.193Z :Consumer :Subscribing
21:02:15.636Z :Consumer :Consuming
21:02:15.910Z :Producer :Topic: topic1, TC:1, GC:1
21:02:15.914Z :Consumer :Topic: topic1, TC:1 OK, GC:1 OK
21:02:15.916Z :Producer :Topic: topic1, TC:2, GC:2
21:02:15.920Z :Consumer :Topic: topic1, TC:2 OK, GC:2 OK
21:02:15.922Z :Producer :Topic: topic1, TC:3, GC:3
21:02:15.927Z :Producer :Topic: topic1, TC:4, GC:4
21:02:15.932Z :Producer :Topic: topic1, TC:5, GC:5
21:02:15.932Z :Consumer :Topic: topic1, TC:3 OK, GC:3 OK
21:02:15.933Z :Consumer :Topic: topic1, TC:4 OK, GC:4 OK
21:02:15.934Z :Consumer :Topic: topic1, TC:5 OK, GC:5 OK
21:02:15.936Z :Producer :Topic: topic1, TC:6, GC:6
21:02:15.941Z :Producer :Topic: topic1, TC:7, GC:7
21:02:15.944Z :Consumer :Topic: topic1, TC:6 OK, GC:6 OK
21:02:15.945Z :Consumer :Topic: topic1, TC:7 OK, GC:7 OK
21:02:15.946Z :Producer :Topic: topic1, TC:8, GC:8
21:02:15.951Z :Producer :Topic: topic1, TC:9, GC:9
21:02:15.955Z :Consumer :Topic: topic1, TC:8 OK, GC:8 OK
21:02:15.955Z :Consumer :Topic: topic1, TC:9 OK, GC:9 OK
21:02:15.955Z :Producer :Topic: topic1, TC:10, GC:10
21:02:15.960Z :Producer :Topic: topic2, TC:1, GC:11
21:02:15.961Z :Consumer :Topic: topic2, TC:1 OK, GC:11 JUMP FORWARD 1
21:02:15.965Z :Producer :Topic: topic1, TC:11, GC:12
21:02:15.965Z :Consumer :Topic: topic1, TC:10 OK, GC:10 JUMP BACKWARDS 1
21:02:15.966Z :Consumer :Topic: topic1, TC:11 OK, GC:12 JUMP FORWARD 1
21:02:15.970Z :Producer :Topic: topic1, TC:12, GC:13
21:02:15.975Z :Producer :Topic: topic1, TC:13, GC:14
21:02:15.977Z :Consumer :Topic: topic1, TC:12 OK, GC:13 OK
21:02:15.978Z :Consumer :Topic: topic1, TC:13 OK, GC:14 OK
21:02:15.980Z :Producer :Topic: topic1, TC:14, GC:15
21:02:15.985Z :Producer :Topic: topic1, TC:15, GC:16
21:02:15.988Z :Consumer :Topic: topic1, TC:14 OK, GC:15 OK
21:02:15.988Z :Consumer :Topic: topic1, TC:15 OK, GC:16 OK
21:02:15.989Z :Producer :Topic: topic1, TC:16, GC:17
21:02:15.994Z :Producer :Topic: topic1, TC:17, GC:18
21:02:15.998Z :Consumer :Topic: topic1, TC:16 OK, GC:17 OK
21:02:15.999Z :Consumer :Topic: topic1, TC:17 OK, GC:18 OK
21:02:15.999Z :Producer :Topic: topic1, TC:18, GC:19
21:02:16.004Z :Producer :Topic: topic1, TC:19, GC:20
21:02:16.008Z :Producer :Topic: topic1, TC:20, GC:21
21:02:16.010Z :Consumer :Topic: topic1, TC:18 OK, GC:19 OK
21:02:16.010Z :Consumer :Topic: topic1, TC:19 OK, GC:20 OK
21:02:16.010Z :Consumer :Topic: topic1, TC:20 OK, GC:21 OK
21:02:16.013Z :Producer :Topic: topic2, TC:2, GC:22
21:02:16.014Z :Consumer :Topic: topic2, TC:2 OK, GC:22 OK
21:02:16.018Z :Producer :Topic: topic1, TC:21, GC:23
21:02:16.020Z :Consumer :Topic: topic1, TC:21 OK, GC:23 OK
21:02:16.023Z :Producer :Topic: topic1, TC:22, GC:24
21:02:16.027Z :Producer :Topic: topic1, TC:23, GC:25
21:02:16.032Z :Consumer :Topic: topic1, TC:22 OK, GC:24 OK
21:02:16.032Z :Consumer :Topic: topic1, TC:23 OK, GC:25 OK
21:02:16.032Z :Producer :Topic: topic1, TC:24, GC:26
21:02:16.037Z :Producer :Topic: topic1, TC:25, GC:27
21:02:16.041Z :Producer :Topic: topic1, TC:26, GC:28
21:02:16.043Z :Consumer :Topic: topic1, TC:24 OK, GC:26 OK
21:02:16.043Z :Consumer :Topic: topic1, TC:25 OK, GC:27 OK
21:02:16.043Z :Consumer :Topic: topic1, TC:26 OK, GC:28 OK
21:02:16.046Z :Producer :Topic: topic1, TC:27, GC:29
21:02:16.051Z :Producer :Topic: topic1, TC:28, GC:30
21:02:16.054Z :Consumer :Topic: topic1, TC:27 OK, GC:29 OK
21:02:16.054Z :Consumer :Topic: topic1, TC:28 OK, GC:30 OK
21:02:16.055Z :Producer :Topic: topic1, TC:29, GC:31
21:02:16.060Z :Producer :Topic: topic1, TC:30, GC:32
21:02:16.065Z :Producer :Topic: topic2, TC:3, GC:33
21:02:16.065Z :Consumer :Topic: topic1, TC:29 OK, GC:31 OK
21:02:16.066Z :Consumer :Topic: topic1, TC:30 OK, GC:32 OK
21:02:16.066Z :Consumer :Topic: topic2, TC:3 OK, GC:33 OK
21:02:16.070Z :Producer :Topic: topic1, TC:31, GC:34
21:02:16.075Z :Producer :Topic: topic1, TC:32, GC:35
21:02:16.076Z :Consumer :Topic: topic1, TC:31 OK, GC:34 OK
21:02:16.076Z :Consumer :Topic: topic1, TC:32 OK, GC:35 OK
21:02:16.080Z :Producer :Topic: topic1, TC:33, GC:36
21:02:16.085Z :Producer :Topic: topic1, TC:34, GC:37
21:02:16.087Z :Consumer :Topic: topic1, TC:33 OK, GC:36 OK
21:02:16.087Z :Consumer :Topic: topic1, TC:34 OK, GC:37 OK
21:02:16.090Z :Producer :Topic: topic1, TC:35, GC:38
21:02:16.095Z :Producer :Topic: topic1, TC:36, GC:39
21:02:16.098Z :Consumer :Topic: topic1, TC:35 OK, GC:38 OK
21:02:16.098Z :Consumer :Topic: topic1, TC:36 OK, GC:39 OK
21:02:16.100Z :Producer :Topic: topic1, TC:37, GC:40
21:02:16.105Z :Producer :Topic: topic1, TC:38, GC:41
21:02:16.109Z :Consumer :Topic: topic1, TC:37 OK, GC:40 OK
21:02:16.109Z :Consumer :Topic: topic1, TC:38 OK, GC:41 OK
21:02:16.111Z :Producer :Topic: topic1, TC:39, GC:42
21:02:16.118Z :Producer :Topic: topic1, TC:40, GC:43
21:02:16.120Z :Consumer :Topic: topic1, TC:39 OK, GC:42 OK
21:02:16.121Z :Consumer :Topic: topic1, TC:40 OK, GC:43 OK
21:02:16.123Z :Producer :Topic: topic2, TC:4, GC:44
21:02:16.124Z :Consumer :Topic: topic2, TC:4 OK, GC:44 OK
21:02:16.128Z :Producer :Topic: topic1, TC:41, GC:45
21:02:16.130Z :Consumer :Topic: topic1, TC:41 OK, GC:45 OK
21:02:16.133Z :Producer :Topic: topic1, TC:42, GC:46
21:02:16.138Z :Producer :Topic: topic1, TC:43, GC:47
21:02:16.142Z :Consumer :Topic: topic1, TC:42 OK, GC:46 OK
21:02:16.142Z :Consumer :Topic: topic1, TC:43 OK, GC:47 OK
21:02:16.142Z :Producer :Topic: topic1, TC:44, GC:48
21:02:16.148Z :Producer :Topic: topic1, TC:45, GC:49
21:02:16.153Z :Producer :Topic: topic1, TC:46, GC:50
21:02:16.153Z :Consumer :Topic: topic1, TC:44 OK, GC:48 OK
21:02:16.153Z :Consumer :Topic: topic1, TC:45 OK, GC:49 OK
21:02:16.153Z :Consumer :Topic: topic1, TC:46 OK, GC:50 OK
21:02:16.157Z :Producer :Topic: topic1, TC:47, GC:51
21:02:16.162Z :Producer :Topic: topic1, TC:48, GC:52
21:02:16.164Z :Consumer :Topic: topic1, TC:47 OK, GC:51 OK
21:02:16.164Z :Consumer :Topic: topic1, TC:48 OK, GC:52 OK
21:02:16.168Z :Producer :Topic: topic1, TC:49, GC:53
21:02:16.172Z :Producer :Topic: topic1, TC:50, GC:54
21:02:16.175Z :Consumer :Topic: topic1, TC:49 OK, GC:53 OK
21:02:16.175Z :Consumer :Topic: topic1, TC:50 OK, GC:54 OK
21:02:16.177Z :Producer :Topic: topic2, TC:5, GC:55
21:02:16.178Z :Consumer :Topic: topic2, TC:5 OK, GC:55 OK
21:02:16.182Z :Producer :Topic: topic1, TC:51, GC:56
21:02:16.186Z :Consumer :Topic: topic1, TC:51 OK, GC:56 OK
21:02:16.187Z :Producer :Topic: topic1, TC:52, GC:57
21:02:16.192Z :Producer :Topic: topic1, TC:53, GC:58
21:02:16.197Z :Producer :Topic: topic1, TC:54, GC:59
21:02:16.198Z :Consumer :Topic: topic1, TC:52 OK, GC:57 OK
21:02:16.199Z :Consumer :Topic: topic1, TC:53 OK, GC:58 OK
21:02:16.202Z :Producer :Topic: topic1, TC:55, GC:60
21:02:16.207Z :Producer :Topic: topic1, TC:56, GC:61
21:02:16.208Z :Consumer :Topic: topic1, TC:54 OK, GC:59 OK
21:02:16.208Z :Consumer :Topic: topic1, TC:55 OK, GC:60 OK
21:02:16.209Z :Consumer :Topic: topic1, TC:56 OK, GC:61 OK
21:02:16.212Z :Producer :Topic: topic1, TC:57, GC:62
21:02:16.217Z :Producer :Topic: topic1, TC:58, GC:63
21:02:16.220Z :Consumer :Topic: topic1, TC:57 OK, GC:62 OK
21:02:16.220Z :Consumer :Topic: topic1, TC:58 OK, GC:63 OK
21:02:16.221Z :Producer :Topic: topic1, TC:59, GC:64
21:02:16.226Z :Producer :Topic: topic1, TC:60, GC:65
21:02:16.232Z :Producer :Topic: topic2, TC:6, GC:66
21:02:16.232Z :Consumer :Topic: topic1, TC:59 OK, GC:64 OK
21:02:16.232Z :Consumer :Topic: topic1, TC:60 OK, GC:65 OK
21:02:16.232Z :Consumer :Topic: topic2, TC:6 OK, GC:66 OK
21:02:16.236Z :Producer :Topic: topic1, TC:61, GC:67
21:02:16.241Z :Producer :Topic: topic1, TC:62, GC:68
21:02:16.242Z :Consumer :Topic: topic1, TC:61 OK, GC:67 OK
21:02:16.243Z :Consumer :Topic: topic1, TC:62 OK, GC:68 OK
21:02:16.246Z :Producer :Topic: topic1, TC:63, GC:69
21:02:16.252Z :Producer :Topic: topic1, TC:64, GC:70
21:02:16.253Z :Consumer :Topic: topic1, TC:63 OK, GC:69 OK
21:02:16.253Z :Consumer :Topic: topic1, TC:64 OK, GC:70 OK
21:02:16.257Z :Producer :Topic: topic1, TC:65, GC:71
21:02:16.263Z :Producer :Topic: topic1, TC:66, GC:72
21:02:16.264Z :Consumer :Topic: topic1, TC:65 OK, GC:71 OK
21:02:16.264Z :Consumer :Topic: topic1, TC:66 OK, GC:72 OK
21:02:16.267Z :Producer :Topic: topic1, TC:67, GC:73
21:02:16.271Z :Producer :Topic: topic1, TC:68, GC:74
21:02:16.275Z :Consumer :Topic: topic1, TC:67 OK, GC:73 OK
21:02:16.275Z :Consumer :Topic: topic1, TC:68 OK, GC:74 OK
21:02:16.275Z :Producer :Topic: topic1, TC:69, GC:75
21:02:16.280Z :Producer :Topic: topic1, TC:70, GC:76
21:02:16.286Z :Consumer :Topic: topic1, TC:69 OK, GC:75 OK
21:02:16.286Z :Consumer :Topic: topic1, TC:70 OK, GC:76 OK
21:02:16.286Z :Producer :Topic: topic2, TC:7, GC:77
21:02:16.287Z :Consumer :Topic: topic2, TC:7 OK, GC:77 OK
21:02:16.292Z :Producer :Topic: topic1, TC:71, GC:78
21:02:16.297Z :Producer :Topic: topic1, TC:72, GC:79
21:02:16.297Z :Consumer :Topic: topic1, TC:71 OK, GC:78 OK
21:02:16.301Z :Producer :Topic: topic1, TC:73, GC:80
21:02:16.306Z :Producer :Topic: topic1, TC:74, GC:81
21:02:16.307Z :Consumer :Topic: topic1, TC:72 OK, GC:79 OK
21:02:16.308Z :Consumer :Topic: topic1, TC:73 OK, GC:80 OK
21:02:16.308Z :Consumer :Topic: topic1, TC:74 OK, GC:81 OK
21:02:16.311Z :Producer :Topic: topic1, TC:75, GC:82
21:02:16.316Z :Producer :Topic: topic1, TC:76, GC:83
21:02:16.319Z :Consumer :Topic: topic1, TC:75 OK, GC:82 OK
21:02:16.319Z :Consumer :Topic: topic1, TC:76 OK, GC:83 OK
21:02:16.321Z :Producer :Topic: topic1, TC:77, GC:84
21:02:16.325Z :Producer :Topic: topic1, TC:78, GC:85
21:02:16.330Z :Consumer :Topic: topic1, TC:77 OK, GC:84 OK
21:02:16.331Z :Consumer :Topic: topic1, TC:78 OK, GC:85 OK
21:02:16.331Z :Producer :Topic: topic1, TC:79, GC:86
21:02:16.336Z :Producer :Topic: topic1, TC:80, GC:87
21:02:16.341Z :Consumer :Topic: topic1, TC:79 OK, GC:86 OK
21:02:16.341Z :Consumer :Topic: topic1, TC:80 OK, GC:87 OK
21:02:16.341Z :Producer :Topic: topic2, TC:8, GC:88
21:02:16.342Z :Consumer :Topic: topic2, TC:8 OK, GC:88 OK
21:02:16.346Z :Producer :Topic: topic1, TC:81, GC:89
21:02:16.351Z :Producer :Topic: topic1, TC:82, GC:90
21:02:16.352Z :Consumer :Topic: topic1, TC:81 OK, GC:89 OK
21:02:16.352Z :Consumer :Topic: topic1, TC:82 OK, GC:90 OK
21:02:16.356Z :Producer :Topic: topic1, TC:83, GC:91
21:02:16.360Z :Producer :Topic: topic1, TC:84, GC:92
21:02:16.363Z :Consumer :Topic: topic1, TC:83 OK, GC:91 OK
21:02:16.363Z :Consumer :Topic: topic1, TC:84 OK, GC:92 OK
21:02:16.365Z :Producer :Topic: topic1, TC:85, GC:93
21:02:16.369Z :Producer :Topic: topic1, TC:86, GC:94
21:02:16.374Z :Consumer :Topic: topic1, TC:85 OK, GC:93 OK
21:02:16.374Z :Consumer :Topic: topic1, TC:86 OK, GC:94 OK
21:02:16.374Z :Producer :Topic: topic1, TC:87, GC:95
21:02:16.379Z :Producer :Topic: topic1, TC:88, GC:96
21:02:16.383Z :Producer :Topic: topic1, TC:89, GC:97
21:02:16.385Z :Consumer :Topic: topic1, TC:87 OK, GC:95 OK
21:02:16.385Z :Consumer :Topic: topic1, TC:88 OK, GC:96 OK
21:02:16.385Z :Consumer :Topic: topic1, TC:89 OK, GC:97 OK
21:02:16.388Z :Producer :Topic: topic1, TC:90, GC:98
21:02:16.392Z :Producer :Topic: topic2, TC:9, GC:99
21:02:16.393Z :Consumer :Topic: topic2, TC:9 OK, GC:99 JUMP FORWARD 1
21:02:16.396Z :Consumer :Topic: topic1, TC:90 OK, GC:98 JUMP BACKWARDS 1
21:02:16.397Z :Producer :Topic: topic1, TC:91, GC:100
21:02:16.407Z :Consumer :Topic: topic1, TC:91 OK, GC:100 JUMP FORWARD 1
在 100 条消息中,共有 6 条消息的全局消息排序是错误的,这与测试 1 的结果相同。因此,在实时消费的情况下,消息到达 topic 的速度不同,对于跨主题的消息排序,不会造成影响。
测试 6:尾随 topic,消息不等比到达,短恒定时长
与测试 2 相似,不同之处是,topic 1 与 topic 2 的消息到达比为 10:1。在测试 2 中,我们得到了完美的消息排序。
21:04:52.488Z :Consumer :Subscribing
21:04:52.939Z :Consumer :Consuming
21:04:53.217Z :Producer :Topic: topic1, TC:1, GC:1
21:04:53.222Z :Consumer :Topic: topic1, TC:1 OK, GC:1 OK
21:04:53.234Z :Producer :Topic: topic1, TC:2, GC:2
21:04:53.235Z :Consumer :Topic: topic1, TC:2 OK, GC:2 OK
21:04:53.250Z :Producer :Topic: topic1, TC:3, GC:3
21:04:53.251Z :Consumer :Topic: topic1, TC:3 OK, GC:3 OK
21:04:53.267Z :Producer :Topic: topic1, TC:4, GC:4
21:04:53.268Z :Consumer :Topic: topic1, TC:4 OK, GC:4 OK
21:04:53.284Z :Producer :Topic: topic1, TC:5, GC:5
21:04:53.284Z :Consumer :Topic: topic1, TC:5 OK, GC:5 OK
21:04:53.299Z :Producer :Topic: topic1, TC:6, GC:6
21:04:53.300Z :Consumer :Topic: topic1, TC:6 OK, GC:6 OK
21:04:53.315Z :Producer :Topic: topic1, TC:7, GC:7
21:04:53.316Z :Consumer :Topic: topic1, TC:7 OK, GC:7 OK
21:04:53.331Z :Producer :Topic: topic1, TC:8, GC:8
21:04:53.332Z :Consumer :Topic: topic1, TC:8 OK, GC:8 OK
21:04:53.348Z :Producer :Topic: topic1, TC:9, GC:9
21:04:53.348Z :Consumer :Topic: topic1, TC:9 OK, GC:9 OK
21:04:53.364Z :Producer :Topic: topic1, TC:10, GC:10
21:04:53.365Z :Consumer :Topic: topic1, TC:10 OK, GC:10 OK
21:04:53.381Z :Producer :Topic: topic2, TC:1, GC:11
21:04:53.382Z :Consumer :Topic: topic2, TC:1 OK, GC:11 OK
21:04:53.397Z :Producer :Topic: topic1, TC:11, GC:12
21:04:53.398Z :Consumer :Topic: topic1, TC:11 OK, GC:12 OK
21:04:53.413Z :Producer :Topic: topic1, TC:12, GC:13
21:04:53.414Z :Consumer :Topic: topic1, TC:12 OK, GC:13 OK
21:04:53.429Z :Producer :Topic: topic1, TC:13, GC:14
21:04:53.430Z :Consumer :Topic: topic1, TC:13 OK, GC:14 OK
21:04:53.445Z :Producer :Topic: topic1, TC:14, GC:15
21:04:53.446Z :Consumer :Topic: topic1, TC:14 OK, GC:15 OK
21:04:53.462Z :Producer :Topic: topic1, TC:15, GC:16
21:04:53.462Z :Consumer :Topic: topic1, TC:15 OK, GC:16 OK
21:04:53.478Z :Producer :Topic: topic1, TC:16, GC:17
21:04:53.479Z :Consumer :Topic: topic1, TC:16 OK, GC:17 OK
21:04:53.494Z :Producer :Topic: topic1, TC:17, GC:18
21:04:53.494Z :Consumer :Topic: topic1, TC:17 OK, GC:18 OK
21:04:53.509Z :Producer :Topic: topic1, TC:18, GC:19
21:04:53.509Z :Consumer :Topic: topic1, TC:18 OK, GC:19 OK
21:04:53.524Z :Producer :Topic: topic1, TC:19, GC:20
21:04:53.525Z :Consumer :Topic: topic1, TC:19 OK, GC:20 OK
21:04:53.540Z :Producer :Topic: topic1, TC:20, GC:21
21:04:53.541Z :Consumer :Topic: topic1, TC:20 OK, GC:21 OK
21:04:53.557Z :Producer :Topic: topic2, TC:2, GC:22
21:04:53.557Z :Consumer :Topic: topic2, TC:2 OK, GC:22 OK
21:04:53.572Z :Producer :Topic: topic1, TC:21, GC:23
21:04:53.573Z :Consumer :Topic: topic1, TC:21 OK, GC:23 OK
21:04:53.588Z :Producer :Topic: topic1, TC:22, GC:24
21:04:53.589Z :Consumer :Topic: topic1, TC:22 OK, GC:24 OK
21:04:53.605Z :Producer :Topic: topic1, TC:23, GC:25
21:04:53.605Z :Consumer :Topic: topic1, TC:23 OK, GC:25 OK
21:04:53.621Z :Producer :Topic: topic1, TC:24, GC:26
21:04:53.621Z :Consumer :Topic: topic1, TC:24 OK, GC:26 OK
21:04:53.637Z :Producer :Topic: topic1, TC:25, GC:27
21:04:53.637Z :Consumer :Topic: topic1, TC:25 OK, GC:27 OK
21:04:53.653Z :Producer :Topic: topic1, TC:26, GC:28
21:04:53.653Z :Consumer :Topic: topic1, TC:26 OK, GC:28 OK
21:04:53.669Z :Producer :Topic: topic1, TC:27, GC:29
21:04:53.670Z :Consumer :Topic: topic1, TC:27 OK, GC:29 OK
21:04:53.686Z :Producer :Topic: topic1, TC:28, GC:30
21:04:53.687Z :Consumer :Topic: topic1, TC:28 OK, GC:30 OK
21:04:53.703Z :Producer :Topic: topic1, TC:29, GC:31
21:04:53.703Z :Consumer :Topic: topic1, TC:29 OK, GC:31 OK
21:04:53.718Z :Producer :Topic: topic1, TC:30, GC:32
21:04:53.719Z :Consumer :Topic: topic1, TC:30 OK, GC:32 OK
21:04:53.735Z :Producer :Topic: topic2, TC:3, GC:33
21:04:53.735Z :Consumer :Topic: topic2, TC:3 OK, GC:33 OK
21:04:53.751Z :Producer :Topic: topic1, TC:31, GC:34
21:04:53.752Z :Consumer :Topic: topic1, TC:31 OK, GC:34 OK
21:04:53.767Z :Producer :Topic: topic1, TC:32, GC:35
21:04:53.767Z :Consumer :Topic: topic1, TC:32 OK, GC:35 OK
21:04:53.783Z :Producer :Topic: topic1, TC:33, GC:36
21:04:53.784Z :Consumer :Topic: topic1, TC:33 OK, GC:36 OK
21:04:53.799Z :Producer :Topic: topic1, TC:34, GC:37
21:04:53.800Z :Consumer :Topic: topic1, TC:34 OK, GC:37 OK
21:04:53.814Z :Producer :Topic: topic1, TC:35, GC:38
21:04:53.815Z :Consumer :Topic: topic1, TC:35 OK, GC:38 OK
21:04:53.831Z :Producer :Topic: topic1, TC:36, GC:39
21:04:53.832Z :Consumer :Topic: topic1, TC:36 OK, GC:39 OK
21:04:53.846Z :Producer :Topic: topic1, TC:37, GC:40
21:04:53.847Z :Consumer :Topic: topic1, TC:37 OK, GC:40 OK
21:04:53.863Z :Producer :Topic: topic1, TC:38, GC:41
21:04:53.863Z :Consumer :Topic: topic1, TC:38 OK, GC:41 OK
21:04:53.879Z :Producer :Topic: topic1, TC:39, GC:42
21:04:53.879Z :Consumer :Topic: topic1, TC:39 OK, GC:42 OK
21:04:53.894Z :Producer :Topic: topic1, TC:40, GC:43
21:04:53.895Z :Consumer :Topic: topic1, TC:40 OK, GC:43 OK
21:04:53.910Z :Producer :Topic: topic2, TC:4, GC:44
21:04:53.911Z :Consumer :Topic: topic2, TC:4 OK, GC:44 OK
21:04:53.927Z :Producer :Topic: topic1, TC:41, GC:45
21:04:53.927Z :Consumer :Topic: topic1, TC:41 OK, GC:45 OK
21:04:53.942Z :Producer :Topic: topic1, TC:42, GC:46
21:04:53.943Z :Consumer :Topic: topic1, TC:42 OK, GC:46 OK
21:04:53.958Z :Producer :Topic: topic1, TC:43, GC:47
21:04:53.959Z :Consumer :Topic: topic1, TC:43 OK, GC:47 OK
21:04:53.974Z :Producer :Topic: topic1, TC:44, GC:48
21:04:53.975Z :Consumer :Topic: topic1, TC:44 OK, GC:48 OK
21:04:53.991Z :Producer :Topic: topic1, TC:45, GC:49
21:04:53.991Z :Consumer :Topic: topic1, TC:45 OK, GC:49 OK
21:04:54.007Z :Producer :Topic: topic1, TC:46, GC:50
21:04:54.008Z :Consumer :Topic: topic1, TC:46 OK, GC:50 OK
21:04:54.022Z :Producer :Topic: topic1, TC:47, GC:51
21:04:54.023Z :Consumer :Topic: topic1, TC:47 OK, GC:51 OK
21:04:54.039Z :Producer :Topic: topic1, TC:48, GC:52
21:04:54.041Z :Consumer :Topic: topic1, TC:48 OK, GC:52 OK
21:04:54.056Z :Producer :Topic: topic1, TC:49, GC:53
21:04:54.057Z :Consumer :Topic: topic1, TC:49 OK, GC:53 OK
21:04:54.072Z :Producer :Topic: topic1, TC:50, GC:54
21:04:54.073Z :Consumer :Topic: topic1, TC:50 OK, GC:54 OK
21:04:54.089Z :Producer :Topic: topic2, TC:5, GC:55
21:04:54.090Z :Consumer :Topic: topic2, TC:5 OK, GC:55 OK
21:04:54.104Z :Producer :Topic: topic1, TC:51, GC:56
21:04:54.105Z :Consumer :Topic: topic1, TC:51 OK, GC:56 OK
21:04:54.120Z :Producer :Topic: topic1, TC:52, GC:57
21:04:54.121Z :Consumer :Topic: topic1, TC:52 OK, GC:57 OK
21:04:54.136Z :Producer :Topic: topic1, TC:53, GC:58
21:04:54.137Z :Consumer :Topic: topic1, TC:53 OK, GC:58 OK
21:04:54.154Z :Producer :Topic: topic1, TC:54, GC:59
21:04:54.154Z :Consumer :Topic: topic1, TC:54 OK, GC:59 OK
21:04:54.169Z :Producer :Topic: topic1, TC:55, GC:60
21:04:54.170Z :Consumer :Topic: topic1, TC:55 OK, GC:60 OK
21:04:54.185Z :Producer :Topic: topic1, TC:56, GC:61
21:04:54.186Z :Consumer :Topic: topic1, TC:56 OK, GC:61 OK
21:04:54.201Z :Producer :Topic: topic1, TC:57, GC:62
21:04:54.202Z :Consumer :Topic: topic1, TC:57 OK, GC:62 OK
21:04:54.217Z :Producer :Topic: topic1, TC:58, GC:63
21:04:54.218Z :Consumer :Topic: topic1, TC:58 OK, GC:63 OK
21:04:54.234Z :Producer :Topic: topic1, TC:59, GC:64
21:04:54.234Z :Consumer :Topic: topic1, TC:59 OK, GC:64 OK
21:04:54.250Z :Producer :Topic: topic1, TC:60, GC:65
21:04:54.250Z :Consumer :Topic: topic1, TC:60 OK, GC:65 OK
21:04:54.266Z :Producer :Topic: topic2, TC:6, GC:66
21:04:54.266Z :Consumer :Topic: topic2, TC:6 OK, GC:66 OK
21:04:54.281Z :Producer :Topic: topic1, TC:61, GC:67
21:04:54.282Z :Consumer :Topic: topic1, TC:61 OK, GC:67 OK
21:04:54.299Z :Producer :Topic: topic1, TC:62, GC:68
21:04:54.300Z :Consumer :Topic: topic1, TC:62 OK, GC:68 OK
21:04:54.315Z :Producer :Topic: topic1, TC:63, GC:69
21:04:54.315Z :Consumer :Topic: topic1, TC:63 OK, GC:69 OK
21:04:54.331Z :Producer :Topic: topic1, TC:64, GC:70
21:04:54.332Z :Consumer :Topic: topic1, TC:64 OK, GC:70 OK
21:04:54.347Z :Producer :Topic: topic1, TC:65, GC:71
21:04:54.347Z :Consumer :Topic: topic1, TC:65 OK, GC:71 OK
21:04:54.363Z :Producer :Topic: topic1, TC:66, GC:72
21:04:54.363Z :Consumer :Topic: topic1, TC:66 OK, GC:72 OK
21:04:54.378Z :Producer :Topic: topic1, TC:67, GC:73
21:04:54.379Z :Consumer :Topic: topic1, TC:67 OK, GC:73 OK
21:04:54.393Z :Producer :Topic: topic1, TC:68, GC:74
21:04:54.394Z :Consumer :Topic: topic1, TC:68 OK, GC:74 OK
21:04:54.408Z :Producer :Topic: topic1, TC:69, GC:75
21:04:54.409Z :Consumer :Topic: topic1, TC:69 OK, GC:75 OK
21:04:54.424Z :Producer :Topic: topic1, TC:70, GC:76
21:04:54.425Z :Consumer :Topic: topic1, TC:70 OK, GC:76 OK
21:04:54.441Z :Producer :Topic: topic2, TC:7, GC:77
21:04:54.442Z :Consumer :Topic: topic2, TC:7 OK, GC:77 OK
21:04:54.456Z :Producer :Topic: topic1, TC:71, GC:78
21:04:54.457Z :Consumer :Topic: topic1, TC:71 OK, GC:78 OK
21:04:54.471Z :Producer :Topic: topic1, TC:72, GC:79
21:04:54.472Z :Consumer :Topic: topic1, TC:72 OK, GC:79 OK
21:04:54.487Z :Producer :Topic: topic1, TC:73, GC:80
21:04:54.488Z :Consumer :Topic: topic1, TC:73 OK, GC:80 OK
21:04:54.503Z :Producer :Topic: topic1, TC:74, GC:81
21:04:54.504Z :Consumer :Topic: topic1, TC:74 OK, GC:81 OK
21:04:54.519Z :Producer :Topic: topic1, TC:75, GC:82
21:04:54.520Z :Consumer :Topic: topic1, TC:75 OK, GC:82 OK
21:04:54.536Z :Producer :Topic: topic1, TC:76, GC:83
21:04:54.536Z :Consumer :Topic: topic1, TC:76 OK, GC:83 OK
21:04:54.551Z :Producer :Topic: topic1, TC:77, GC:84
21:04:54.552Z :Consumer :Topic: topic1, TC:77 OK, GC:84 OK
21:04:54.568Z :Producer :Topic: topic1, TC:78, GC:85
21:04:54.569Z :Consumer :Topic: topic1, TC:78 OK, GC:85 OK
21:04:54.586Z :Producer :Topic: topic1, TC:79, GC:86
21:04:54.587Z :Consumer :Topic: topic1, TC:79 OK, GC:86 OK
21:04:54.602Z :Producer :Topic: topic1, TC:80, GC:87
21:04:54.603Z :Consumer :Topic: topic1, TC:80 OK, GC:87 OK
21:04:54.617Z :Producer :Topic: topic2, TC:8, GC:88
21:04:54.618Z :Consumer :Topic: topic2, TC:8 OK, GC:88 OK
21:04:54.632Z :Producer :Topic: topic1, TC:81, GC:89
21:04:54.633Z :Consumer :Topic: topic1, TC:81 OK, GC:89 OK
21:04:54.648Z :Producer :Topic: topic1, TC:82, GC:90
21:04:54.649Z :Consumer :Topic: topic1, TC:82 OK, GC:90 OK
21:04:54.664Z :Producer :Topic: topic1, TC:83, GC:91
21:04:54.665Z :Consumer :Topic: topic1, TC:83 OK, GC:91 OK
21:04:54.680Z :Producer :Topic: topic1, TC:84, GC:92
21:04:54.681Z :Consumer :Topic: topic1, TC:84 OK, GC:92 OK
21:04:54.696Z :Producer :Topic: topic1, TC:85, GC:93
21:04:54.697Z :Consumer :Topic: topic1, TC:85 OK, GC:93 OK
21:04:54.712Z :Producer :Topic: topic1, TC:86, GC:94
21:04:54.712Z :Consumer :Topic: topic1, TC:86 OK, GC:94 OK
21:04:54.727Z :Producer :Topic: topic1, TC:87, GC:95
21:04:54.728Z :Consumer :Topic: topic1, TC:87 OK, GC:95 OK
21:04:54.743Z :Producer :Topic: topic1, TC:88, GC:96
21:04:54.745Z :Consumer :Topic: topic1, TC:88 OK, GC:96 OK
21:04:54.759Z :Producer :Topic: topic1, TC:89, GC:97
21:04:54.760Z :Consumer :Topic: topic1, TC:89 OK, GC:97 OK
21:04:54.775Z :Producer :Topic: topic1, TC:90, GC:98
21:04:54.776Z :Consumer :Topic: topic1, TC:90 OK, GC:98 OK
21:04:54.791Z :Producer :Topic: topic2, TC:9, GC:99
21:04:54.792Z :Consumer :Topic: topic2, TC:9 OK, GC:99 OK
21:04:54.806Z :Producer :Topic: topic1, TC:91, GC:100
21:04:54.807Z :Consumer :Topic: topic1, TC:91 OK, GC:100 OK
本轮测试同样得到了严格的“发送-消费”式消息交错,以及跨主题场景下完美的消息排序——实时消费使完美排序成为可能。
测试 7:尾随 topic,消息不等比到达,短抖动时长
与测试 3 相似,不同之处是,topic 1 与 topic 2 的消息到达比为 10:1。在测试 3 中,我们得到了完美的消息排序。
21:07:11.030Z :Consumer :Subscribing
21:07:11.497Z :Consumer :Consuming
21:07:11.766Z :Producer :Topic: topic1ic1ic1, TC:1, GC:1
21:07:11.770Z :Consumer :Topic: topic1, TC:1 OK, GC:1 OK
21:07:11.796Z :Producer :Topic: topic1, TC:2, GC:2
21:07:11.820Z :Consumer :Topic: topic1, TC:2 OK, GC:2 OK
21:07:11.827Z :Producer :Topic: topic1, TC:3, GC:3
21:07:11.835Z :Consumer :Topic: topic1, TC:3 OK, GC:3 OK
21:07:11.858Z :Producer :Topic: topic1, TC:4, GC:4
21:07:11.858Z :Consumer :Topic: topic1, TC:4 OK, GC:4 OK
21:07:11.888Z :Producer :Topic: topic1, TC:5, GC:5
21:07:11.889Z :Consumer :Topic: topic1, TC:5 OK, GC:5 OK
21:07:11.919Z :Producer :Topic: topic1, TC:6, GC:6
21:07:11.931Z :Consumer :Topic: topic1, TC:6 OK, GC:6 OK
21:07:11.951Z :Producer :Topic: topic1, TC:7, GC:7
21:07:11.982Z :Producer :Topic: topic1, TC:8, GC:8
21:07:12.013Z :Producer :Topic: topic1, TC:9, GC:9
21:07:12.045Z :Producer :Topic: topic1, TC:10, GC:10
21:07:12.076Z :Producer :Topic: topic2, TC:1, GC:11
21:07:12.106Z :Producer :Topic: topic1, TC:11, GC:12
21:07:12.137Z :Producer :Topic: topic1, TC:12, GC:13
21:07:12.144Z :Consumer :Topic: topic1, TC:7 OK, GC:7 OK
21:07:12.168Z :Producer :Topic: topic1, TC:13, GC:14
21:07:12.176Z :Consumer :Topic: topic1, TC:8 OK, GC:8 OK
21:07:12.200Z :Producer :Topic: topic1, TC:14, GC:15
21:07:12.220Z :Consumer :Topic: topic1, TC:9 OK, GC:9 OK
21:07:12.223Z :Consumer :Topic: topic1, TC:10 OK, GC:10 OK
21:07:12.228Z :Consumer :Topic: topic2, TC:1 OK, GC:11 OK
21:07:12.231Z :Producer :Topic: topic1, TC:15, GC:16
21:07:12.252Z :Consumer :Topic: topic1, TC:11 OK, GC:12 OK
21:07:12.260Z :Producer :Topic: topic1, TC:16, GC:17
21:07:12.292Z :Producer :Topic: topic1, TC:17, GC:18
21:07:12.297Z :Consumer :Topic: topic1, TC:12 OK, GC:13 OK
21:07:12.323Z :Producer :Topic: topic1, TC:18, GC:19
21:07:12.345Z :Consumer :Topic: topic1, TC:13 OK, GC:14 OK
21:07:12.354Z :Producer :Topic: topic1, TC:19, GC:20
21:07:12.365Z :Consumer :Topic: topic1, TC:14 OK, GC:15 OK
21:07:12.385Z :Producer :Topic: topic1, TC:20, GC:21
21:07:12.409Z :Consumer :Topic: topic1, TC:15 OK, GC:16 OK
21:07:12.417Z :Producer :Topic: topic2, TC:2, GC:22
21:07:12.432Z :Consumer :Topic: topic1, TC:16 OK, GC:17 OK
21:07:12.446Z :Consumer :Topic: topic1, TC:17 OK, GC:18 OK
21:07:12.447Z :Producer :Topic: topic1, TC:21, GC:23
21:07:12.464Z :Consumer :Topic: topic1, TC:18 OK, GC:19 OK
21:07:12.478Z :Producer :Topic: topic1, TC:22, GC:24
21:07:12.490Z :Consumer :Topic: topic1, TC:19 OK, GC:20 OK
21:07:12.496Z :Consumer :Topic: topic1, TC:20 OK, GC:21 OK
21:07:12.509Z :Producer :Topic: topic1, TC:23, GC:25
21:07:12.532Z :Consumer :Topic: topic2, TC:2 OK, GC:22 OK
21:07:12.540Z :Producer :Topic: topic1, TC:24, GC:26
21:07:12.542Z :Consumer :Topic: topic1, TC:21 OK, GC:23 OK
21:07:12.570Z :Producer :Topic: topic1, TC:25, GC:27
21:07:12.584Z :Consumer :Topic: topic1, TC:22 OK, GC:24 OK
21:07:12.602Z :Producer :Topic: topic1, TC:26, GC:28
21:07:12.610Z :Consumer :Topic: topic1, TC:23 OK, GC:25 OK
21:07:12.633Z :Producer :Topic: topic1, TC:27, GC:29
21:07:12.648Z :Consumer :Topic: topic1, TC:24 OK, GC:26 OK
21:07:12.664Z :Producer :Topic: topic1, TC:28, GC:30
21:07:12.697Z :Producer :Topic: topic1, TC:29, GC:31
21:07:12.697Z :Consumer :Topic: topic1, TC:25 OK, GC:27 OK
21:07:12.727Z :Producer :Topic: topic1, TC:30, GC:32
21:07:12.747Z :Consumer :Topic: topic1, TC:26 OK, GC:28 OK
21:07:12.758Z :Producer :Topic: topic2, TC:3, GC:33
21:07:12.772Z :Consumer :Topic: topic1, TC:27 OK, GC:29 OK
21:07:12.779Z :Consumer :Topic: topic1, TC:28 OK, GC:30 OK
21:07:12.785Z :Consumer :Topic: topic1, TC:29 OK, GC:31 OK
21:07:12.789Z :Producer :Topic: topic1, TC:31, GC:34
21:07:12.816Z :Consumer :Topic: topic1, TC:30 OK, GC:32 OK
21:07:12.820Z :Producer :Topic: topic1, TC:32, GC:35
21:07:12.851Z :Producer :Topic: topic1, TC:33, GC:36
21:07:12.883Z :Producer :Topic: topic1, TC:34, GC:37
21:07:12.915Z :Producer :Topic: topic1, TC:35, GC:38
21:07:12.947Z :Producer :Topic: topic1, TC:36, GC:39
21:07:12.978Z :Producer :Topic: topic1, TC:37, GC:40
21:07:13.008Z :Producer :Topic: topic1, TC:38, GC:41
21:07:13.039Z :Producer :Topic: topic1, TC:39, GC:42
21:07:13.043Z :Consumer :Topic: topic2, TC:3 OK, GC:33 OK
21:07:13.071Z :Producer :Topic: topic1, TC:40, GC:43
21:07:13.077Z :Consumer :Topic: topic1, TC:31 OK, GC:34 OK
21:07:13.080Z :Consumer :Topic: topic1, TC:32 OK, GC:35 OK
21:07:13.088Z :Consumer :Topic: topic1, TC:33 OK, GC:36 OK
21:07:13.103Z :Producer :Topic: topic2, TC:4, GC:44
21:07:13.104Z :Consumer :Topic: topic1, TC:34 OK, GC:37 OK
21:07:13.110Z :Consumer :Topic: topic1, TC:35 OK, GC:38 OK
21:07:13.134Z :Producer :Topic: topic1, TC:41, GC:45
21:07:13.151Z :Consumer :Topic: topic1, TC:36 OK, GC:39 OK
21:07:13.166Z :Producer :Topic: topic1, TC:42, GC:46
21:07:13.188Z :Consumer :Topic: topic1, TC:37 OK, GC:40 OK
21:07:13.196Z :Producer :Topic: topic1, TC:43, GC:47
21:07:13.227Z :Producer :Topic: topic1, TC:44, GC:48
21:07:13.235Z :Consumer :Topic: topic1, TC:38 OK, GC:41 OK
21:07:13.248Z :Consumer :Topic: topic1, TC:39 OK, GC:42 OK
21:07:13.257Z :Producer :Topic: topic1, TC:45, GC:49
21:07:13.265Z :Consumer :Topic: topic1, TC:40 OK, GC:43 OK
21:07:13.287Z :Consumer :Topic: topic2, TC:4 OK, GC:44 OK
21:07:13.290Z :Producer :Topic: topic1, TC:46, GC:50
21:07:13.320Z :Producer :Topic: topic1, TC:47, GC:51
21:07:13.322Z :Consumer :Topic: topic1, TC:41 OK, GC:45 OK
21:07:13.347Z :Consumer :Topic: topic1, TC:42 OK, GC:46 OK
21:07:13.351Z :Producer :Topic: topic1, TC:48, GC:52
21:07:13.380Z :Consumer :Topic: topic1, TC:43 OK, GC:47 OK
21:07:13.382Z :Producer :Topic: topic1, TC:49, GC:53
21:07:13.383Z :Consumer :Topic: topic1, TC:44 OK, GC:48 OK
21:07:13.394Z :Consumer :Topic: topic1, TC:45 OK, GC:49 OK
21:07:13.413Z :Consumer :Topic: topic1, TC:46 OK, GC:50 OK
21:07:13.414Z :Producer :Topic: topic1, TC:50, GC:54
21:07:13.435Z :Consumer :Topic: topic1, TC:47 OK, GC:51 OK
21:07:13.438Z :Consumer :Topic: topic1, TC:48 OK, GC:52 OK
21:07:13.444Z :Producer :Topic: topic2, TC:5, GC:55
21:07:13.475Z :Producer :Topic: topic1, TC:51, GC:56
21:07:13.488Z :Consumer :Topic: topic1, TC:49 OK, GC:53 OK
21:07:13.506Z :Producer :Topic: topic1, TC:52, GC:57
21:07:13.519Z :Consumer :Topic: topic1, TC:50 OK, GC:54 OK
21:07:13.532Z :Consumer :Topic: topic2, TC:5 OK, GC:55 OK
21:07:13.538Z :Producer :Topic: topic1, TC:53, GC:58
21:07:13.567Z :Consumer :Topic: topic1, TC:51 OK, GC:56 OK
21:07:13.569Z :Producer :Topic: topic1, TC:54, GC:59
21:07:13.585Z :Consumer :Topic: topic1, TC:52 OK, GC:57 OK
21:07:13.590Z :Consumer :Topic: topic1, TC:53 OK, GC:58 OK
21:07:13.599Z :Producer :Topic: topic1, TC:55, GC:60
21:07:13.631Z :Producer :Topic: topic1, TC:56, GC:61
21:07:13.631Z :Consumer :Topic: topic1, TC:54 OK, GC:59 OK
21:07:13.662Z :Producer :Topic: topic1, TC:57, GC:62
21:07:13.680Z :Consumer :Topic: topic1, TC:55 OK, GC:60 OK
21:07:13.692Z :Producer :Topic: topic1, TC:58, GC:63
21:07:13.704Z :Consumer :Topic: topic1, TC:56 OK, GC:61 OK
21:07:13.722Z :Consumer :Topic: topic1, TC:57 OK, GC:62 OK
21:07:13.723Z :Producer :Topic: topic1, TC:59, GC:64
21:07:13.755Z :Producer :Topic: topic1, TC:60, GC:65
21:07:13.767Z :Consumer :Topic: topic1, TC:58 OK, GC:63 OK
21:07:13.787Z :Producer :Topic: topic2, TC:6, GC:66
21:07:13.794Z :Consumer :Topic: topic1, TC:59 OK, GC:64 OK
21:07:13.798Z :Consumer :Topic: topic1, TC:60 OK, GC:65 OK
21:07:13.804Z :Consumer :Topic: topic2, TC:6 OK, GC:66 OK
21:07:13.818Z :Producer :Topic: topic1, TC:61, GC:67
21:07:13.829Z :Consumer :Topic: topic1, TC:61 OK, GC:67 OK
21:07:13.848Z :Producer :Topic: topic1, TC:62, GC:68
21:07:13.875Z :Consumer :Topic: topic1, TC:62 OK, GC:68 OK
21:07:13.878Z :Producer :Topic: topic1, TC:63, GC:69
21:07:13.890Z :Consumer :Topic: topic1, TC:63 OK, GC:69 OK
21:07:13.909Z :Producer :Topic: topic1, TC:64, GC:70
21:07:13.910Z :Consumer :Topic: topic1, TC:64 OK, GC:70 OK
21:07:13.940Z :Producer :Topic: topic1, TC:65, GC:71
21:07:13.972Z :Producer :Topic: topic1, TC:66, GC:72
21:07:14.003Z :Producer :Topic: topic1, TC:67, GC:73
21:07:14.034Z :Producer :Topic: topic1, TC:68, GC:74
21:07:14.065Z :Producer :Topic: topic1, TC:69, GC:75
21:07:14.096Z :Producer :Topic: topic1, TC:70, GC:76
21:07:14.127Z :Producer :Topic: topic2, TC:7, GC:77
21:07:14.143Z :Consumer :Topic: topic1, TC:65 OK, GC:71 OK
21:07:14.158Z :Producer :Topic: topic1, TC:71, GC:78
21:07:14.189Z :Producer :Topic: topic1, TC:72, GC:79
21:07:14.219Z :Producer :Topic: topic1, TC:73, GC:80
21:07:14.250Z :Producer :Topic: topic1, TC:74, GC:81
21:07:14.281Z :Producer :Topic: topic1, TC:75, GC:82
21:07:14.312Z :Producer :Topic: topic1, TC:76, GC:83
21:07:14.342Z :Producer :Topic: topic1, TC:77, GC:84
21:07:14.357Z :Consumer :Topic: topic1, TC:66 OK, GC:72 OK
21:07:14.364Z :Consumer :Topic: topic1, TC:67 OK, GC:73 OK
21:07:14.373Z :Producer :Topic: topic1, TC:78, GC:85
21:07:14.405Z :Producer :Topic: topic1, TC:79, GC:86
21:07:14.407Z :Consumer :Topic: topic1, TC:68 OK, GC:74 OK
21:07:14.437Z :Producer :Topic: topic1, TC:80, GC:87
21:07:14.437Z :Consumer :Topic: topic1, TC:69 OK, GC:75 OK
21:07:14.468Z :Producer :Topic: topic2, TC:8, GC:88
21:07:14.469Z :Consumer :Topic: topic1, TC:70 OK, GC:76 OK
21:07:14.499Z :Producer :Topic: topic1, TC:81, GC:89
21:07:14.512Z :Consumer :Topic: topic2, TC:7 OK, GC:77 OK
21:07:14.530Z :Producer :Topic: topic1, TC:82, GC:90
21:07:14.534Z :Consumer :Topic: topic1, TC:71 OK, GC:78 OK
21:07:14.552Z :Consumer :Topic: topic1, TC:72 OK, GC:79 OK
21:07:14.561Z :Producer :Topic: topic1, TC:83, GC:91
21:07:14.592Z :Producer :Topic: topic1, TC:84, GC:92
21:07:14.623Z :Producer :Topic: topic1, TC:85, GC:93
21:07:14.654Z :Producer :Topic: topic1, TC:86, GC:94
21:07:14.685Z :Producer :Topic: topic1, TC:87, GC:95
21:07:14.716Z :Producer :Topic: topic1, TC:88, GC:96
21:07:14.747Z :Producer :Topic: topic1, TC:89, GC:97
21:07:14.778Z :Producer :Topic: topic1, TC:90, GC:98
21:07:14.784Z :Consumer :Topic: topic1, TC:73 OK, GC:80 OK
21:07:14.787Z :Consumer :Topic: topic1, TC:74 OK, GC:81 OK
21:07:14.809Z :Producer :Topic: topic2, TC:9, GC:99
21:07:14.817Z :Consumer :Topic: topic1, TC:75 OK, GC:82 OK
21:07:14.837Z :Consumer :Topic: topic1, TC:76 OK, GC:83 OK
21:07:14.840Z :Producer :Topic: topic1, TC:91, GC:100
21:07:14.869Z :Consumer :Topic: topic1, TC:77 OK, GC:84 OK
21:07:14.873Z :Consumer :Topic: topic1, TC:78 OK, GC:85 OK
21:07:14.894Z :Consumer :Topic: topic1, TC:79 OK, GC:86 OK
21:07:14.912Z :Consumer :Topic: topic1, TC:80 OK, GC:87 OK
21:07:14.953Z :Consumer :Topic: topic2, TC:8 OK, GC:88 OK
21:07:14.959Z :Consumer :Topic: topic1, TC:81 OK, GC:89 OK
21:07:14.999Z :Consumer :Topic: topic1, TC:82 OK, GC:90 OK
21:07:15.042Z :Consumer :Topic: topic1, TC:83 OK, GC:91 OK
21:07:15.090Z :Consumer :Topic: topic1, TC:84 OK, GC:92 OK
21:07:15.111Z :Consumer :Topic: topic1, TC:85 OK, GC:93 OK
21:07:15.146Z :Consumer :Topic: topic1, TC:86 OK, GC:94 OK
21:07:15.188Z :Consumer :Topic: topic1, TC:87 OK, GC:95 OK
21:07:15.226Z :Consumer :Topic: topic1, TC:88 OK, GC:96 OK
21:07:15.253Z :Consumer :Topic: topic1, TC:89 OK, GC:97 OK
21:07:15.268Z :Consumer :Topic: topic1, TC:90 OK, GC:98 OK
21:07:15.279Z :Consumer :Topic: topic2, TC:9 OK, GC:99 OK
21:07:15.296Z :Consumer :Topic: topic1, TC:91 OK, GC:100 OK
默认的内部缓存可容纳 1000 条消息,因此,虽然我们的测试代码没有呈现出严格的“发送-消费”式消息交错,但是 Pulsar 还是很好地实现了完美的消息排序。所以,在实时消费的情况下,到达 topic 的速度不会影响消息排序。
测试 8:尾随 topic,消息不等比到达,快速 producer
与测试 4 相似,不同之处是,为了避免与 producer 同步,我们减小了内部缓存的大小,将其设置为 10。
这一次,我们失去了实时消费,同时,发送到 topic 的消息也不等比。
21:13:51.601Z :Consumer :Subscribing
21:13:52.054Z :Consumer :Consuming
21:13:57.224Z :Producer :Topic: topic1, TC:1, GC:1
21:13:57.229Z :Consumer :Topic: topic1, TC:1 OK, GC:1 OK
21:13:57.232Z :Producer :Topic: topic1, TC:2, GC:2
21:13:57.237Z :Producer :Topic: topic1, TC:3, GC:3
21:13:57.243Z :Producer :Topic: topic1, TC:4, GC:4
21:13:57.248Z :Producer :Topic: topic1, TC:5, GC:5
21:13:57.255Z :Producer :Topic: topic1, TC:6, GC:6
21:13:57.263Z :Producer :Topic: topic1, TC:7, GC:7
21:13:57.269Z :Producer :Topic: topic1, TC:8, GC:8
21:13:57.275Z :Producer :Topic: topic1, TC:9, GC:9
21:13:57.282Z :Producer :Topic: topic1, TC:10, GC:10
21:13:57.288Z :Producer :Topic: topic2, TC:1, GC:11
21:13:57.295Z :Producer :Topic: topic1, TC:11, GC:12
21:13:57.301Z :Producer :Topic: topic1, TC:12, GC:13
21:13:57.308Z :Producer :Topic: topic1, TC:13, GC:14
21:13:57.315Z :Producer :Topic: topic1, TC:14, GC:15
21:13:57.321Z :Producer :Topic: topic1, TC:15, GC:16
21:13:57.327Z :Producer :Topic: topic1, TC:16, GC:17
21:13:57.331Z :Consumer :Topic: topic1, TC:2 OK, GC:2 OK
21:13:57.333Z :Producer :Topic: topic1, TC:17, GC:18
21:13:57.339Z :Producer :Topic: topic1, TC:18, GC:19
21:13:57.345Z :Producer :Topic: topic1, TC:19, GC:20
21:13:57.351Z :Producer :Topic: topic1, TC:20, GC:21
21:13:57.359Z :Producer :Topic: topic2, TC:2, GC:22
21:13:57.366Z :Producer :Topic: topic1, TC:21, GC:23
21:13:57.372Z :Producer :Topic: topic1, TC:22, GC:24
21:13:57.379Z :Producer :Topic: topic1, TC:23, GC:25
21:13:57.385Z :Producer :Topic: topic1, TC:24, GC:26
21:13:57.391Z :Producer :Topic: topic1, TC:25, GC:27
21:13:57.398Z :Producer :Topic: topic1, TC:26, GC:28
21:13:57.404Z :Producer :Topic: topic1, TC:27, GC:29
21:13:57.411Z :Producer :Topic: topic1, TC:28, GC:30
21:13:57.417Z :Producer :Topic: topic1, TC:29, GC:31
21:13:57.423Z :Producer :Topic: topic1, TC:30, GC:32
21:13:57.430Z :Producer :Topic: topic2, TC:3, GC:33
21:13:57.432Z :Consumer :Topic: topic1, TC:3 OK, GC:3 OK
21:13:57.437Z :Producer :Topic: topic1, TC:31, GC:34
21:13:57.444Z :Producer :Topic: topic1, TC:32, GC:35
21:13:57.451Z :Producer :Topic: topic1, TC:33, GC:36
21:13:57.458Z :Producer :Topic: topic1, TC:34, GC:37
21:13:57.465Z :Producer :Topic: topic1, TC:35, GC:38
21:13:57.471Z :Producer :Topic: topic1, TC:36, GC:39
21:13:57.479Z :Producer :Topic: topic1, TC:37, GC:40
21:13:57.486Z :Producer :Topic: topic1, TC:38, GC:41
21:13:57.492Z :Producer :Topic: topic1, TC:39, GC:42
21:13:57.499Z :Producer :Topic: topic1, TC:40, GC:43
21:13:57.505Z :Producer :Topic: topic2, TC:4, GC:44
21:13:57.512Z :Producer :Topic: topic1, TC:41, GC:45
21:13:57.518Z :Producer :Topic: topic1, TC:42, GC:46
21:13:57.525Z :Producer :Topic: topic1, TC:43, GC:47
21:13:57.532Z :Producer :Topic: topic1, TC:44, GC:48
21:13:57.532Z :Consumer :Topic: topic1, TC:4 OK, GC:4 OK
21:13:57.539Z :Producer :Topic: topic1, TC:45, GC:49
21:13:57.547Z :Producer :Topic: topic1, TC:46, GC:50
21:13:57.553Z :Producer :Topic: topic1, TC:47, GC:51
21:13:57.560Z :Producer :Topic: topic1, TC:48, GC:52
21:13:57.567Z :Producer :Topic: topic1, TC:49, GC:53
21:13:57.573Z :Producer :Topic: topic1, TC:50, GC:54
21:13:57.579Z :Producer :Topic: topic2, TC:5, GC:55
21:13:57.586Z :Producer :Topic: topic1, TC:51, GC:56
21:13:57.592Z :Producer :Topic: topic1, TC:52, GC:57
21:13:57.599Z :Producer :Topic: topic1, TC:53, GC:58
21:13:57.606Z :Producer :Topic: topic1, TC:54, GC:59
21:13:57.613Z :Producer :Topic: topic1, TC:55, GC:60
21:13:57.619Z :Producer :Topic: topic1, TC:56, GC:61
21:13:57.625Z :Producer :Topic: topic1, TC:57, GC:62
21:13:57.632Z :Producer :Topic: topic1, TC:58, GC:63
21:13:57.634Z :Consumer :Topic: topic1, TC:5 OK, GC:5 OK
21:13:57.639Z :Producer :Topic: topic1, TC:59, GC:64
21:13:57.646Z :Producer :Topic: topic1, TC:60, GC:65
21:13:57.652Z :Producer :Topic: topic2, TC:6, GC:66
21:13:57.659Z :Producer :Topic: topic1, TC:61, GC:67
21:13:57.666Z :Producer :Topic: topic1, TC:62, GC:68
21:13:57.672Z :Producer :Topic: topic1, TC:63, GC:69
21:13:57.679Z :Producer :Topic: topic1, TC:64, GC:70
21:13:57.686Z :Producer :Topic: topic1, TC:65, GC:71
21:13:57.692Z :Producer :Topic: topic1, TC:66, GC:72
21:13:57.699Z :Producer :Topic: topic1, TC:67, GC:73
21:13:57.705Z :Producer :Topic: topic1, TC:68, GC:74
21:13:57.712Z :Producer :Topic: topic1, TC:69, GC:75
21:13:57.719Z :Producer :Topic: topic1, TC:70, GC:76
21:13:57.726Z :Producer :Topic: topic2, TC:7, GC:77
21:13:57.733Z :Producer :Topic: topic1, TC:71, GC:78
21:13:57.735Z :Consumer :Topic: topic1, TC:6 OK, GC:6 OK
21:13:57.740Z :Producer :Topic: topic1, TC:72, GC:79
21:13:57.747Z :Producer :Topic: topic1, TC:73, GC:80
21:13:57.753Z :Producer :Topic: topic1, TC:74, GC:81
21:13:57.759Z :Producer :Topic: topic1, TC:75, GC:82
21:13:57.766Z :Producer :Topic: topic1, TC:76, GC:83
21:13:57.771Z :Producer :Topic: topic1, TC:77, GC:84
21:13:57.776Z :Producer :Topic: topic1, TC:78, GC:85
21:13:57.783Z :Producer :Topic: topic1, TC:79, GC:86
21:13:57.788Z :Producer :Topic: topic1, TC:80, GC:87
21:13:57.795Z :Producer :Topic: topic2, TC:8, GC:88
21:13:57.802Z :Producer :Topic: topic1, TC:81, GC:89
21:13:57.809Z :Producer :Topic: topic1, TC:82, GC:90
21:13:57.816Z :Producer :Topic: topic1, TC:83, GC:91
21:13:57.822Z :Producer :Topic: topic1, TC:84, GC:92
21:13:57.828Z :Producer :Topic: topic1, TC:85, GC:93
21:13:57.835Z :Producer :Topic: topic1, TC:86, GC:94
21:13:57.837Z :Consumer :Topic: topic1, TC:7 OK, GC:7 OK
21:13:57.842Z :Producer :Topic: topic1, TC:87, GC:95
21:13:57.848Z :Producer :Topic: topic1, TC:88, GC:96
21:13:57.855Z :Producer :Topic: topic1, TC:89, GC:97
21:13:57.863Z :Producer :Topic: topic1, TC:90, GC:98
21:13:57.870Z :Producer :Topic: topic2, TC:9, GC:99
21:13:57.877Z :Producer :Topic: topic1, TC:91, GC:100
21:13:57.938Z :Consumer :Topic: topic1, TC:8 OK, GC:8 OK
21:13:58.039Z :Consumer :Topic: topic1, TC:9 OK, GC:9 OK
21:13:58.141Z :Consumer :Topic: topic1, TC:10 OK, GC:10 OK
21:13:58.242Z :Consumer :Topic: topic2, TC:1 OK, GC:11 OK
21:13:58.343Z :Consumer :Topic: topic1, TC:11 OK, GC:12 OK
21:13:58.444Z :Consumer :Topic: topic2, TC:2 OK, GC:22 JUMP FORWARD 9
21:13:58.545Z :Consumer :Topic: topic1, TC:12 OK, GC:13 JUMP BACKWARDS 9
21:13:58.646Z :Consumer :Topic: topic2, TC:3 OK, GC:33 JUMP FORWARD 19
21:13:58.747Z :Consumer :Topic: topic1, TC:13 OK, GC:14 JUMP BACKWARDS 19
21:13:58.848Z :Consumer :Topic: topic2, TC:4 OK, GC:44 JUMP FORWARD 29
21:13:58.949Z :Consumer :Topic: topic1, TC:14 OK, GC:15 JUMP BACKWARDS 29
21:13:59.050Z :Consumer :Topic: topic2, TC:5 OK, GC:55 JUMP FORWARD 39
21:13:59.150Z :Consumer :Topic: topic1, TC:15 OK, GC:16 JUMP BACKWARDS 39
21:13:59.252Z :Consumer :Topic: topic2, TC:6 OK, GC:66 JUMP FORWARD 49
21:13:59.353Z :Consumer :Topic: topic1, TC:16 OK, GC:17 JUMP BACKWARDS 49
21:13:59.454Z :Consumer :Topic: topic2, TC:7 OK, GC:77 JUMP FORWARD 59
21:13:59.555Z :Consumer :Topic: topic1, TC:17 OK, GC:18 JUMP BACKWARDS 59
21:13:59.656Z :Consumer :Topic: topic2, TC:8 OK, GC:88 JUMP FORWARD 69
21:13:59.757Z :Consumer :Topic: topic1, TC:18 OK, GC:19 JUMP BACKWARDS 69
21:13:59.858Z :Consumer :Topic: topic2, TC:9 OK, GC:99 JUMP FORWARD 79
21:13:59.959Z :Consumer :Topic: topic1, TC:19 OK, GC:20 JUMP BACKWARDS 79
21:14:00.060Z :Consumer :Topic: topic1, TC:20 OK, GC:21 OK
21:14:00.161Z :Consumer :Topic: topic1, TC:21 OK, GC:23 JUMP FORWARD 1
21:14:00.262Z :Consumer :Topic: topic1, TC:22 OK, GC:24 OK
21:14:00.363Z :Consumer :Topic: topic1, TC:23 OK, GC:25 OK
21:14:00.464Z :Consumer :Topic: topic1, TC:24 OK, GC:26 OK
21:14:00.565Z :Consumer :Topic: topic1, TC:25 OK, GC:27 OK
21:14:00.666Z :Consumer :Topic: topic1, TC:26 OK, GC:28 OK
21:14:00.767Z :Consumer :Topic: topic1, TC:27 OK, GC:29 OK
21:14:00.868Z :Consumer :Topic: topic1, TC:28 OK, GC:30 OK
21:14:00.968Z :Consumer :Topic: topic1, TC:29 OK, GC:31 OK
21:14:01.070Z :Consumer :Topic: topic1, TC:30 OK, GC:32 OK
21:14:01.171Z :Consumer :Topic: topic1, TC:31 OK, GC:34 JUMP FORWARD 1
21:14:01.272Z :Consumer :Topic: topic1, TC:32 OK, GC:35 OK
21:14:01.373Z :Consumer :Topic: topic1, TC:33 OK, GC:36 OK
21:14:01.474Z :Consumer :Topic: topic1, TC:34 OK, GC:37 OK
21:14:01.574Z :Consumer :Topic: topic1, TC:35 OK, GC:38 OK
21:14:01.675Z :Consumer :Topic: topic1, TC:36 OK, GC:39 OK
21:14:01.776Z :Consumer :Topic: topic1, TC:37 OK, GC:40 OK
21:14:01.877Z :Consumer :Topic: topic1, TC:38 OK, GC:41 OK
21:14:01.979Z :Consumer :Topic: topic1, TC:39 OK, GC:42 OK
21:14:02.080Z :Consumer :Topic: topic1, TC:40 OK, GC:43 OK
21:14:02.181Z :Consumer :Topic: topic1, TC:41 OK, GC:45 JUMP FORWARD 1
21:14:02.282Z :Consumer :Topic: topic1, TC:42 OK, GC:46 OK
21:14:02.383Z :Consumer :Topic: topic1, TC:43 OK, GC:47 OK
21:14:02.484Z :Consumer :Topic: topic1, TC:44 OK, GC:48 OK
21:14:02.585Z :Consumer :Topic: topic1, TC:45 OK, GC:49 OK
21:14:02.686Z :Consumer :Topic: topic1, TC:46 OK, GC:50 OK
21:14:02.787Z :Consumer :Topic: topic1, TC:47 OK, GC:51 OK
21:14:02.889Z :Consumer :Topic: topic1, TC:48 OK, GC:52 OK
21:14:02.989Z :Consumer :Topic: topic1, TC:49 OK, GC:53 OK
21:14:03.090Z :Consumer :Topic: topic1, TC:50 OK, GC:54 OK
21:14:03.192Z :Consumer :Topic: topic1, TC:51 OK, GC:56 JUMP FORWARD 1
21:14:03.293Z :Consumer :Topic: topic1, TC:52 OK, GC:57 OK
21:14:03.394Z :Consumer :Topic: topic1, TC:53 OK, GC:58 OK
21:14:03.495Z :Consumer :Topic: topic1, TC:54 OK, GC:59 OK
21:14:03.596Z :Consumer :Topic: topic1, TC:55 OK, GC:60 OK
21:14:03.697Z :Consumer :Topic: topic1, TC:56 OK, GC:61 OK
21:14:03.798Z :Consumer :Topic: topic1, TC:57 OK, GC:62 OK
21:14:03.899Z :Consumer :Topic: topic1, TC:58 OK, GC:63 OK
21:14:04Z :Consumer :Topic: topic1, TC:59 OK, GC:64 OK
21:14:04.101Z :Consumer :Topic: topic1, TC:60 OK, GC:65 OK
21:14:04.202Z :Consumer :Topic: topic1, TC:61 OK, GC:67 JUMP FORWARD 1
21:14:04.303Z :Consumer :Topic: topic1, TC:62 OK, GC:68 OK
21:14:04.404Z :Consumer :Topic: topic1, TC:63 OK, GC:69 OK
21:14:04.505Z :Consumer :Topic: topic1, TC:64 OK, GC:70 OK
21:14:04.606Z :Consumer :Topic: topic1, TC:65 OK, GC:71 OK
21:14:04.707Z :Consumer :Topic: topic1, TC:66 OK, GC:72 OK
21:14:04.808Z :Consumer :Topic: topic1, TC:67 OK, GC:73 OK
21:14:04.909Z :Consumer :Topic: topic1, TC:68 OK, GC:74 OK
21:14:05.010Z :Consumer :Topic: topic1, TC:69 OK, GC:75 OK
21:14:05.112Z :Consumer :Topic: topic1, TC:70 OK, GC:76 OK
21:14:05.213Z :Consumer :Topic: topic1, TC:71 OK, GC:78 JUMP FORWARD 1
21:14:05.314Z :Consumer :Topic: topic1, TC:72 OK, GC:79 OK
21:14:05.415Z :Consumer :Topic: topic1, TC:73 OK, GC:80 OK
21:14:05.516Z :Consumer :Topic: topic1, TC:74 OK, GC:81 OK
21:14:05.617Z :Consumer :Topic: topic1, TC:75 OK, GC:82 OK
21:14:05.717Z :Consumer :Topic: topic1, TC:76 OK, GC:83 OK
21:14:05.818Z :Consumer :Topic: topic1, TC:77 OK, GC:84 OK
21:14:05.920Z :Consumer :Topic: topic1, TC:78 OK, GC:85 OK
21:14:06.021Z :Consumer :Topic: topic1, TC:79 OK, GC:86 OK
21:14:06.122Z :Consumer :Topic: topic1, TC:80 OK, GC:87 OK
21:14:06.223Z :Consumer :Topic: topic1, TC:81 OK, GC:89 JUMP FORWARD 1
21:14:06.324Z :Consumer :Topic: topic1, TC:82 OK, GC:90 OK
21:14:06.425Z :Consumer :Topic: topic1, TC:83 OK, GC:91 OK
21:14:06.526Z :Consumer :Topic: topic1, TC:84 OK, GC:92 OK
21:14:06.627Z :Consumer :Topic: topic1, TC:85 OK, GC:93 OK
21:14:06.728Z :Consumer :Topic: topic1, TC:86 OK, GC:94 OK
21:14:06.829Z :Consumer :Topic: topic1, TC:87 OK, GC:95 OK
21:14:06.930Z :Consumer :Topic: topic1, TC:88 OK, GC:96 OK
21:14:07.031Z :Consumer :Topic: topic1, TC:89 OK, GC:97 OK
21:14:07.133Z :Consumer :Topic: topic1, TC:90 OK, GC:98 OK
21:14:07.233Z :Consumer :Topic: topic1, TC:91 OK, GC:100 JUMP FORWARD 1
本轮测试的结果中存在大量的消息乱序。consumer 以相同速度消费每个 topic 中的消息,这就意味着仅包含 10 条消息的 topic2 会比较快地被消费,导致全局消息排序出现大规模的前后跳跃。直到所有 topic 的消息被消费,topic 1 每 10 个消息出现 1 次 JUMP FORWARDS 1,因为 topic 2 的消息已被消费。
所以可以看出,consumer 需要实时消费消息,才能获得不错的消息排序效果。
这也把我们带到了“时间旅行”的测试。显然它不是实时的,所以我们一定会看到排序问题。
测试 9:时间旅行,消息等比到达
本轮测试中,producer 将发送全部 100 条消息,然后 consumer 再从头读取 topic 的消息。
21:27:30.167Z :Producer :Topic: topic1, TC:1, GC:1
21:27:30.176Z :Producer :Topic: topic2, TC:1, GC:2
21:27:30.183Z :Producer :Topic: topic1, TC:2, GC:3
21:27:30.191Z :Producer :Topic: topic2, TC:2, GC:4
21:27:30.198Z :Producer :Topic: topic1, TC:3, GC:5
21:27:30.204Z :Producer :Topic: topic2, TC:3, GC:6
21:27:30.212Z :Producer :Topic: topic1, TC:4, GC:7
21:27:30.219Z :Producer :Topic: topic2, TC:4, GC:8
21:27:30.229Z :Producer :Topic: topic1, TC:5, GC:9
21:27:30.238Z :Producer :Topic: topic2, TC:5, GC:10
21:27:30.244Z :Producer :Topic: topic1, TC:6, GC:11
21:27:30.252Z :Producer :Topic: topic2, TC:6, GC:12
21:27:30.259Z :Producer :Topic: topic1, TC:7, GC:13
21:27:30.266Z :Producer :Topic: topic2, TC:7, GC:14
21:27:30.273Z :Producer :Topic: topic1, TC:8, GC:15
21:27:30.280Z :Producer :Topic: topic2, TC:8, GC:16
21:27:30.287Z :Producer :Topic: topic1, TC:9, GC:17
21:27:30.293Z :Producer :Topic: topic2, TC:9, GC:18
21:27:30.300Z :Producer :Topic: topic1, TC:10, GC:19
21:27:30.307Z :Producer :Topic: topic2, TC:10, GC:20
21:27:30.314Z :Producer :Topic: topic1, TC:11, GC:21
21:27:30.321Z :Producer :Topic: topic2, TC:11, GC:22
21:27:30.328Z :Producer :Topic: topic1, TC:12, GC:23
21:27:30.335Z :Producer :Topic: topic2, TC:12, GC:24
21:27:30.341Z :Producer :Topic: topic1, TC:13, GC:25
21:27:30.347Z :Producer :Topic: topic2, TC:13, GC:26
21:27:30.354Z :Producer :Topic: topic1, TC:14, GC:27
21:27:30.359Z :Producer :Topic: topic2, TC:14, GC:28
21:27:30.365Z :Producer :Topic: topic1, TC:15, GC:29
21:27:30.371Z :Producer :Topic: topic2, TC:15, GC:30
21:27:30.378Z :Producer :Topic: topic1, TC:16, GC:31
21:27:30.384Z :Producer :Topic: topic2, TC:16, GC:32
21:27:30.391Z :Producer :Topic: topic1, TC:17, GC:33
21:27:30.398Z :Producer :Topic: topic2, TC:17, GC:34
21:27:30.404Z :Producer :Topic: topic1, TC:18, GC:35
21:27:30.411Z :Producer :Topic: topic2, TC:18, GC:36
21:27:30.417Z :Producer :Topic: topic1, TC:19, GC:37
21:27:30.423Z :Producer :Topic: topic2, TC:19, GC:38
21:27:30.429Z :Producer :Topic: topic1, TC:20, GC:39
21:27:30.436Z :Producer :Topic: topic2, TC:20, GC:40
21:27:30.443Z :Producer :Topic: topic1, TC:21, GC:41
21:27:30.450Z :Producer :Topic: topic2, TC:21, GC:42
21:27:30.456Z :Producer :Topic: topic1, TC:22, GC:43
21:27:30.463Z :Producer :Topic: topic2, TC:22, GC:44
21:27:30.470Z :Producer :Topic: topic1, TC:23, GC:45
21:27:30.476Z :Producer :Topic: topic2, TC:23, GC:46
21:27:30.482Z :Producer :Topic: topic1, TC:24, GC:47
21:27:30.488Z :Producer :Topic: topic2, TC:24, GC:48
21:27:30.494Z :Producer :Topic: topic1, TC:25, GC:49
21:27:30.500Z :Producer :Topic: topic2, TC:25, GC:50
21:27:30.506Z :Producer :Topic: topic1, TC:26, GC:51
21:27:30.512Z :Producer :Topic: topic2, TC:26, GC:52
21:27:30.518Z :Producer :Topic: topic1, TC:27, GC:53
21:27:30.524Z :Producer :Topic: topic2, TC:27, GC:54
21:27:30.530Z :Producer :Topic: topic1, TC:28, GC:55
21:27:30.537Z :Producer :Topic: topic2, TC:28, GC:56
21:27:30.542Z :Producer :Topic: topic1, TC:29, GC:57
21:27:30.549Z :Producer :Topic: topic2, TC:29, GC:58
21:27:30.555Z :Producer :Topic: topic1, TC:30, GC:59
21:27:30.561Z :Producer :Topic: topic2, TC:30, GC:60
21:27:30.567Z :Producer :Topic: topic1, TC:31, GC:61
21:27:30.573Z :Producer :Topic: topic2, TC:31, GC:62
21:27:30.580Z :Producer :Topic: topic1, TC:32, GC:63
21:27:30.586Z :Producer :Topic: topic2, TC:32, GC:64
21:27:30.592Z :Producer :Topic: topic1, TC:33, GC:65
21:27:30.598Z :Producer :Topic: topic2, TC:33, GC:66
21:27:30.604Z :Producer :Topic: topic1, TC:34, GC:67
21:27:30.611Z :Producer :Topic: topic2, TC:34, GC:68
21:27:30.618Z :Producer :Topic: topic1, TC:35, GC:69
21:27:30.624Z :Producer :Topic: topic2, TC:35, GC:70
21:27:30.631Z :Producer :Topic: topic1, TC:36, GC:71
21:27:30.637Z :Producer :Topic: topic2, TC:36, GC:72
21:27:30.644Z :Producer :Topic: topic1, TC:37, GC:73
21:27:30.651Z :Producer :Topic: topic2, TC:37, GC:74
21:27:30.658Z :Producer :Topic: topic1, TC:38, GC:75
21:27:30.665Z :Producer :Topic: topic2, TC:38, GC:76
21:27:30.672Z :Producer :Topic: topic1, TC:39, GC:77
21:27:30.679Z :Producer :Topic: topic2, TC:39, GC:78
21:27:30.686Z :Producer :Topic: topic1, TC:40, GC:79
21:27:30.693Z :Producer :Topic: topic2, TC:40, GC:80
21:27:30.700Z :Producer :Topic: topic1, TC:41, GC:81
21:27:30.706Z :Producer :Topic: topic2, TC:41, GC:82
21:27:30.713Z :Producer :Topic: topic1, TC:42, GC:83
21:27:30.721Z :Producer :Topic: topic2, TC:42, GC:84
21:27:30.727Z :Producer :Topic: topic1, TC:43, GC:85
21:27:30.733Z :Producer :Topic: topic2, TC:43, GC:86
21:27:30.739Z :Producer :Topic: topic1, TC:44, GC:87
21:27:30.744Z :Producer :Topic: topic2, TC:44, GC:88
21:27:30.750Z :Producer :Topic: topic1, TC:45, GC:89
21:27:30.756Z :Producer :Topic: topic2, TC:45, GC:90
21:27:30.762Z :Producer :Topic: topic1, TC:46, GC:91
21:27:30.769Z :Producer :Topic: topic2, TC:46, GC:92
21:27:30.774Z :Producer :Topic: topic1, TC:47, GC:93
21:27:30.780Z :Producer :Topic: topic2, TC:47, GC:94
21:27:30.787Z :Producer :Topic: topic1, TC:48, GC:95
21:27:30.793Z :Producer :Topic: topic2, TC:48, GC:96
21:27:30.799Z :Producer :Topic: topic1, TC:49, GC:97
21:27:30.805Z :Producer :Topic: topic2, TC:49, GC:98
21:27:30.811Z :Producer :Topic: topic1, TC:50, GC:99
21:27:30.816Z :Producer :Topic: topic2, TC:50, GC:100
21:27:30.830Z :Consumer :Subscribing
21:27:30.937Z :Consumer :Consuming
21:27:30.961Z :Consumer :Topic: topic2, TC:1 OK, GC:2 JUMP FORWARD 1
21:27:30.966Z :Consumer :Topic: topic2, TC:2 OK, GC:4 JUMP FORWARD 1
21:27:30.966Z :Consumer :Topic: topic2, TC:3 OK, GC:6 JUMP FORWARD 1
21:27:30.966Z :Consumer :Topic: topic2, TC:4 OK, GC:8 JUMP FORWARD 1
21:27:30.966Z :Consumer :Topic: topic2, TC:5 OK, GC:10 JUMP FORWARD 1
21:27:30.966Z :Consumer :Topic: topic2, TC:6 OK, GC:12 JUMP FORWARD 1
21:27:30.966Z :Consumer :Topic: topic2, TC:7 OK, GC:14 JUMP FORWARD 1
21:27:30.966Z :Consumer :Topic: topic2, TC:8 OK, GC:16 JUMP FORWARD 1
21:27:30.967Z :Consumer :Topic: topic2, TC:9 OK, GC:18 JUMP FORWARD 1
21:27:30.968Z :Consumer :Topic: topic2, TC:10 OK, GC:20 JUMP FORWARD 1
21:27:30.968Z :Consumer :Topic: topic2, TC:11 OK, GC:22 JUMP FORWARD 1
21:27:30.968Z :Consumer :Topic: topic2, TC:12 OK, GC:24 JUMP FORWARD 1
21:27:30.968Z :Consumer :Topic: topic2, TC:13 OK, GC:26 JUMP FORWARD 1
21:27:30.968Z :Consumer :Topic: topic2, TC:14 OK, GC:28 JUMP FORWARD 1
21:27:30.968Z :Consumer :Topic: topic2, TC:15 OK, GC:30 JUMP FORWARD 1
21:27:30.969Z :Consumer :Topic: topic2, TC:16 OK, GC:32 JUMP FORWARD 1
21:27:30.969Z :Consumer :Topic: topic2, TC:17 OK, GC:34 JUMP FORWARD 1
21:27:30.969Z :Consumer :Topic: topic2, TC:18 OK, GC:36 JUMP FORWARD 1
21:27:30.969Z :Consumer :Topic: topic2, TC:19 OK, GC:38 JUMP FORWARD 1
21:27:30.970Z :Consumer :Topic: topic2, TC:20 OK, GC:40 JUMP FORWARD 1
21:27:30.970Z :Consumer :Topic: topic2, TC:21 OK, GC:42 JUMP FORWARD 1
21:27:30.970Z :Consumer :Topic: topic2, TC:22 OK, GC:44 JUMP FORWARD 1
21:27:30.971Z :Consumer :Topic: topic2, TC:23 OK, GC:46 JUMP FORWARD 1
21:27:30.971Z :Consumer :Topic: topic2, TC:24 OK, GC:48 JUMP FORWARD 1
21:27:30.971Z :Consumer :Topic: topic2, TC:25 OK, GC:50 JUMP FORWARD 1
21:27:30.971Z :Consumer :Topic: topic2, TC:26 OK, GC:52 JUMP FORWARD 1
21:27:30.971Z :Consumer :Topic: topic2, TC:27 OK, GC:54 JUMP FORWARD 1
21:27:30.971Z :Consumer :Topic: topic2, TC:28 OK, GC:56 JUMP FORWARD 1
21:27:30.972Z :Consumer :Topic: topic2, TC:29 OK, GC:58 JUMP FORWARD 1
21:27:30.972Z :Consumer :Topic: topic2, TC:30 OK, GC:60 JUMP FORWARD 1
21:27:30.972Z :Consumer :Topic: topic2, TC:31 OK, GC:62 JUMP FORWARD 1
21:27:30.972Z :Consumer :Topic: topic2, TC:32 OK, GC:64 JUMP FORWARD 1
21:27:30.972Z :Consumer :Topic: topic2, TC:33 OK, GC:66 JUMP FORWARD 1
21:27:30.972Z :Consumer :Topic: topic2, TC:34 OK, GC:68 JUMP FORWARD 1
21:27:30.973Z :Consumer :Topic: topic2, TC:35 OK, GC:70 JUMP FORWARD 1
21:27:30.973Z :Consumer :Topic: topic2, TC:36 OK, GC:72 JUMP FORWARD 1
21:27:30.973Z :Consumer :Topic: topic2, TC:37 OK, GC:74 JUMP FORWARD 1
21:27:30.973Z :Consumer :Topic: topic2, TC:38 OK, GC:76 JUMP FORWARD 1
21:27:30.973Z :Consumer :Topic: topic2, TC:39 OK, GC:78 JUMP FORWARD 1
21:27:30.973Z :Consumer :Topic: topic2, TC:40 OK, GC:80 JUMP FORWARD 1
21:27:30.973Z :Consumer :Topic: topic2, TC:41 OK, GC:82 JUMP FORWARD 1
21:27:30.974Z :Consumer :Topic: topic2, TC:42 OK, GC:84 JUMP FORWARD 1
21:27:30.974Z :Consumer :Topic: topic2, TC:43 OK, GC:86 JUMP FORWARD 1
21:27:30.974Z :Consumer :Topic: topic2, TC:44 OK, GC:88 JUMP FORWARD 1
21:27:30.974Z :Consumer :Topic: topic2, TC:45 OK, GC:90 JUMP FORWARD 1
21:27:30.975Z :Consumer :Topic: topic2, TC:46 OK, GC:92 JUMP FORWARD 1
21:27:30.975Z :Consumer :Topic: topic2, TC:47 OK, GC:94 JUMP FORWARD 1
21:27:30.975Z :Consumer :Topic: topic2, TC:48 OK, GC:96 JUMP FORWARD 1
21:27:30.975Z :Consumer :Topic: topic2, TC:49 OK, GC:98 JUMP FORWARD 1
21:27:30.975Z :Consumer :Topic: topic2, TC:50 OK, GC:100 JUMP FORWARD 1
21:27:30.975Z :Consumer :Topic: topic1, TC:1 OK, GC:1 JUMP BACKWARDS 99
21:27:30.975Z :Consumer :Topic: topic1, TC:2 OK, GC:3 JUMP FORWARD 1
21:27:30.976Z :Consumer :Topic: topic1, TC:3 OK, GC:5 JUMP FORWARD 1
21:27:30.976Z :Consumer :Topic: topic1, TC:4 OK, GC:7 JUMP FORWARD 1
21:27:30.976Z :Consumer :Topic: topic1, TC:5 OK, GC:9 JUMP FORWARD 1
21:27:30.977Z :Consumer :Topic: topic1, TC:6 OK, GC:11 JUMP FORWARD 1
21:27:30.977Z :Consumer :Topic: topic1, TC:7 OK, GC:13 JUMP FORWARD 1
21:27:30.977Z :Consumer :Topic: topic1, TC:8 OK, GC:15 JUMP FORWARD 1
21:27:30.977Z :Consumer :Topic: topic1, TC:9 OK, GC:17 JUMP FORWARD 1
21:27:30.977Z :Consumer :Topic: topic1, TC:10 OK, GC:19 JUMP FORWARD 1
21:27:30.978Z :Consumer :Topic: topic1, TC:11 OK, GC:21 JUMP FORWARD 1
21:27:30.978Z :Consumer :Topic: topic1, TC:12 OK, GC:23 JUMP FORWARD 1
21:27:30.978Z :Consumer :Topic: topic1, TC:13 OK, GC:25 JUMP FORWARD 1
21:27:30.978Z :Consumer :Topic: topic1, TC:14 OK, GC:27 JUMP FORWARD 1
21:27:30.978Z :Consumer :Topic: topic1, TC:15 OK, GC:29 JUMP FORWARD 1
21:27:30.978Z :Consumer :Topic: topic1, TC:16 OK, GC:31 JUMP FORWARD 1
21:27:30.978Z :Consumer :Topic: topic1, TC:17 OK, GC:33 JUMP FORWARD 1
21:27:30.978Z :Consumer :Topic: topic1, TC:18 OK, GC:35 JUMP FORWARD 1
21:27:30.978Z :Consumer :Topic: topic1, TC:19 OK, GC:37 JUMP FORWARD 1
21:27:30.978Z :Consumer :Topic: topic1, TC:20 OK, GC:39 JUMP FORWARD 1
21:27:30.979Z :Consumer :Topic: topic1, TC:21 OK, GC:41 JUMP FORWARD 1
21:27:30.979Z :Consumer :Topic: topic1, TC:22 OK, GC:43 JUMP FORWARD 1
21:27:30.979Z :Consumer :Topic: topic1, TC:23 OK, GC:45 JUMP FORWARD 1
21:27:30.979Z :Consumer :Topic: topic1, TC:24 OK, GC:47 JUMP FORWARD 1
21:27:30.979Z :Consumer :Topic: topic1, TC:25 OK, GC:49 JUMP FORWARD 1
21:27:30.979Z :Consumer :Topic: topic1, TC:26 OK, GC:51 JUMP FORWARD 1
21:27:30.979Z :Consumer :Topic: topic1, TC:27 OK, GC:53 JUMP FORWARD 1
21:27:30.979Z :Consumer :Topic: topic1, TC:28 OK, GC:55 JUMP FORWARD 1
21:27:30.980Z :Consumer :Topic: topic1, TC:29 OK, GC:57 JUMP FORWARD 1
21:27:30.980Z :Consumer :Topic: topic1, TC:30 OK, GC:59 JUMP FORWARD 1
21:27:30.980Z :Consumer :Topic: topic1, TC:31 OK, GC:61 JUMP FORWARD 1
21:27:30.980Z :Consumer :Topic: topic1, TC:32 OK, GC:63 JUMP FORWARD 1
21:27:30.980Z :Consumer :Topic: topic1, TC:33 OK, GC:65 JUMP FORWARD 1
21:27:30.980Z :Consumer :Topic: topic1, TC:34 OK, GC:67 JUMP FORWARD 1
21:27:30.980Z :Consumer :Topic: topic1, TC:35 OK, GC:69 JUMP FORWARD 1
21:27:30.980Z :Consumer :Topic: topic1, TC:36 OK, GC:71 JUMP FORWARD 1
21:27:30.981Z :Consumer :Topic: topic1, TC:37 OK, GC:73 JUMP FORWARD 1
21:27:30.981Z :Consumer :Topic: topic1, TC:38 OK, GC:75 JUMP FORWARD 1
21:27:30.981Z :Consumer :Topic: topic1, TC:39 OK, GC:77 JUMP FORWARD 1
21:27:30.981Z :Consumer :Topic: topic1, TC:40 OK, GC:79 JUMP FORWARD 1
21:27:30.981Z :Consumer :Topic: topic1, TC:41 OK, GC:81 JUMP FORWARD 1
21:27:30.982Z :Consumer :Topic: topic1, TC:42 OK, GC:83 JUMP FORWARD 1
21:27:30.982Z :Consumer :Topic: topic1, TC:43 OK, GC:85 JUMP FORWARD 1
21:27:30.982Z :Consumer :Topic: topic1, TC:44 OK, GC:87 JUMP FORWARD 1
21:27:30.983Z :Consumer :Topic: topic1, TC:45 OK, GC:89 JUMP FORWARD 1
21:27:30.983Z :Consumer :Topic: topic1, TC:46 OK, GC:91 JUMP FORWARD 1
21:27:30.983Z :Consumer :Topic: topic1, TC:47 OK, GC:93 JUMP FORWARD 1
21:27:30.983Z :Consumer :Topic: topic1, TC:48 OK, GC:95 JUMP FORWARD 1
21:27:30.983Z :Consumer :Topic: topic1, TC:49 OK, GC:97 JUMP FORWARD 1
21:27:30.983Z :Consumer :Topic: topic1, TC:50 OK, GC:99 JUMP FORWARD 1
我们看到,consumer 首先消费 topic2,然后消费 topic1,这个顺序与 Kafka 完全相同。
但是,当消息数从 100 条增加到 10000 条时,我们看到的情况是,consumer 首先处理 1 个 topic 的延伸,然后转到其他 topic,在二者之间进行轮询,然后再处理 1 个 topic 的延伸。全局消息排序的前后跳跃不超过 2000 ——这一点比 Kafka 做得好。Kafka 的处理方式是,先处理一个完整的 topic(独立分层),然后再处理下一个,尽管 topic 可能很大。
如果你认为 topic 之间的排序是有益的,则可以根据自己的实际情况,将其分类为“有用”或者“无用”。如果你有澎湃的消息流,每分钟成千上万那种,那么确实可以得到更好的消息排序;如果你只有一条涓涓细流,那么它可能不太会是“有用”的排序。
测试 10:时间旅行,消息不等比到达
与测试 9 相似,但是不同之处是,topic 1 与 topic 2 的消息到达比为 10:1。
21:36:16.014Z :Producer :Topic: topic1, TC:1, GC:1
21:36:16.024Z :Producer :Topic: topic1, TC:2, GC:2
21:36:16.031Z :Producer :Topic: topic1, TC:3, GC:3
21:36:16.038Z :Producer :Topic: topic1, TC:4, GC:4
21:36:16.045Z :Producer :Topic: topic1, TC:5, GC:5
21:36:16.052Z :Producer :Topic: topic1, TC:6, GC:6
21:36:16.058Z :Producer :Topic: topic1, TC:7, GC:7
21:36:16.066Z :Producer :Topic: topic1, TC:8, GC:8
21:36:16.073Z :Producer :Topic: topic1, TC:9, GC:9
21:36:16.080Z :Producer :Topic: topic1, TC:10, GC:10
21:36:16.088Z :Producer :Topic: topic2, TC:1, GC:11
21:36:16.095Z :Producer :Topic: topic1, TC:11, GC:12
21:36:16.101Z :Producer :Topic: topic1, TC:12, GC:13
21:36:16.107Z :Producer :Topic: topic1, TC:13, GC:14
21:36:16.115Z :Producer :Topic: topic1, TC:14, GC:15
21:36:16.121Z :Producer :Topic: topic1, TC:15, GC:16
21:36:16.129Z :Producer :Topic: topic1, TC:16, GC:17
21:36:16.135Z :Producer :Topic: topic1, TC:17, GC:18
21:36:16.141Z :Producer :Topic: topic1, TC:18, GC:19
21:36:16.148Z :Producer :Topic: topic1, TC:19, GC:20
21:36:16.155Z :Producer :Topic: topic1, TC:20, GC:21
21:36:16.162Z :Producer :Topic: topic2, TC:2, GC:22
21:36:16.168Z :Producer :Topic: topic1, TC:21, GC:23
21:36:16.174Z :Producer :Topic: topic1, TC:22, GC:24
21:36:16.181Z :Producer :Topic: topic1, TC:23, GC:25
21:36:16.188Z :Producer :Topic: topic1, TC:24, GC:26
21:36:16.193Z :Producer :Topic: topic1, TC:25, GC:27
21:36:16.200Z :Producer :Topic: topic1, TC:26, GC:28
21:36:16.206Z :Producer :Topic: topic1, TC:27, GC:29
21:36:16.212Z :Producer :Topic: topic1, TC:28, GC:30
21:36:16.220Z :Producer :Topic: topic1, TC:29, GC:31
21:36:16.226Z :Producer :Topic: topic1, TC:30, GC:32
21:36:16.233Z :Producer :Topic: topic2, TC:3, GC:33
21:36:16.239Z :Producer :Topic: topic1, TC:31, GC:34
21:36:16.246Z :Producer :Topic: topic1, TC:32, GC:35
21:36:16.252Z :Producer :Topic: topic1, TC:33, GC:36
21:36:16.259Z :Producer :Topic: topic1, TC:34, GC:37
21:36:16.266Z :Producer :Topic: topic1, TC:35, GC:38
21:36:16.272Z :Producer :Topic: topic1, TC:36, GC:39
21:36:16.279Z :Producer :Topic: topic1, TC:37, GC:40
21:36:16.285Z :Producer :Topic: topic1, TC:38, GC:41
21:36:16.292Z :Producer :Topic: topic1, TC:39, GC:42
21:36:16.299Z :Producer :Topic: topic1, TC:40, GC:43
21:36:16.306Z :Producer :Topic: topic2, TC:4, GC:44
21:36:16.312Z :Producer :Topic: topic1, TC:41, GC:45
21:36:16.318Z :Producer :Topic: topic1, TC:42, GC:46
21:36:16.326Z :Producer :Topic: topic1, TC:43, GC:47
21:36:16.333Z :Producer :Topic: topic1, TC:44, GC:48
21:36:16.339Z :Producer :Topic: topic1, TC:45, GC:49
21:36:16.345Z :Producer :Topic: topic1, TC:46, GC:50
21:36:16.351Z :Producer :Topic: topic1, TC:47, GC:51
21:36:16.357Z :Producer :Topic: topic1, TC:48, GC:52
21:36:16.364Z :Producer :Topic: topic1, TC:49, GC:53
21:36:16.371Z :Producer :Topic: topic1, TC:50, GC:54
21:36:16.377Z :Producer :Topic: topic2, TC:5, GC:55
21:36:16.383Z :Producer :Topic: topic1, TC:51, GC:56
21:36:16.391Z :Producer :Topic: topic1, TC:52, GC:57
21:36:16.398Z :Producer :Topic: topic1, TC:53, GC:58
21:36:16.404Z :Producer :Topic: topic1, TC:54, GC:59
21:36:16.411Z :Producer :Topic: topic1, TC:55, GC:60
21:36:16.418Z :Producer :Topic: topic1, TC:56, GC:61
21:36:16.425Z :Producer :Topic: topic1, TC:57, GC:62
21:36:16.432Z :Producer :Topic: topic1, TC:58, GC:63
21:36:16.439Z :Producer :Topic: topic1, TC:59, GC:64
21:36:16.446Z :Producer :Topic: topic1, TC:60, GC:65
21:36:16.452Z :Producer :Topic: topic2, TC:6, GC:66
21:36:16.458Z :Producer :Topic: topic1, TC:61, GC:67
21:36:16.465Z :Producer :Topic: topic1, TC:62, GC:68
21:36:16.471Z :Producer :Topic: topic1, TC:63, GC:69
21:36:16.478Z :Producer :Topic: topic1, TC:64, GC:70
21:36:16.483Z :Producer :Topic: topic1, TC:65, GC:71
21:36:16.489Z :Producer :Topic: topic1, TC:66, GC:72
21:36:16.496Z :Producer :Topic: topic1, TC:67, GC:73
21:36:16.502Z :Producer :Topic: topic1, TC:68, GC:74
21:36:16.508Z :Producer :Topic: topic1, TC:69, GC:75
21:36:16.515Z :Producer :Topic: topic1, TC:70, GC:76
21:36:16.522Z :Producer :Topic: topic2, TC:7, GC:77
21:36:16.528Z :Producer :Topic: topic1, TC:71, GC:78
21:36:16.535Z :Producer :Topic: topic1, TC:72, GC:79
21:36:16.542Z :Producer :Topic: topic1, TC:73, GC:80
21:36:16.548Z :Producer :Topic: topic1, TC:74, GC:81
21:36:16.553Z :Producer :Topic: topic1, TC:75, GC:82
21:36:16.559Z :Producer :Topic: topic1, TC:76, GC:83
21:36:16.565Z :Producer :Topic: topic1, TC:77, GC:84
21:36:16.570Z :Producer :Topic: topic1, TC:78, GC:85
21:36:16.576Z :Producer :Topic: topic1, TC:79, GC:86
21:36:16.582Z :Producer :Topic: topic1, TC:80, GC:87
21:36:16.589Z :Producer :Topic: topic2, TC:8, GC:88
21:36:16.596Z :Producer :Topic: topic1, TC:81, GC:89
21:36:16.602Z :Producer :Topic: topic1, TC:82, GC:90
21:36:16.609Z :Producer :Topic: topic1, TC:83, GC:91
21:36:16.615Z :Producer :Topic: topic1, TC:84, GC:92
21:36:16.621Z :Producer :Topic: topic1, TC:85, GC:93
21:36:16.627Z :Producer :Topic: topic1, TC:86, GC:94
21:36:16.634Z :Producer :Topic: topic1, TC:87, GC:95
21:36:16.640Z :Producer :Topic: topic1, TC:88, GC:96
21:36:16.647Z :Producer :Topic: topic1, TC:89, GC:97
21:36:16.652Z :Producer :Topic: topic1, TC:90, GC:98
21:36:16.659Z :Producer :Topic: topic2, TC:9, GC:99
21:36:16.666Z :Producer :Topic: topic1, TC:91, GC:100
21:36:16.679Z :Consumer :Subscribing
21:36:16.792Z :Consumer :Consuming
21:36:16.800Z :Consumer :Topic: topic2, TC:1 OK, GC:11 JUMP FORWARD 10
21:36:16.801Z :Consumer :Topic: topic2, TC:2 OK, GC:22 JUMP FORWARD 10
21:36:16.801Z :Consumer :Topic: topic2, TC:3 OK, GC:33 JUMP FORWARD 10
21:36:16.801Z :Consumer :Topic: topic2, TC:4 OK, GC:44 JUMP FORWARD 10
21:36:16.802Z :Consumer :Topic: topic2, TC:5 OK, GC:55 JUMP FORWARD 10
21:36:16.803Z :Consumer :Topic: topic2, TC:6 OK, GC:66 JUMP FORWARD 10
21:36:16.803Z :Consumer :Topic: topic2, TC:7 OK, GC:77 JUMP FORWARD 10
21:36:16.803Z :Consumer :Topic: topic2, TC:8 OK, GC:88 JUMP FORWARD 10
21:36:16.804Z :Consumer :Topic: topic2, TC:9 OK, GC:99 JUMP FORWARD 10
21:36:16.804Z :Consumer :Topic: topic1, TC:1 OK, GC:1 JUMP BACKWARDS 98
21:36:16.813Z :Consumer :Topic: topic1, TC:2 OK, GC:2 OK
21:36:16.813Z :Consumer :Topic: topic1, TC:3 OK, GC:3 OK
21:36:16.813Z :Consumer :Topic: topic1, TC:4 OK, GC:4 OK
21:36:16.814Z :Consumer :Topic: topic1, TC:5 OK, GC:5 OK
21:36:16.814Z :Consumer :Topic: topic1, TC:6 OK, GC:6 OK
21:36:16.814Z :Consumer :Topic: topic1, TC:7 OK, GC:7 OK
21:36:16.814Z :Consumer :Topic: topic1, TC:8 OK, GC:8 OK
21:36:16.814Z :Consumer :Topic: topic1, TC:9 OK, GC:9 OK
21:36:16.814Z :Consumer :Topic: topic1, TC:10 OK, GC:10 OK
21:36:16.815Z :Consumer :Topic: topic1, TC:11 OK, GC:12 JUMP FORWARD 1
21:36:16.816Z :Consumer :Topic: topic1, TC:12 OK, GC:13 OK
21:36:16.817Z :Consumer :Topic: topic1, TC:13 OK, GC:14 OK
21:36:16.817Z :Consumer :Topic: topic1, TC:14 OK, GC:15 OK
21:36:16.817Z :Consumer :Topic: topic1, TC:15 OK, GC:16 OK
21:36:16.817Z :Consumer :Topic: topic1, TC:16 OK, GC:17 OK
21:36:16.817Z :Consumer :Topic: topic1, TC:17 OK, GC:18 OK
21:36:16.817Z :Consumer :Topic: topic1, TC:18 OK, GC:19 OK
21:36:16.817Z :Consumer :Topic: topic1, TC:19 OK, GC:20 OK
21:36:16.817Z :Consumer :Topic: topic1, TC:20 OK, GC:21 OK
21:36:16.818Z :Consumer :Topic: topic1, TC:21 OK, GC:23 JUMP FORWARD 1
21:36:16.818Z :Consumer :Topic: topic1, TC:22 OK, GC:24 OK
21:36:16.818Z :Consumer :Topic: topic1, TC:23 OK, GC:25 OK
21:36:16.818Z :Consumer :Topic: topic1, TC:24 OK, GC:26 OK
21:36:16.818Z :Consumer :Topic: topic1, TC:25 OK, GC:27 OK
21:36:16.818Z :Consumer :Topic: topic1, TC:26 OK, GC:28 OK
21:36:16.818Z :Consumer :Topic: topic1, TC:27 OK, GC:29 OK
21:36:16.818Z :Consumer :Topic: topic1, TC:28 OK, GC:30 OK
21:36:16.819Z :Consumer :Topic: topic1, TC:29 OK, GC:31 OK
21:36:16.819Z :Consumer :Topic: topic1, TC:30 OK, GC:32 OK
21:36:16.819Z :Consumer :Topic: topic1, TC:31 OK, GC:34 JUMP FORWARD 1
21:36:16.819Z :Consumer :Topic: topic1, TC:32 OK, GC:35 OK
21:36:16.819Z :Consumer :Topic: topic1, TC:33 OK, GC:36 OK
21:36:16.819Z :Consumer :Topic: topic1, TC:34 OK, GC:37 OK
21:36:16.819Z :Consumer :Topic: topic1, TC:35 OK, GC:38 OK
21:36:16.819Z :Consumer :Topic: topic1, TC:36 OK, GC:39 OK
21:36:16.820Z :Consumer :Topic: topic1, TC:37 OK, GC:40 OK
21:36:16.820Z :Consumer :Topic: topic1, TC:38 OK, GC:41 OK
21:36:16.820Z :Consumer :Topic: topic1, TC:39 OK, GC:42 OK
21:36:16.820Z :Consumer :Topic: topic1, TC:40 OK, GC:43 OK
21:36:16.820Z :Consumer :Topic: topic1, TC:41 OK, GC:45 JUMP FORWARD 1
21:36:16.820Z :Consumer :Topic: topic1, TC:42 OK, GC:46 OK
21:36:16.821Z :Consumer :Topic: topic1, TC:43 OK, GC:47 OK
21:36:16.821Z :Consumer :Topic: topic1, TC:44 OK, GC:48 OK
21:36:16.821Z :Consumer :Topic: topic1, TC:45 OK, GC:49 OK
21:36:16.821Z :Consumer :Topic: topic1, TC:46 OK, GC:50 OK
21:36:16.821Z :Consumer :Topic: topic1, TC:47 OK, GC:51 OK
21:36:16.821Z :Consumer :Topic: topic1, TC:48 OK, GC:52 OK
21:36:16.821Z :Consumer :Topic: topic1, TC:49 OK, GC:53 OK
21:36:16.821Z :Consumer :Topic: topic1, TC:50 OK, GC:54 OK
21:36:16.822Z :Consumer :Topic: topic1, TC:51 OK, GC:56 JUMP FORWARD 1
21:36:16.822Z :Consumer :Topic: topic1, TC:52 OK, GC:57 OK
21:36:16.822Z :Consumer :Topic: topic1, TC:53 OK, GC:58 OK
21:36:16.822Z :Consumer :Topic: topic1, TC:54 OK, GC:59 OK
21:36:16.822Z :Consumer :Topic: topic1, TC:55 OK, GC:60 OK
21:36:16.823Z :Consumer :Topic: topic1, TC:56 OK, GC:61 OK
21:36:16.823Z :Consumer :Topic: topic1, TC:57 OK, GC:62 OK
21:36:16.823Z :Consumer :Topic: topic1, TC:58 OK, GC:63 OK
21:36:16.823Z :Consumer :Topic: topic1, TC:59 OK, GC:64 OK
21:36:16.823Z :Consumer :Topic: topic1, TC:60 OK, GC:65 OK
21:36:16.823Z :Consumer :Topic: topic1, TC:61 OK, GC:67 JUMP FORWARD 1
21:36:16.824Z :Consumer :Topic: topic1, TC:62 OK, GC:68 OK
21:36:16.824Z :Consumer :Topic: topic1, TC:63 OK, GC:69 OK
21:36:16.824Z :Consumer :Topic: topic1, TC:64 OK, GC:70 OK
21:36:16.824Z :Consumer :Topic: topic1, TC:65 OK, GC:71 OK
21:36:16.824Z :Consumer :Topic: topic1, TC:66 OK, GC:72 OK
21:36:16.824Z :Consumer :Topic: topic1, TC:67 OK, GC:73 OK
21:36:16.824Z :Consumer :Topic: topic1, TC:68 OK, GC:74 OK
21:36:16.824Z :Consumer :Topic: topic1, TC:69 OK, GC:75 OK
21:36:16.825Z :Consumer :Topic: topic1, TC:70 OK, GC:76 OK
21:36:16.825Z :Consumer :Topic: topic1, TC:71 OK, GC:78 JUMP FORWARD 1
21:36:16.825Z :Consumer :Topic: topic1, TC:72 OK, GC:79 OK
21:36:16.825Z :Consumer :Topic: topic1, TC:73 OK, GC:80 OK
21:36:16.825Z :Consumer :Topic: topic1, TC:74 OK, GC:81 OK
21:36:16.825Z :Consumer :Topic: topic1, TC:75 OK, GC:82 OK
21:36:16.825Z :Consumer :Topic: topic1, TC:76 OK, GC:83 OK
21:36:16.825Z :Consumer :Topic: topic1, TC:77 OK, GC:84 OK
21:36:16.825Z :Consumer :Topic: topic1, TC:78 OK, GC:85 OK
21:36:16.826Z :Consumer :Topic: topic1, TC:79 OK, GC:86 OK
21:36:16.826Z :Consumer :Topic: topic1, TC:80 OK, GC:87 OK
21:36:16.826Z :Consumer :Topic: topic1, TC:81 OK, GC:89 JUMP FORWARD 1
21:36:16.826Z :Consumer :Topic: topic1, TC:82 OK, GC:90 OK
21:36:16.827Z :Consumer :Topic: topic1, TC:83 OK, GC:91 OK
21:36:16.827Z :Consumer :Topic: topic1, TC:84 OK, GC:92 OK
21:36:16.827Z :Consumer :Topic: topic1, TC:85 OK, GC:93 OK
21:36:16.827Z :Consumer :Topic: topic1, TC:86 OK, GC:94 OK
21:36:16.827Z :Consumer :Topic: topic1, TC:87 OK, GC:95 OK
21:36:16.827Z :Consumer :Topic: topic1, TC:88 OK, GC:96 OK
21:36:16.827Z :Consumer :Topic: topic1, TC:89 OK, GC:97 OK
21:36:16.828Z :Consumer :Topic: topic1, TC:90 OK, GC:98 OK
21:36:16.828Z :Consumer :Topic: topic1, TC:91 OK, GC:100 JUMP FORWARD 1
测试结果显示,Pulsar 首先处理 topic2 的消息,然后再处理 topic1。
同样地,在 10000 条消息的情况下,全局消息排序的前后跳跃不超过 2000。
结论
可以看到,如果你有一个可以对消息进行实时消费的快速 consumer,那么 Pulsar 可以在 2 个 topic 之间,按照正确的排序向你传递消息。别把房子压上,不能保证一定如此。
在这种情况下,我们将“严格实时”定义为内部缓存(默认可容纳 1000 条消息)不被充满,这意味着,即便应用程序可能稍有滞后,Pulsar 依然可以通过内部缓存的调节保持同步更新,直到应用程序赶上来。
但是,一旦我们失去了实时消费的状态,那么跨主题的消息也将失去正确的排序。
这就带给我们一个问题:如果我们可以实现跨主题的消息排序,但又无法保证这一点,我应该使用它吗?
如果你希望你的应用达到 100% 的一致性,那么答案是:否;如果只求做到尽可能好,那么不妨再进一步了解。这就与分布式系统所面临的问题是一样的,在正常操作的情况下选择一致性,在遇到诸如网络分区之类的问题时选择可用性——应用开发人员仍然需要对数据服务中的一致性问题进行处理,尽管在正常情况下,一致性是可以得到保证的。
支持排序的多主题订阅的替代方案
由于 topic 之间存在关联,那么排序可能是非常必要的。这里还有其他一些替代方案可供选择:
-
如果是一些具有相关性的事件,你希望对它们进行排序,那么可以把它们发送到同一个 topic。这样做可以得到 100% 的有序性保证。再由 consumer 过滤掉不感兴趣的消息。我不知道 Pulsar 是如何做到在使用模式注册表时在 1 个 topic 之内提供多种事件类型。Kafka 的相关内容参见:
https://www.confluent.io/blog/put-several-event-types-kafka-topic/ -
在下一个标题的段落中,我将提供一个“尽力”的跨主题消息排序解决方案。
-
以不体现相关事件排序的方式构建系统(比较难)
-
删除目标系统中的参照完整性
-
尝试使用单调性:
https://blog.acolyer.org/2019/03/06/keeping-calm-when-distributed-consistency-is-easy/ -
如有更好的模式建议,请Twitter我!
在单个 topic 中混合多个事件的缺点在于,这个决策将会影响到所有消费消息的应用程序。topic 是可被共用的基础设施,而混合事件则成为架构决策。这不会影响 RabbitMQ,因为每个应用程序都可以将单个队列绑定到多个交换器,并且在不影响其他应用的情况下进行自定义混合,而这正是RabbitMQ 的杀手锏。因此,当你发现很难在 Kafka 或 Pulsar 中为事件建立模型时,可以考虑 RabbitMQ。
基于发布时间戳的“尽力”策略
我们正式进入危险地带,基于发布时间戳进行排序。之所以写这一段,是为了使你摆脱这个念头,因为如果不遭遇任何严重的时钟偏移,确实可以实现不错的排序……(我知道)。
基本思路是,针对每个想要消费的 topic,建立 1 个 consumer,并将其放入独立的线程中。保持 consumer 内部缓存充满消息,主线程查看第一条消息,并且仅处理 producer时间戳最小的消息。在 Kafka 中,我们可以通过 broker 设置 LogAppendTime,虽然依然存在风险,但是风险程度较低。
图 2:“尽力”的时间戳策略
那么,有可能会出错吗?1 个 producer 的时钟可能落后于其他 producer(例如1小时或1年)。这将导致它的消息在排序中被优先一些处理,甚至,如果它是个快速 producer 的话,还可能完全阻止对其他 topic 的消费。这实际上与优先级队列的问题是一样的,在高负载的情况下,优先级较高的消息会导致优先级较低的消息被卡住。
但是,如果你控制了所有 producer(的时钟),并且不打算让某些 producer 成为超级快速的 producer,那么这仍不失为一种策略。当时钟完全同步时,测试结果显示,在实时消费和时间旅行的场景下,都可以实现完美的消息排序。再次重申,不要再在 Twitter 与我争辩时钟倾斜的问题,这里应该已经写得很清楚了!
篇后语
就是这些内容,也不知道是否对你有用,但至少我们了解了一些有关 Pulsar 的知识,以及多主题订阅可以带来的好处。你还可以到 CloudKarafka 博客查阅关于 Apache Kafka 的分析。
相关文章:
其他推荐:
实施:GitHub + MarkDown 文档系统的工作环境部署及工作流程说明 | 技术传播
这次他们说好要“讲真的” | 传播
在座都别吵了,你们还有我 | 技术传播
睿齐
技术传播从业者
品牌内容策划
自由摄影师
自由撰稿人
汪力迪
公众号:techcomm / htstory
微信号:bgrichi
邮箱:hash_0813@163.com
转载自原文链接, 如需删除请联系管理员。
原文链接:翻译:关于 Apache Pulsar 的多主题订阅 | 技术,转载请注明来源!