Commit b6d2503
authored
KAFKA-20410: Add DLQ configuration parameters for Share Groups (KIP-1191) (#21979)
## Summary
This PR adds the configuration foundation for Share Groups Dead-Letter
Queue (DLQ) functionality as specified in [KIP-1191: Dead-letter queues
for share
groups](https://cwiki.apache.org/confluence/display/KAFKA/KIP-1191%3A+Dead-letter+queues+for+share+groups).
## Changes
### New Configurations
**Cluster-level configs (GroupCoordinatorConfig.java):**
1. `errors.deadletterqueue.auto.create.topics.enable` (default: `false`)
- Enables automatic creation of DLQ topics
2. `errors.deadletterqueue.topic.name.prefix` (default: `"dlq."`)
- Required prefix for DLQ topic names when auto-create is enabled
**Group-level configs (GroupConfig.java):**
1. `errors.deadletterqueue.topic.name` (default: `""`)
- Specifies the DLQ topic name for a share group
- Empty string means DLQ disabled for that group
2. `errors.deadletterqueue.copy.record.enable` (default: `false`)
- When `true`: Copy full original record to DLQ
- When `false`: Copy only context metadata (offset, delivery count,
reason)
**Topic-level config (TopicConfig.java):**
1. `errors.deadletterqueue.group.enable` (default: not set)
- Marks a topic as eligible for use as a DLQ
### Validation Logic
- DLQ topic names **must not** start with `__` (reserved for internal
topics)
Reviewers: David Jacot <david.jacot@gmail.com>, Andrew Schofield
<aschofield@confluent.io>1 parent 75052bb commit b6d2503
7 files changed
Lines changed: 196 additions & 6 deletions
File tree
- clients/src/main/java/org/apache/kafka/common/config
- core/src/test/scala/unit/kafka/server
- group-coordinator/src
- main/java/org/apache/kafka/coordinator/group
- test/java/org/apache/kafka/coordinator/group
Lines changed: 6 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
235 | 235 | | |
236 | 236 | | |
237 | 237 | | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
238 | 244 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
77 | 77 | | |
78 | 78 | | |
79 | 79 | | |
80 | | - | |
| 80 | + | |
81 | 81 | | |
82 | 82 | | |
83 | 83 | | |
| |||
379 | 379 | | |
380 | 380 | | |
381 | 381 | | |
| 382 | + | |
| 383 | + | |
382 | 384 | | |
383 | 385 | | |
384 | 386 | | |
| |||
Lines changed: 5 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
33 | | - | |
| 33 | + | |
34 | 34 | | |
35 | | - | |
36 | 35 | | |
37 | 36 | | |
38 | 37 | | |
| |||
1071 | 1070 | | |
1072 | 1071 | | |
1073 | 1072 | | |
| 1073 | + | |
| 1074 | + | |
| 1075 | + | |
| 1076 | + | |
1074 | 1077 | | |
1075 | 1078 | | |
1076 | 1079 | | |
| |||
Lines changed: 54 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
105 | 105 | | |
106 | 106 | | |
107 | 107 | | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
108 | 116 | | |
109 | 117 | | |
110 | 118 | | |
| |||
147 | 155 | | |
148 | 156 | | |
149 | 157 | | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
150 | 162 | | |
151 | 163 | | |
152 | 164 | | |
| |||
269 | 281 | | |
270 | 282 | | |
271 | 283 | | |
272 | | - | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
273 | 297 | | |
274 | 298 | | |
275 | 299 | | |
| |||
301 | 325 | | |
302 | 326 | | |
303 | 327 | | |
304 | | - | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
305 | 333 | | |
306 | 334 | | |
307 | 335 | | |
| |||
337 | 365 | | |
338 | 366 | | |
339 | 367 | | |
| 368 | + | |
| 369 | + | |
340 | 370 | | |
341 | 371 | | |
342 | 372 | | |
| |||
535 | 565 | | |
536 | 566 | | |
537 | 567 | | |
| 568 | + | |
| 569 | + | |
| 570 | + | |
| 571 | + | |
| 572 | + | |
| 573 | + | |
| 574 | + | |
| 575 | + | |
538 | 576 | | |
539 | 577 | | |
540 | 578 | | |
| |||
1060 | 1098 | | |
1061 | 1099 | | |
1062 | 1100 | | |
| 1101 | + | |
| 1102 | + | |
| 1103 | + | |
| 1104 | + | |
| 1105 | + | |
| 1106 | + | |
| 1107 | + | |
| 1108 | + | |
| 1109 | + | |
| 1110 | + | |
| 1111 | + | |
| 1112 | + | |
| 1113 | + | |
| 1114 | + | |
1063 | 1115 | | |
1064 | 1116 | | |
1065 | 1117 | | |
| |||
Lines changed: 35 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
314 | 314 | | |
315 | 315 | | |
316 | 316 | | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
317 | 328 | | |
318 | 329 | | |
319 | 330 | | |
| |||
451 | 462 | | |
452 | 463 | | |
453 | 464 | | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
454 | 469 | | |
455 | 470 | | |
456 | 471 | | |
| |||
515 | 530 | | |
516 | 531 | | |
517 | 532 | | |
| 533 | + | |
| 534 | + | |
| 535 | + | |
518 | 536 | | |
519 | 537 | | |
520 | 538 | | |
| |||
579 | 597 | | |
580 | 598 | | |
581 | 599 | | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
582 | 603 | | |
583 | 604 | | |
584 | 605 | | |
| |||
1188 | 1209 | | |
1189 | 1210 | | |
1190 | 1211 | | |
| 1212 | + | |
| 1213 | + | |
| 1214 | + | |
| 1215 | + | |
| 1216 | + | |
| 1217 | + | |
| 1218 | + | |
| 1219 | + | |
| 1220 | + | |
| 1221 | + | |
| 1222 | + | |
| 1223 | + | |
| 1224 | + | |
| 1225 | + | |
1191 | 1226 | | |
1192 | 1227 | | |
1193 | 1228 | | |
| |||
0 commit comments