I noticed that retention is showing as infinite on all topics, even those that don’t have persistence enabled. I thought data would be discarded after a certain period of time, or when the topic “gets full”.
Also, what is the column “In Use”?
Thanks!
This column is for the topic retention. The topic retention means how long the messages will be available on Kafka itself. Persisting a topic (using the toggle) will save the streams into storage and make it available for querying using our data explorer. That is outside Kafka.
Kafka persistence can be configured on multiple axis. One is retention bytes, but could also be limited by duration. For example I could have ‘infinite bytes’ but only for a week. Or ‘10GB’ forever. In either of those scenarios, your messages won’t actually live forever.
If you’re using Quix provided Kafka, your options are limited by what quotas your account has, while if you use an external one - such as our Confluent connector -, then we no longer impose quotas on you.
Note: Currently the UI only exposes the time for which your messages are kept for, not the bytes.
‘In use’ column lets you know if any deployment is currently using said topic. If you hover over the number, it lets you know which ones are using it.
Example:
