Pinterest's streaming ingestion has no job-completion signal to tell downstream jobs a partition is ready, so it now computes a non-regressing watermark in the Flink-to-Iceberg sink and stores it as an Iceberg table property. By default it takes the minimum event time across recent commits, and percentile policies can trade some completeness for fresher data.

— via InfoQ, Leela Kumili