fix(outputs.influxdb_v2): Handle serialization errors correctly

[srebhan]

Summary

The current code fails to handle serialization errors, e.g. due to NaN values or invalid field keys, as batching returns the serialization error straight away. Subsequently, the error triggers an early exit in the http client code because the used variable is zero.

This PR fixes two issues:

Do not return early in the http client if one of the batches does not contain any payload. Instead check if the size of the limited serializer is exceeded and if not continue to process the remaining batches. This prevents early exits and thus prevent the plugin to get stuck on serialization errors.

Reject metrics causing serialization errors to prevent infinite resend-loops and remove rejected metrics from the list of metrics acceptable later. Removing the metrics (or rather the indices) is required to not double-account the metric.

Checklist

• No AI generated code was used in this PR
• AI generated code used in this PR follows the InfluxData Policy on AI-Generated Code Contributions

Related issues

resolves #17205
resolves #17235

[telegraf-tiger]

Download PR build artifacts for linux_amd64.tar.gz, darwin_arm64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

☺️ This pull request doesn't significantly change the Telegraf binary size (less than 1%)

📦 Click here to get additional PR build artifacts

Artifact URLs

. DEB	. RPM	. TAR . GZ	. ZIP
amd64.deb	aarch64.rpm	darwin_amd64.tar.gz	windows_amd64.zip
arm64.deb	armel.rpm	darwin_arm64.tar.gz	windows_arm64.zip
armel.deb	armv6hl.rpm	freebsd_amd64.tar.gz	windows_i386.zip
armhf.deb	i386.rpm	freebsd_armv7.tar.gz
i386.deb	ppc64le.rpm	freebsd_i386.tar.gz
mips.deb	riscv64.rpm	linux_amd64.tar.gz
mipsel.deb	s390x.rpm	linux_arm64.tar.gz
ppc64el.deb	x86_64.rpm	linux_armel.tar.gz
riscv64.deb		linux_armhf.tar.gz
s390x.deb		linux_i386.tar.gz
		linux_mips.tar.gz
		linux_mipsel.tar.gz
		linux_ppc64le.tar.gz
		linux_riscv64.tar.gz
		linux_s390x.tar.gz

[lowjoel]

LGTM apart from some clarifications. I can't run this right now since I don't seem to be getting NaNs from my data source right now.

[senpro-ingwersenk]

Getting to it. Thanks for the ping; will come back with results in a bit!

[srebhan]

@lowjoel any chance you test the binary and report back in the issue?

[lowjoel]

@lowjoel any chance you test the binary and report back in the issue?

I can test the binary but I don't have any NaNs that can trip the bug now - the source was sporadically causing the metric rejection because it was sporadically writing NaNs. I couldn't figure out how but it hasn't in a few weeks.

[srebhan]

Ok, as @senpro-ingwersenk gave a go and my unit-test reproduces the NaN case I think we are good for review...

[skartikey]

@srebhan The implementation is solid!
This correctly fixes the critical data loss bug where serialization errors would cause valid metrics to be dropped.

[IngwiePhoenix]

Oops, should've replied here instead of in #17235
Either way; LGTM! Seems to have resolved the issue and surfaced a new one - but none that seems to be too problematic.

Thank you for the PR!

[srebhan]

@IngwiePhoenix no worries, I always check both the issue and the PR, replying to both is fine... ;-)