fix(similarity): Use count instead of length in backfill query

[jangjodi]

Use .count() instead of len in postgres query

[wedamija]

You're now making two queries here - one to get the count, and one to get the end group id for the batch, in addition to fetching the batch on 183.

Is there any reason we can't just perform this logic after 183?

Any reason we can't move this logic after we filter to groups_to_backfill_batch?

[wedamija]

_make_postgres_call_with_retry doesn't work properly afaict. This just constructs the queryset

sentry/src/sentry/tasks/embeddings_grouping/utils.py

Lines 124 to 133 in 2376850

	groups_to_backfill_batch_raw = (
	Group.objects.filter(
	group_id_filter,
	project_id=project_id,
	type=ErrorGroupType.type_id,
	times_seen__gt=1,
	)
	.values_list("id", "data", "status", "last_seen")
	.order_by("-id")[:batch_size]
	)

. But since we don't convert it to a list, or attempt to iterate over it, then it never runs the query.

That means you'll never enter your retry loop here:

sentry/src/sentry/tasks/embeddings_grouping/utils.py

Lines 143 to 149 in 2376850

	except OperationalError:
	batch_size = batch_size // 2
	try:
	logger.info(
	"tasks.backfill_seer_grouping_records.postgres_query_retry",
	extra={"project_id": project_id, "batch_size": batch_size},
	)

You should coerce to a list in the _make_postgres_call. That also saves you the trouble of using .count here since you won't have a queryset.

[jangjodi]

Thanks for the review @wedamija! I've opened up a follow up PR to address these

[sentry]

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

• ‼️ OperationalError: QueryCanceled('canceling statement due to user request\n') sentry.tasks.backfill_seer_grouping_records View Issue

_{Did you find this useful? React with a 👍 or 👎}