Enable indexing while uploading

[xzfc]

Related to qdrant/qdrant#6325, qdrant/qdrant#6355.

This PR removes code that disables indexing while uploading. Immediate indexing is now enabled unconditionally (i.e. both for dev and master), so we can compare how incremental indexing improves the situation.

---

This PR changes the logic that disables indexing while uploading, making it configurable.

Old logic:

• Before uploading: max_optimization_threads = 0.
  I.e. indexing disabled unconditionally.
• After uploading: max_optimization_threads = 100_000.
  I.e. start indexing using CPU quota unconditionally.

New logic:

• Before uploading: max_optimization_threads defaults to 0 but could be overwritten in JSON.
• After uploading: If max_optimization_threads is 0, set it to 100_000.
  I.e. start indexing if it was disabled, but do not touch if it was not disabled.

---

In this PR, max_optimization_threads is set to null (aka qdrants default, aka "auto", aka enable indexing using available CPU resources) for configurations that are used in continuous benchmark action:

• qdrant-all-on-disk-scalar-q/qdrant-all-on-disk-scalar-q-with-payload
• qdrant-continuous-benchmark/qdrant-continuous-benchmark-with-payload
• qdrant-bq-continuous-benchmark/qdrant-bq-continuous-benchmark-with-payload

For every other configuration, including qdrant-default, the behavior is not changed.

[Copilot]

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

[Copilot]

Copilot reviewed 2 out of 4 changed files in this pull request and generated no comments.

Files not reviewed (2)

• experiments/configurations/qdrant-on-disk.json: Language not supported
• experiments/configurations/qdrant-single-node.json: Language not supported

Comments suppressed due to low confidence (2)

engine/clients/qdrant/upload.py:63

• [nitpick] Consider defining a constant (e.g. INDEXING_DISABLED) for the '0' value to improve clarity and maintainability.

if collection.config.optimizer_config.max_optimization_threads == 0:

engine/clients/qdrant/configure.py:72

• [nitpick] Consider using an explicit assignment for 'optimizers_config' instead of 'setdefault' for improved readability of the configuration initialization.

optimizers_config = self.collection_params.setdefault("optimizers_config", {})