Questions for CONNECTION in COPY and CREATE STAGE

[Xuanwo]

I have a few questions about this new feature:

• The existing document already says users can optionally specify an endpoint URL for s3 external located files. See https://databend.rs/doc/reference/sql/dml/dml-copy-into-table#externallocation-for-amazon-s3-like. So now, after being included in the CONNECTION part, is it still optional, or a must now?
• Same question for ACCESS_KEY_ID & SECRET_ACCESS_KEY. The existing document indicates they are optional as well. How about now?

Originally posted by @soyeric128 in #6573 (comment)

[Xuanwo]

The existing document already says users can optionally specify an endpoint URL for s3 external located files.

The doc is out-of-date. ENDPOINT_URL didn't implement at all.

We will place all connection-related options with the new change into CONNECTION.

And CONNECTION is required for now, because most services require at least one option.

Same question for ACCESS_KEY_ID & SECRET_ACCESS_KEY. The existing document indicates they are optional as well. How about now?

No, ACCESS_KEY_ID and SECRET_ACCESS_KEY are required now. Databend could return an error if those values are missing. Those values will check based on different services:

• s3: require: ACCESS_KEY_ID, SECRET_ACCESS_KEY, optional: ENDPOINT_URL, REGION, ENABLE_VIRTUAL_HOST_STYLE
• azblob: require ACCOUT_NAME, ACCOUNT_KEY, ENDPOINT_URL.

Those restrictions apply to both COPY and CREATE STAGE.