Config overhaul: use an explicit section for metadata

[josecelano]

Parent issue: #401

After a discussion today, we decided to make the "namespace" in the configuration explicit by using the project name or package name. The main benefit is you can parse the file to know which service the configuration belongs to.

Current version

version = "2.0.0"

# ...

Proposal 1

[torrust_tracker]
version = "2.0.0"

# ...

Proposal 2

[project]
name = "torrust_tracker"
version = "2.0.0"

# ...

Hi @da2ce7, I came up with this solution after seeing this Python feature.

I think it's better to move the namespace value to an option value. The type in Rust would be the same for all Torrust projects.

Proposal 3

[metadata]
name = "torrust_tracker"
version = "2.0.0"

# ...

I also think that maybe version is too ambiguous if we use the "project" keyword because you don't know if it's the app version of the config file version because there is no native way to add metadata in TOMl files. In our case the config version does not match the app version.

[josecelano]

Proposal 4

Hi @da2ce7 I have another crazy idea. What if we use a kind of "attributes" that are inserted in comments like this?

# Metadata: name = "torrust_tracker"
# Metadata: author = "Jane Doe"
# Metadata: version = "2.0.0"
# Metadata: description = "This is an example TOML file with metadata in comments."
# Metadata: license = "MIT"
# Metadata: created_at = "2024-07-11T12:34:56Z"

# Actual configuration data
[settings]
theme = "dark"
timeout = 30

[database]
host = "localhost"
port = 5432
user = "admin"
password = "secret"

This would be like the "meta" tag in HTML or YAML directives. It could be as simple as the prefix Metadata and a key-value pair.

See also: toml-lang/toml#816

[josecelano]

I would implement proposal 3 bacuase:

• It's simpler.
• You can preparse the TOML file anyway taking only the [metedata] section. It was one of the reason I was thinking about proposal 4. You only need to read lines until you find the version. You don't need to deserialize a TOML format. The other reason was you can decouple metadata from config. Metedata can evolve independently from the configuration (I dont' think we need this now).
• It forces the admin to define the metadata.

[mario-nt]

@josecelano @da2ce7 I like proposal #3 too. Maybe we can call the field "project_version" instead of "version" to make it more clear? (I don't know if we can use underscores in a toml file.)

[josecelano]

@josecelano @da2ce7 I like proposal #3 too. Maybe we can call the field "project_version" instead of "version" to make it more clear? (I don't know if we can use underscores in a toml file.)

Hi @mario-nt, yes we can. Maybe it could even be crate instead of project.

Proposal 5

[metadata]
crate_name = "torrust_tracker"
version = "2.0.0"

# ...

On the other hand, I think some people could misunderstand version. Some people might think it's the crate version nor the configuration version.

Proposal 6

[metadata]
crate_name = "torrust_tracker"
config_version = "2.0.0"

# ...

[da2ce7]

@josecelano

I like the idea of making it more explicit: how about:

[torrust_tracker_config]
schema_version = "2.0.0"

[josecelano]

@josecelano

I like the idea of making it more explicit: how about:

[torrust_tracker_config]
schema_version = "2.0.0"

Hi @da2ce7, the more I think about this, the more I like proposal 4. The problem with using the config file itself to include the version is you have to parse the contents twice, and I don't see a clean way to do it.I think we need to somehow extract the version first to later use Figment with the right data types according to the version we extracted in the first round. I only see two options for the first round:

1. Just iterate over the file lines and capture only the section torrust_tracker_config and the following keys until you fine a new section. Then you can deserialize only those lines. In the second round it doesn't matter if you include that section again as long as the section remains the same for all config versions.
2. Use low-level serde deserialization in the first round to extract only the torrust_tracker_config section.

If we use a comment like:

# Metadata: version = "2.0.0"

in the first round we only need to iterate over the lines searching for that pattern. We could use the latest one if the metadata was not included or throw an error if it's missing. It could be a mandatory medatada field.

Alternatively, we could keep it simple line forcing the user to include a line at the top of the config file like this:

torrust_tracker_config_schema_version = "2.0.0"

That would make it really easy to get the version in the first round. I don't see the need to introduce a section if we don't need any other field.

[josecelano]

We discussed it, and we decided to use this format:

Proposal 7

[metadata]
crate = "torrust_tracker"
purpose = "configuration"
schema_version = "2.0.0"

The process to parse the congifuration file will have three stages:

1. We read the file lines, looking for the [metadata] section. If it's missing, we throw an error.
2. We parse with serde only the metadata section to get the scheme version.
3. We parse with Figment the whole file with the types mathicng the schema version.

We discarded this option:

[torrust_tracker_config]
schema_version = "2.0.0"

Because it would generate a different JSON schema for each crate's configuration file (tracker, index, ...).

cc @da2ce7

[josecelano]

I can't use the keyword crate, so finally It will be:

[metadata]
app = "torrust-tracker"
purpose = "configuration"
schema_version = "2.0.0"