Add a new GMT setting to control the frequency of the data server refreshing

[seisman]

Description of the desired feature

It would be good to add one GMT setting (e.g., GMT_DATA_SERVER_FREQUENCY or other better names) so that users can control the frequency of the refreshing (24-hour by default).

The reason is that some users may want to use the remote files but don't care about if the data are the latest or not.

Are you willing to help implement and maintain this feature? No

[PaulWessel]

Just thought of something. Let's say you set that to 6 months. Meanwhile, we make lots of changes on the server, and some files are added and other go away. You decide to request a @file which your old gmt_data_server.txt file contains, but there is no longer such a file on the server, and you will get some libcurl message about that. I guess we would have to give a list of possible explanations for what might have gone wrong, including this scenario.

[seisman]

Yes, the error message could be improved.

What will happen if new files are added on the server, but my local gmt_data_server.txt is <= 6-months old and isn't updated?

[PaulWessel]

The basically do not exist for you. Cannot have it both ways. We depend on the dataset structure to know specific details about the datasets. Only data in that structure "exists" for the user [assuming we have not deleted some]

[seisman]

I think it's OK that users can't download the new data if they explicitly choose not to refresh the data server file frequently or never.

[PaulWessel]

Hi @seisman , Shall we introduce

GMT_DATA_SERVER_UPDATE = time

as a bug-fix to the slow-internet complaints and thus do this in 6.1? I am working on a rst file to explain how our remote data works, the policies, and what users can do to control the process, so it would be nice to not have to revise this in a few weeks/months.

[seisman]

I think @joa-quim generally agrees with the "new" workflow.

If we decide to adopt the new workflow, we should merge all fixes and PRs to the master branch. If we think a PR/fix is also necessary in 6.1, then we should backport the commit into 6.1 branch instead. The backporting operation can be done manually or automatically (I can do the backport manually before we set up a backport bot).

[seisman]

Back to your question, although GMT_DATA_SERVER_UPDATE is a new feature, I think it's critical for GMT 6.1. So yes, you can do it in master first. After merging your PR into master, then I will manually backport it to 6.1 branch.

[PaulWessel]

OK, I will do this in master for both that new keyword as well as the documentation of remote files.

[seisman]

Done in #3630.