amend the mission statement and the roadmap to reflect landscape and Sage changes 20 years on

[dimpase]

The original Sage mission statement ("Creating a viable free open source alternative to Magma, Maple, Mathematica and Matlab") appeared close to 20 years ago, and since then there were big changes in Python and non-Python maths computing landscape, as well as in Sage:

1. Numerical computing in Python (and in Julia) really took off, and nowadays e.g. SciPy+Jupyter is a viable alternative to many uses of Matlab. We're not competing with SciPy; our probably only extras, compared to SciPy, on this front are interfaces to several optimisation packages.
2. Python-based visualisation/GUI via Jupyter(lab) has really become a standard widely available, easily installed, etc. We provide an inferior vendoring installation of it (but we don't have to).
3. Python packaging matured enough, allowing to rely on externally installed from PyPI packages much more than we do. PyPI is now the prevalent way of delivering Python packages. One can pretty much install all the Python packages we need from PyPI rather than from our vendored collection.
4. CI and other automation appeared and changed the ways software is tested.
5. Docker and other container technologies simplified delivery of complete pre-installed set-ups of complex maths software tools, such as tensorflow etc.
6. Anaconda/conda allows installing and running almost everything we package.
7. Sage has outgrown its manual inefficient system of package management, with over 400 packages placed in a flat directory structure, and manual handling of dependencies, versions, etc.
8. Sage is being transformed from an application (like Magma, Maple, Mathematica, Matlab) to a system of modern Python libraries, provided by distribution packages that can be installed and used separately, cf. Meta-ticket: Modularize sagelib into separate distributions (pip-installable packages) sagemath-... for Sage 10.x #29705

Nevertheless, one sees the (old) mission statement invoked now and then to defend the current uncontrolled growth of the number of packages, continued vendoring of most everything, etc.
Therefore the need to amend the mission statement is crucial, we don't want it to lead to complete unraveling of the project.

While I can't at the moment of writing this produce a short new version, here are few points which I see as crucial

a. Sage should be a good, economic citizen of Python universe; it should not duplicate efforts elsewhere.
Existing solutions should be reused and existing standards should be followed whenever possible. Sage currently has many complex 'customizations', that might have been improvements over the state-of-the-art solutions at the time they were introduced, but are now preventing an easy adaptation of the new standards (examples are custom package manager, custom test runner, custom doctest format, custom sphinx setup).

b. Sage (the distribution) should slim down, in view of a; e.g. there is no need to ship its own GUI or a set of compilers.

[williamstein]

As author of the original mission statement, I do agree that much has changed in the world in the last 20 years, as outlined here.

[tobiasdiez]

a. Sage should be a good, economic citizen of Python universe; it should not duplicate efforts elsewhere.

This is an important point. To make it more concrete, I would add that existing solutions should be reused and existing standards should be followed whenever possible. Sage currently has many complex 'customizations', that might have been improvements over the state-of-the-art solutions at the time they were introduced, but are now preventing an easy adaptation of the new standards (examples are custom package manager, custom test runner, custom doctest format, custom sphinx setup).

[mkoeppe]

I also think that the mission statement is no longer appropriate.

But the perhaps most important point is missing in the list above:

8. Sage is being transformed from an application (like Magma, Maple, Mathematica, Matlab) to a system of modern Python libraries, provided by distribution packages that can be installed and used separately. (That's Meta-ticket: Modularize sagelib into separate distributions (pip-installable packages) sagemath-... for Sage 10.x #29705)

[mkoeppe]

This also means that focusing the discussion on the Sage distribution in its role of providing the Sage "application" to end users is fundamentally misguided. (See the Apr 2023 sage-devel thread "What was/is/will be the purpose of maintaining the Sage distribution?"

Instead, we need to focus on what the Sage distribution provides for the development and distribution activities of Sage (as an upstream project).

• For building wheels for PyPI using cibuildwheels: Every Python package that builds platform wheels for PyPI contains ad-hoc installation scripts that builds the library dependencies from source. Examples are in spkg-configure.m4 for most external Python pkgs #36777 (comment) and the surrounding discussion. The Sage distribution provides the installation scripts in a cleaner, more systematic way.
• As a "reference distribution" for Sage development, where we have direct control over key dependencies.

[williamstein]

Dima and Matthias: do I have your permissions to delete all the off topic argumentative comments above? Please just answer yes or thumbs up. Thanks!

[Edit: I've deleted most of this discussion. Please have a civil and useful discussion. Thanks!]