State of inner products in Base

[juliohm]

If a user needs to define custom inner products for general Hilbert spaces, what is the current state in Base for this type of generalization? #334 is a related, but less general issue. My concern is with new types that aren't arrays.

I'd like to propose renaming dot to inner, or perhaps directing users to define inner(x,y) as a general inner product between objects x and y, including the array case:

inner(x::AbstractVector, y::AbstractVector) = dot(x, y)

In case the change is reasonable, could it be part of Julia v1.0?

[andreasnoack]

Could you explain a little bit more about your use case and why having it in Base instead of just defining it in your package is beneficial? A concrete example would be best. Do you expect several inner definitions across packages loaded simultaneously?

[juliohm]

I think having a formal interface for these mathematical spaces will help users exploit the type system better. For example, I would expect clustering methods to work on any metric space. If I could define my type with an inner product, I would benefit from Clustering.jl out of the box (after the package is fixed accordingly). Many other distance-based or projection-based algorithms could also be generalized.

As a concrete example, I came across this limitation today trying to define a geometry for compositional data: https://github.com/juliohm/CoDa.jl I'd rather specialize on a well-known inner function defined in Base than define my own interface that no one else will be aware of.

[andyferris]

Why not extend dot for your Hilbert space types? I’m pretty sure it’s designed with being the generic inner product in mind.

[juliohm]

The concept of dot product is more strict than the concept of inner product. Whereas the latter is defined for general spaces, a dot product is only defined when there is the notion of a coordinate system defined by a finite basis. The semantics of dot(x,y) is x'*y where x and y represent coordinates of the objects in a Cartesian world. Mathematical textbooks will rarely mention the term dot product as the authors are usually interested treating the material in more general (not necessarily finite nor Euclidean) spaces.

To distinguish further, in a Hilbert space with inner product <x,y> (or inner(x,y)) objects can be infinite and the semantics x'*y doesn't apply. For example, in functional data analysis the objects are functions f and g and the inner product is usually obtained by numerical integration: inner(f,g) = quadrature(f*g). Calling this operation a dot product is misleading.

Another example as I pointed out in my CoDa.jl package is compositional data. Composition objects lie in a simplex world for which the operation x'*y doesn't make any sense. However, there exists a isometric transformation (the log-ratio transformation) that one can use to map compositions into another geometry where one can then apply the dot product with coordinates. Working with coordinates is not necessary, but it is common practice in this field. The result can be back transformed to the original space where the objects exist.

I don't see benefit in maintaining the term dot in the language, but if one asks for backward compatibility, the generalization inner(x::AbstractVector, y::AbstractVector) = dot(x,y) works perfectly.

Can you elaborate on the objections for this change?

[StefanKarpinski]

Can you elaborate on the objections for this change?

We generally require a fair amount of justification for adding new public functions to Base, that's the objection. This could be provided by an InnerProducts package. Why does it need to be built into the language itself? This was the first question that @andreasnoack asked above – it got a somewhat vague answer of "I think having a formal interface for these mathematical spaces will help users exploit the type system better". There's no reason that an interface defined in a package is any less formal than one in Base. What does having Base.inner offer that InnerProducts.inner doesn't? This is a genuine question that could have a convincing answer, but I don't know what that answer might be, which is why the question is being asked.

[juliohm]

I don't see a good argument to define a basic mathematical concept like inner products elsewhere that is not in Base. A language whose main audience is scientific computing folks would benefit from correct terminology. Why the concept of norm is defined in Base.LinAlg and inner, which is on the same cohort, should be defined in a package? Besides this inconsistency, the language already has dot, which makes me wonder why it should have something so specific rather than a more general concept?

[StefanKarpinski]

So you want all possible mathematical concepts in the base language? Not having something defined in Base doesn't force people to use the wrong terminology. The norm function is exported from LinAlg because it is defined and used in LinAlg. Similar for dot. Are you proposing that dot should be renamed to inner?

[juliohm]

So you want all possible mathematical concepts in the base language?

I never said that.

Not having something defined in Base doesn't force people to use the wrong terminology.

I am sure that it doesn't. Promoting the wrong terminology is the issue. People coming from a less mathematical background will adopt the usage of dot because they see it in Base. Usage of the term "dot" product to represent the concept of inner product is incorrect. It is also harmful to the mathematical community, which struggles every now and then to fix these scars that wrong terminology has left. Students from my generation are consistently having to refer to old books to get the terminology right, this shouldn't be the case.

Are you proposing that dot should be renamed to inner

That would already be a major improvement in my opinion. See all the examples I gave above on functional and compositional data. People in these communities would never use the term dot in their work. "dot" is more like a computer science term than anything else.

[StefanKarpinski]

Renaming dot to inner is quite a different proposal than adding inner to Base in addition to dot. That's more of a "correct terminology" question, which you and other linalg folks will have to hash out, although I seem to recall we bikeshedded this once and concluded that dot was the correct name for what this function implements.

[andreasnoack]

There was a little discussion of this in #433 and JuliaLang/julia#22220

[juliohm]

Renaming dot to inner is quite a different proposal than adding inner to Base in addition to dot.

This is what I proposed in my first message in this thread:

I'd like to propose renaming dot to inner, or perhaps directing users to define inner(x,y) as a general inner product between objects x and y

I repeat dot product is the incorrect term for the operation I am discussing here. Inner, outer, scalar product... these are mathematical objects. "dot product" is a computational object: it gets two sequences of numbers and performs x1*y1 + x2*y2 + ... xn*yn, a useless operation in other mathematical spaces.

[StefanKarpinski]

I had focused on the second option you proposed, which seems to have been adding Base.inner with a fallback to call Base.dot. Either option is possible, but both require some justification: to add a new operation, one needs a reason why it can't just be in a package (what the initial part of this discussion was about); to rename, it needs to be decided that dot is the wrong name and inner is the correct one (what the conversation seems to have turned to).

[andyferris]

@juliohm It's probably worth (re)stating that there is an active effort currently trying to shrink Base and encourage the use of packages. In this case dot seems to be correct for all the types participating in linear algebra provided in standard Julia (i.e. Number and Array - so yes, there is a definite, known, finite basis in all cases - thus I don't think we've made a mistake in terminology, though there may be better choices). I'm not against this proposal - but wanted to point this to clarify why you might be experiencing some "latent" resistance to change.

Also worth keeping in mind that a fair number of Julia newcomers may be familiar with a dot product but not an inner product (say, they did a bit of physics at university, but aren't math majors) so there are also some reasons to keep dot (not to mention that we have an infix operator that it corresponds with - we could just map it to inner I suppose but that's slightly less obvious). We also don't have an outer function, or a variety of other possible operations.

Thus, there is a burden to make a reasonable case for how putting this in Base (or LinAlg) is strictly better than putting this in a user package. The primary reason seems to be to provide an interface that can be shared and extended by others - is that a reasonable summary? The argument about letting generic code from Clustering.jl work with your inner product seems pretty compelling. Also, in the context that we seem to be splitting LinAlg into a stdlib package - I was thinking that if I were to author a package called LinearAlgebra I'd probably be happy to include an inner function for others to extend.