Parents probably not reclaimed due to too much caching

[robertwb]

Here is a small example illustrating the issue.

The memory footprint of the following piece of code grows indefinitely.

sage: K = GF(1<<55,'t') 
sage: a = K.random_element() 
sage: while 1: 
....:     E = EllipticCurve(j=a); P = E.random_point(); 2*P; del E, P;

E and P get deleted, but when 2*P is computed, the action of integers on A, the abelian group of rational points of the ellitpic curve, gets cached in the corecion model.

A key-value pair is left in coercion_model._action_maps dict:

(ZZ,A,*) : IntegerMulAction

Moreover there is at least also references to A in the IntegerMulAction and one in ZZ._action_hash.

So weak refs should be used in all these places if it does not slow things too much.

To be merged with #11521. Apply:

• attachment: 715_all.patch

and then the patches from #11521.

Depends on #13145
Depends on #13741
Depends on #13746
Depends on #11521

Dependencies: #13145, #13741, #13746, to be merged with #11521

CC: @jpflori @zimmermann6 @vbraun @robertwb @nbruin @malb @orlitzky

Component: coercion

Keywords: weak cache coercion Cernay2012

Author: Simon King, Jean-Pierre Flori

Reviewer: Jean-Pierre Flori, Simon King, Nils Bruin

Merged: sage-5.5.beta0

Issue created by migration from https://trac.sagemath.org/ticket/715

[mwhansen]

comment:3

I think this is a bit too vague for a ticket. Robert, could you be more specific or close this?

[robertwb]

comment:4

The coercion model needs to use weakrefs so that parents aren't needlessly referenced when they're discarded. It is nontrivial to see where the weakrefs need to go, and how to do so without slowing the code down.

The ticket is still valid.

[loefflerd]

Description changed:

--- 
+++ 
@@ -1 +1 @@
-
+(Moving this to "coercion", which is clearly where it belongs.)

[jpflori]

Description changed:

--- 
+++ 
@@ -1 +1,20 @@
-(Moving this to "coercion", which is clearly where it belongs.)
+Here is a small example illustrating the issue.
+
+The memory footprint of the following piece of code grows indefinitely.
+
+```
+sage: K = GF(1<<55,'t') 
+sage: a = K.random_element() 
+sage: while 1: 
+....:     E = EllipticCurve(j=a); P = E.random_point(); 2*P; del E, P;
+
+```
+E and P get deleted, but when 2*P is computed, the action of integers on A, the abelian group of rational points of the ellitpic curve, gets cached in the corecion model.
+
+A key-value pair is left in coercion_model._action_maps dict:
+
+(ZZ,A,*) : IntegerMulAction
+
+Moreover there is at least also references to A in the IntegerMulAction and one in ZZ._action_hash.
+
+So weak refs should be used in all these places if it does not slow things too much.

[jpflori]

comment:7

With the piece of code in the desrciption, there is only one reference to these objects in that ZZ._hash_actions dictionary because to build it we test if A1 == A2 and not A1 is A2 as in coercion_model._action_maps, and because of the current implementation of ellitpic curves, see http://groups.google.com/group/sage-nt/browse_thread/thread/ec8d0ad14a819082 and #11474, and decause the above code use only one j-invariant, only ones gets finally stored.

However with random curves, I guess there would be all of them.

About the weakref, the idea should more be to build something like WeakKeyDictionnary if it does not slow down coercion too much...

[nbruin]

comment:8

The following example also exhibits a suspicious, steady growth in memory use. The only reason I can think of why that would happen is that references to the created finite field remain lying around somewhere, preventing deallocation:

sage: L=prime_range(10^8)
sage: for p in L: k=GF(p)

If you change it to the creation of a polynomial ring the memory use rises much faster:

sage: L=prime_range(10^8)
sage: for p in L: k=GF(p)['t']

Are "unique" parents simply never deallocated?

[jpflori]

comment:9

Be aware that polynomial rings are also cached because of uniqueness of parents, explaining somehow your second memory consumption; see #5970 for example.

For finite fields I did not check.