Include GMP 5.1.3 as an optional package

[Simon King]

Let's package GMP 5.1.3.

Upstream tarball at:

• ​https://ftp.gnu.org/gnu/gmp/gmp-5.1.3.tar.bz2

[Simon King]

Google told me that some people suggested to "include cstdio to obtain std::FILE". So, I tried to add

#include <cstdio.h>

to gmp.h. However, after I attempted to build it, that line was gone!!

Can someone explain how I can avoid that my editing the file is reverted? Perhaps I need to do my change in the "patches" directory?

[Simon King]

Yeah, I made some progress!!

I found that I needed to edit src/gmp-h.in, and I needed to #include <cstdio>, but only #if defined (__cplusplus).

Then, building the package went beyond the point where it failed previously. However, it turns out that the gmp package requires yacc. I am insalling it now.

TODO (for the moment):

• Turn my changes to src/gmp-h.in into a patch, to be put into the patches directory.
• spkg-install should check for the presence of yacc.

[Simon King]

Hooray, the modified package builds on my openSuse laptop. However, spkg-check fails, ending with

Making check in cxx
make[3]: Entering directory `/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/tests/cxx'
make  t-assign t-binary t-cast t-constr t-headers t-istream t-locale t-misc t-ops t-ostream t-prec t-rand t-ternary t-unary
make[4]: Entering directory `/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/tests/cxx'
g++ -DHAVE_CONFIG_H -I. -I. -I../.. -I../.. -I../../tests    -O2 -m64 -mtune=k8 -c -o t-assign.o t-assign.cc
/bin/sh ../../libtool --mode=link g++  -O2 -m64 -mtune=k8   -o t-assign  t-assign.o -L../../.libs ../../tests/libtests.la ../../libgmpxx.la ../../libgmp.la 
mkdir .libs
g++ -O2 -m64 -mtune=k8 -o .libs/t-assign t-assign.o  -L/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs ../../tests/.libs/libtests.a ../../.libs/libgmpxx.so /home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs/libgmp.so ../../.libs/libgmp.so -Wl,--rpath -Wl,/home/simon/SAGE/sage-5.0.beta7/local/lib
creating t-assign
g++ -DHAVE_CONFIG_H -I. -I. -I../.. -I../.. -I../../tests    -O2 -m64 -mtune=k8 -c -o t-binary.o t-binary.cc
/bin/sh ../../libtool --mode=link g++  -O2 -m64 -mtune=k8   -o t-binary  t-binary.o -L../../.libs ../../tests/libtests.la ../../libgmpxx.la ../../libgmp.la 
g++ -O2 -m64 -mtune=k8 -o .libs/t-binary t-binary.o  -L/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs ../../tests/.libs/libtests.a ../../.libs/libgmpxx.so /home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs/libgmp.so ../../.libs/libgmp.so -Wl,--rpath -Wl,/home/simon/SAGE/sage-5.0.beta7/local/lib
creating t-binary
g++ -DHAVE_CONFIG_H -I. -I. -I../.. -I../.. -I../../tests    -O2 -m64 -mtune=k8 -c -o t-cast.o t-cast.cc
/bin/sh ../../libtool --mode=link g++  -O2 -m64 -mtune=k8   -o t-cast  t-cast.o -L../../.libs ../../tests/libtests.la ../../libgmpxx.la ../../libgmp.la 
g++ -O2 -m64 -mtune=k8 -o .libs/t-cast t-cast.o  -L/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs ../../tests/.libs/libtests.a ../../.libs/libgmpxx.so /home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs/libgmp.so ../../.libs/libgmp.so -Wl,--rpath -Wl,/home/simon/SAGE/sage-5.0.beta7/local/lib
creating t-cast
g++ -DHAVE_CONFIG_H -I. -I. -I../.. -I../.. -I../../tests    -O2 -m64 -mtune=k8 -c -o t-constr.o t-constr.cc
/bin/sh ../../libtool --mode=link g++  -O2 -m64 -mtune=k8   -o t-constr  t-constr.o -L../../.libs ../../tests/libtests.la ../../libgmpxx.la ../../libgmp.la 
g++ -O2 -m64 -mtune=k8 -o .libs/t-constr t-constr.o  -L/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs ../../tests/.libs/libtests.a ../../.libs/libgmpxx.so /home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs/libgmp.so ../../.libs/libgmp.so -Wl,--rpath -Wl,/home/simon/SAGE/sage-5.0.beta7/local/lib
creating t-constr
g++ -DHAVE_CONFIG_H -I. -I. -I../.. -I../.. -I../../tests    -O2 -m64 -mtune=k8 -c -o t-headers.o t-headers.cc
/bin/sh ../../libtool --mode=link g++  -O2 -m64 -mtune=k8   -o t-headers  t-headers.o -L../../.libs ../../tests/libtests.la ../../libgmpxx.la ../../libgmp.la 
g++ -O2 -m64 -mtune=k8 -o .libs/t-headers t-headers.o  -L/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs ../../tests/.libs/libtests.a ../../.libs/libgmpxx.so /home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs/libgmp.so ../../.libs/libgmp.so -Wl,--rpath -Wl,/home/simon/SAGE/sage-5.0.beta7/local/lib
creating t-headers
g++ -DHAVE_CONFIG_H -I. -I. -I../.. -I../.. -I../../tests    -O2 -m64 -mtune=k8 -c -o t-istream.o t-istream.cc
/bin/sh ../../libtool --mode=link g++  -O2 -m64 -mtune=k8   -o t-istream  t-istream.o -L../../.libs ../../tests/libtests.la ../../libgmpxx.la ../../libgmp.la 
g++ -O2 -m64 -mtune=k8 -o .libs/t-istream t-istream.o  -L/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs ../../tests/.libs/libtests.a ../../.libs/libgmpxx.so /home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/.libs/libgmp.so ../../.libs/libgmp.so -Wl,--rpath -Wl,/home/simon/SAGE/sage-5.0.beta7/local/lib
creating t-istream
g++ -DHAVE_CONFIG_H -I. -I. -I../.. -I../.. -I../../tests    -O2 -m64 -mtune=k8 -c -o t-locale.o t-locale.cc
t-locale.cc: In function ‘void check_input()’:
t-locale.cc:108:26: error: ‘abort’ was not declared in this scope
t-locale.cc:123:26: error: ‘abort’ was not declared in this scope
t-locale.cc: In function ‘void check_output()’:
t-locale.cc:158:18: error: ‘abort’ was not declared in this scope
make[4]: *** [t-locale.o] Fehler 1
make[4]: Leaving directory `/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/tests/cxx'
make[3]: *** [check-am] Fehler 2
make[3]: Leaving directory `/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/tests/cxx'
make[2]: *** [check-recursive] Fehler 1
make[2]: Leaving directory `/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src/tests'
make[1]: *** [check-recursive] Fehler 1
make[1]: Leaving directory `/home/simon/SAGE/sage-5.0.beta7/spkg/optional/gmp-4.2.1.p11.copy/src'
make: *** [check] Fehler 2

[Simon King]

While I was at it, I tried to upgrade to gmp 5.0.4. Unfortunately, it complains that it can not find bdivmod.c in the path -- the old spkg contains that file in src/mpn/generic, but the original gmp 5.0.4 sources do no contain bdivmod.c in src/mpn/generic. Too bad.

[Simon King]

The advantage of gmp 5.0.4 would be that the sources already include cstdio. So, no need for patching it. The problem is that 5.0.4 has removed and renamed a couple of things. For example, all files starting with the name mullow_ are now starting with mullo_. The patches of the spkg need to be changed accordingly.

The gmp 5.0.4 package seems to build. Keeping my fingers crossed for the test...

[Simon King]

Replying to SimonKing:

The gmp 5.0.4 package seems to build.

No, it doesn't. I get an infinite loop during configuration, it seems. /bin/sh ./config.status --recheck is repeated over and over again.

[Simon King]

See modified ticket description: It was easy to make gmp 5.0.4 build and pass all tests. Simply one has to drop all patches of the old gmp spkg.

[Simon King]

A related ticket: #12661 (optional CLooG spkg).

CLooG needs gmp to build.

[Simon King]

Replying to SimonKing:

CLooG needs gmp to build.

​Leif claims that CLooG could also be built with MPIR rather than GMP.

Anyway. Since the current spkg is broken (at least on openSuse), I still think we should upgrade. But people need to test on different platforms.

[Leif Leonhardy]

FWIW, I did build Sage against GMP (5.0.1 or 5.0.2) a couple of times in 2010 (when we made the "first big PARI upgrade") and never ran into any problems, except for the Lcalc spkg, which was the only one explicitly using -lmpir instead of -lgmp, an issue which AFAIK meanwhile is fixed.

Although I looked at the (btw. completely outdated) optional GMP spkg (on other occasions as well), I didn't use any of it, in particular didn't apply any additional patches to the more recent GMP versions.

---

We'll probably only run into trouble with using GMP instead of MPIR the day we upgrade FLINT, as newer (2.x, but IIRC only the most recent) versions of FLINT use some MPIR internals GMP doesn't provide, or at least isn't compatible with.

But for the moment it should be ok to use GMP instead, although (for CLooG) this shouldn't be necessary.

(In contrast, as Dima recently mentioned on sage-devel and mpir-devel, GAP uses some GMP internals MPIR currently doesn't provide. The [older?] version of GAP we use apparently works with MPIR... :-) )

[Simon King]

Replying to leif:

Although I looked at the (btw. completely outdated) optional GMP spkg ...

Yes, that's what this ticket is about.

in particular didn't apply any additional patches to the more recent GMP versions.

I'm not using any patches either. The question is whether other platforms (Solaris? OS X?) need patches.

We'll probably only run into trouble with using GMP instead of MPIR the day we upgrade FLINT,

When I first tried to build the optional gcc from #12369 with graphite (hence, together with the optional cloog-ppl package from #12666), I thought that cloog-ppl needs GMP (not MPIR). This is why I opened this ticket in the first place.

I found a massive slow-down when I built sage with the optimizations provided by graphite. Big surprise. But some people suggested that the culprit might in fact be GMP. So, I am started over. I can not tell yet whether Sage is faster with MPIR than with GMP. However, GMP is definitely not required by cloog-ppl, which lessens the priority of this ticket considerably, from my perspective...

[Benjamin Jones]

Building the new gmp spkg here on Mac OS X (with gcc 4.6.3) with SAGE_CHECK=yes succeeds and passess all tests!

[Leif Leonhardy]

GMP 5.0.5 is out since a while...

[Simon King]

Replying to leif:

GMP 5.0.5 is out since a while...

This ticket is open since a while...

[Leif Leonhardy]

Replying to SimonKing:

Replying to leif:

GMP 5.0.5 is out since a while...

This ticket is open since a while...

I was wanting to review it since a while...

[François Bissey]

gmp 5.1.x is out and transition to git is needed if we want to keep gmp in sage.

[Jean-Pierre Flori]

Yup, also note that FLINT >= 2.4 can now be built on top of GMP. IIRC it's the only spkg which explicitely depended on MPIR.

[Ralf Stephan]

Also, mpir was not updated since Nov 2012. However gmp isn't even in optional now, so a new ticket would have to be opened to include it.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​2c34755

Initial commit for GMP 5.1.3.

[Jean-Pierre Flori]

Problems with farey_symbol.pyx which uses additional c++ source files (farey.cpp and sl2z.cpp) but cython wants to compile them with CC and that fails when GMP is used underneath. Not sure why it succeeded on top of MPIR.

[Jean-Pierre Flori]

Anyway, it also fails with CXX:

g++ -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/jpflori/sage.git/local/include -I/home/jpflori/sage.git/local/include/csage -I/home/jpflori/sage.git/src -I/home/jpflori/sage.git/src/sage/ext -I/home/jpflori/sage.git/local/include -I/home/jpflori/sage.git/local/include/csage -I/home/jpflori/sage.git/src -I/home/jpflori/sage.git/src/sage/ext -I/home/jpflori/sage.git/local/include/python2.7 -c sage/modular/arithgroup/farey.cpp -o build/temp.linux-ppc64-2.7/sage/modular/arithgroup/farey.o -fno-strict-aliasing -w
sage/modular/arithgroup/farey.cpp: In function ‘mpz_class floor(mpq_class)’:
sage/modular/arithgroup/farey.cpp:147:44: error: conversion from ‘mpq_class {aka __gmp_expr<__mpq_struct [1], __mpq_struct [1]>}’ to non-scalar type ‘mpz_class {aka __gmp_expr<__mpz_struct [1], __mpz_struct [1]>}’ requested

[Jean-Pierre Flori]

Similar probledm and solution here:

• ​https://bugs.kde.org/show_bug.cgi?id=312481

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​a14aaf2

Explicit conversion in farey.cpp as recent GMP do not do it automatically.

[François Bissey]

Replying to jpflori:

Anyway, it also fails with CXX:

g++ -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/jpflori/sage.git/local/include -I/home/jpflori/sage.git/local/include/csage -I/home/jpflori/sage.git/src -I/home/jpflori/sage.git/src/sage/ext -I/home/jpflori/sage.git/local/include -I/home/jpflori/sage.git/local/include/csage -I/home/jpflori/sage.git/src -I/home/jpflori/sage.git/src/sage/ext -I/home/jpflori/sage.git/local/include/python2.7 -c sage/modular/arithgroup/farey.cpp -o build/temp.linux-ppc64-2.7/sage/modular/arithgroup/farey.o -fno-strict-aliasing -w
sage/modular/arithgroup/farey.cpp: In function ‘mpz_class floor(mpq_class)’:
sage/modular/arithgroup/farey.cpp:147:44: error: conversion from ‘mpq_class {aka __gmp_expr<__mpq_struct [1], __mpq_struct [1]>}’ to non-scalar type ‘mpz_class {aka __gmp_expr<__mpz_struct [1], __mpz_struct [1]>}’ requested

Sorry we could have warned you about that since sage-on-gentoo is stuck on gmp we knew about this and how to fix it.

[Jean-Pierre Flori]

Replying to fbissey:

Replying to jpflori:

Anyway, it also fails with CXX:

g++ -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/jpflori/sage.git/local/include -I/home/jpflori/sage.git/local/include/csage -I/home/jpflori/sage.git/src -I/home/jpflori/sage.git/src/sage/ext -I/home/jpflori/sage.git/local/include -I/home/jpflori/sage.git/local/include/csage -I/home/jpflori/sage.git/src -I/home/jpflori/sage.git/src/sage/ext -I/home/jpflori/sage.git/local/include/python2.7 -c sage/modular/arithgroup/farey.cpp -o build/temp.linux-ppc64-2.7/sage/modular/arithgroup/farey.o -fno-strict-aliasing -w
sage/modular/arithgroup/farey.cpp: In function ‘mpz_class floor(mpq_class)’:
sage/modular/arithgroup/farey.cpp:147:44: error: conversion from ‘mpq_class {aka __gmp_expr<__mpq_struct [1], __mpq_struct [1]>}’ to non-scalar type ‘mpz_class {aka __gmp_expr<__mpz_struct [1], __mpz_struct [1]>}’ requested

Sorry we could have warned you about that since sage-on-gentoo is stuck on gmp we knew about this and how to fix it.

No problem, I found the solution more than easily on the web.

Any other suggestion from sage-on-gentoo? E.g., do you have devised any simple way to switch from gmp to mpir and back again easily? Or are you really stuck with gmp and cannot use mpir? (that would be strange.)

[François Bissey]

We are really stuck with gmp unless we make great effort. It is complicated because most of the package of the tree will depend on gmp and few will use mpir directly, at some point I tried to port a lot of ebuild to mpir directly but it was a bit unsustainable. The easiest way is to use the gmp compatibility layer from mpir. Then we have to convince the toolchain people that we should have the possibility of switching between gmp and mpir to build gcc. Then we need to replace all dependencies to gmp in the tree by some virtual package which can be either gmp or mpir passing as gmp. That's a ton of work, require convincing people and so on.

if all package in sage could be configured with either real gmp or real mpir out of the box we may be able to do some easy switching between the two but it is not the case. lmona.de could go its own way and use mpir exclusively if they wanted too (by calling it gmp mainly).

[Jean-Pierre Flori]

FTR here are the only failing doctests:

sage -t --long src/sage/rings/integer.pyx
**********************************************************************
File "src/sage/rings/integer.pyx", line 1955, in sage.rings.integer.Integer.__pow__
Failed example:
    2^(2^63-2)
Expected:
    Traceback (most recent call last):
    ...
    MemoryError: failed to allocate 1152921504606847008 bytes
Got:
    gmp: overflow in mpz type   
    Traceback (most recent call last):
      File "/home/jpflori/sage.git/local/lib/python2.7/site-packages/sage/doctest/fo
rker.py", line 480, in _run
        self.execute(example, compiled, test.globs)
      File "/home/jpflori/sage.git/local/lib/python2.7/site-packages/sage/doctest/fo
rker.py", line 839, in execute  
        exec compiled in globs  
      File "<doctest sage.rings.integer.Integer.__pow__[27]>", line 1, in <module>
        Integer(2)**(Integer(2)**Integer(63)-Integer(2))
      File "integer.pyx", line 1994, in sage.rings.integer.Integer.__pow__ (sage/rin
gs/integer.c:14075)
      File "c_lib.pyx", line 85, in sage.ext.c_lib.sig_raise_exception (sage/ext/c_l
ib.c:1009)
    RuntimeError: Aborted
**********************************************************************
File "src/sage/rings/integer.pyx", line 5401, in sage.rings.integer.Integer._xgcd
Failed example:
    2._xgcd(-2)
Expected:
    (2, 1, 0)
Got:
    (2, 0, -1)
**********************************************************************

and

sage -t --long src/sage/tests/book_stein_modform.py
**********************************************************************
File "src/sage/tests/book_stein_modform.py", line 94, in sage.tests.book_stein_modform
Failed example:
    [x.lift_to_sl2z(2) for x in M.manin_generators()]
Expected:
    [[1, 0, 0, 1], [0, -1, 1, 0], [0, -1, 1, 1]]
Got:
    [[1, 0, 0, 1], [0, -1, 1, 0], [1, 0, 1, 1]]
**********************************************************************

[François Bissey]

I got the first one in sage/rings/integer.pyx and knew it was from gmp vs mpir. I also have sage/tests/book_stein_modform.py but I didn't know it was originating from gmp vs mpir. I don't have the second failure in integer.pyx it may be because I am currently only at gmp 5.1.2.

[Jean-Pierre Flori]

The ones in integer.pyx are really harmless anyway.

Not sure about the one in the Manin generators computation, so I could be easily convinced it is innocuous as well.

[François Bissey]

Actually I got the second one in integer.pyx. It wasn't present when I last made a record of doctests failures in sage-on-gentoo for sage 6.0, but it is in 6.2.beta2.

[François Bissey]

Any reason why this is still stuck at "need work"? I can give this a positive review, fixing the doctests to work with gmp is probably more work than it is worth. Especially the top one in integer.pyx.

Opinions?

[Jean-Pierre Flori]

I agree that I can live with the failing doctests.

As far as setting this to "needs_review" is concerned, I did not change the status because I wanted to include some patches to let SAge actually use GMP rather than MPIR and even potentially being able to switch from one to the other.

[François Bissey]

The only thing I can see right now that prevent sage to build with gmp out of the box is the inclusion of mpir.h instead of gmp.h in memory.c in clib.

[Jean-Pierre Flori]

Flint install script should be modified as well. Ok maybe it works without modification but passing with-gmp is better than with-mpir now that it is supported.

[Jean-Pierre Flori]

And you should also prevent Sage from building mpir I guess.

[François Bissey]

Ambitious I see. Of course I have total control in sage-on-gentoo but I would appreciate just having to switch --with-gmp or something.

[Jean-Pierre Flori]

Sure.

I would be ok to postpone the proper way to use gmp instead of mpir for sage the distrib in a follow up ticket. Let me dig up the few simple fixes needed for minimal support and push them here first.

[Jean-Pierre Flori]

Ok so everything is in fact already here.

[Jean-Pierre Flori]

Follow up at #15957.

[François Bissey]

I don't know that the renaming of sage_mpir_* to sage_gmp_* is necessary. In fact I am not even sure it needs to be named gmp or mpir at all. Aside from that it is all ok and conform to what we do in sage-on-gentoo. It also would cross a patch and a sed from my ebuilds at least.