Upgrade to NumPy 1.15.2

[Samuel Lelièvre]

NumPy 1.15.2 was released.

• ​NumPy releases

Tarball: ​https://files.pythonhosted.org/packages/45/ba/2a781ebbb0cd7962cc1d12a6b65bd4eff57ffda449fdbbae4726dc05fbc3/numpy-1.15.2.zip

Our last upgrade was to NumPy 1.13.3 in #24063.

A bug introduced in NumPy 1.15 prevents building NumPy on machines with many CPUs.

The issue is reported and solved upstream:

• ​NumPy issue 12087
• ​NumPy pull request 12088

[Samuel Lelièvre]

For NumPy 1.13.3 we have the tarball in .zip, should we also use the .zip this time or the .tar.gz?

[François Bissey]

We now support zip files I believe, so this is fine.

[Jeroen Demeyer]

If we support both kinds of archive, we should probably use the smallest one.

[Timo Kaufmann]

I don't know how or if I can add changes here, but I have fixed the doctests for the numpy+scipy upgrade ​here.

Its mostly formatting. Interesting bits:

• ​here I avoid a numpy deprecation warning (empty array should not be used as false) by always making sure points is a regular array and then comparing its len.
• the iprint kwarg in fmin_cobyla is ​deprecated and repaced by disp

Edit: I was a bit too fast and didn't test it properly. I'll post an updated version once thats tested.

[Timo Kaufmann]

I have ​updated the patch. The doctests should pass now.

[Samuel Lelièvre]

Changing target from Numpy 1.14.3 to 1.14.4.

[Timo Kaufmann]

I updated the package and applied my doctest fixes. make ptestlong passes.

---

New commits:

​3322ff1

Upgrade numpy to 1.14.4

[Samuel Lelièvre]

Numpy 1.15.0 just came out.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​ecf870b	Merge remote-tracking branch 'origin/develop' into u/gh-timokau/numpy-1.14.4
​aede54d	Upgrade numpy to 1.15.0

[Timo Kaufmann]

I've done most of the upgrade, but two failures in src/sage/plot/histogram.py remain. I think both are rooted in the following issue:

> import numpy
> numpy.histogram([], range=(1.0,1.0))

TypeError                                 Traceback (most recent call last)
<ipython-input-2-05b803a2f80b> in <module>()
----> 1 numpy.histogram([], range=(RealNumber('1.0'),RealNumber('1.0')))

/home/timo/repos/sage/local/lib/python2.7/site-packages/numpy/lib/histograms.pyc in histogram(a, bins, range, normed, weights, density)
    674     a, weights = _ravel_and_check_weights(a, weights)
    675 
--> 676     bin_edges, uniform_bins = _get_bin_edges(a, bins, range, weights)
    677 
    678     # Histogram is an integer or a float array depending on the weights.

/home/timo/repos/sage/local/lib/python2.7/site-packages/numpy/lib/histograms.pyc in _get_bin_edges(a, bins, range, weights)
    342         # shapes. To avoid this causing problems, we pick a type now and stick
    343         # with it throughout.
--> 344         bin_type = np.result_type(first_edge, last_edge, a)
    345         if np.issubdtype(bin_type, np.integer):
    346             bin_type = np.result_type(bin_type, float)

TypeError: data type not understood

This does not happen outside of sage. It probably has something to do with sages custom number types.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​a5707a1	Merge in latest develop branch
​b584304	Fix incorrect output in old docstring
​6abfb96	fix spacing
​27a7f75	A historgram's range must be a Python float

[Julian Rüth]

I haven't run full doctests but this fixes the issues in histogram.py.

[Jeroen Demeyer]

Avoid map, it's bad for Python 3 compatibility. Replace map(float, options['range']) -> [float(x) for x in options['range']] (which is also more readable to non-mathematicians)

[Jeroen Demeyer]

Mixing abs tol and ... is messy, it should be sufficient to use only abs tol here:

            sage: p = P[0]; p.ydata  # abs tol 1e-8
            [0.0, 0.0, -0.211524990023434..., -0.211524990023434..., 0.244978663126864..., 0.244978663126864..., -0.149106180027477..., -0.149106180027477...]

[Jeroen Demeyer]

Needs rebase to 8.4.beta2.

Also don't forget your name as Author.

[Bryan Gin-ge Chen]

In the meantime 1.15.1 was also released.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​d7b54b3	Merge branch 'develop' into 25260_numpy;
​8494c36	Upgrade numpy package version to 1.15.1

[Bryan Gin-ge Chen]

I fixed the merge conflict (in sage/plot/histogram.py due to the upgrade to matplotlib in #25702) and I bumped the version to 1.15.1.

There's one remaining failing doctest in histogram.py: #25702 added a test for the function get_minmax_data with the deprecated normed option. As of 1.15.0, numpy spits out a DeprecationWarning? and I don't know the standard way to deal with this.

[Timo Kaufmann]

Thanks! Can't you just switch it to use the density option?

[Bryan Gin-ge Chen]

Replying to gh-timokau:

Thanks! Can't you just switch it to use the density option?

The point of the test seems to be to check the normed option; note that the first example in the docstring is essentially identical except it uses density.

I thought about just deleting the test, but I figured I'd ask first.

[Timo Kaufmann]

@epalezzato since you authored #25702, why did you add that test testing the normed argument?

[Samuel Lelièvre]

The new way is to use the density option, but Sage wants to give its users some time to adapt, so keeps normed working but with a deprecation warning.

The test should check that the deprecation warning works. Please adapt it to do that.

For examples of deprecation warnings in doctests, see:

​https://github.com/sagemath/sage/search?q=deprecationwarning

After a year, the deprecated stuff can be removed.

[Bryan Gin-ge Chen]

I ran into an annoying interaction between rel tol and ... in the doctesting framework while trying to fix up the test. Reported as #26183.

For now I will try to work around it by making the test a little cruder.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​3aa0921	Merge branch 'develop' into 25260_numpy
​2d60567	Deprecate 'normed' option, fix doctest

[Bryan Gin-ge Chen]

Oops, I haven't addressed comments 16 and 17 yet.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​164bfdc

Reviewer comments

[Jeroen Demeyer]

if options.get('range', None) : do you mean if 'range' in options?

[Jeroen Demeyer]

I'm not convinced about

        if 'normed' in options:
            from sage.misc.superseded import deprecation
            deprecation(25260, "the 'normed' option is deprecated. Use 'density' instead.")

because it's essentially copying a deprecation warning that numpy already gives.

[Jeroen Demeyer]

And I'm also not convinced about

    points = list(points) # make sure points is a python list

Why was that added?

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​1743f70

'normed' is not quite an alias for 'density'

[Bryan Gin-ge Chen]

Thanks for the quick feedback!

Replying to jdemeyer:

if options.get('range', None) : do you mean if 'range' in options?

if 'range' in options: does not work, since that also triggers upon passing range=None (which is the default).

Replying to jdemeyer:

I'm not convinced about

        if 'normed' in options:
            from sage.misc.superseded import deprecation
            deprecation(25260, "the 'normed' option is deprecated. Use 'density' instead.")

because it's essentially copying a deprecation warning that numpy already gives.

What do you suggest? numpy / matplot (which do indeed also throw deprecation warnings) aren't called until either _render_on_subplot or get_minmax_data are run. I wanted to give the user a notice when they construct the initial Histogram object.

Replying to jdemeyer:

And I'm also not convinced about

    points = list(points) # make sure points is a python list

Why was that added?

points = list(points) is necessary to properly treat the case when points is a generator or other non-list iterable. Note the test line(enumerate(range(2)), marker = 'o', legend_label='a') at line 498 which was apparently added for unrelated reasons in #13690.

I made a change to a string which stated that normed was a deprecated alias for density. Strictly speaking, normed has different (incorrect) behavior from density ​when constructing a histogram with non-uniform bars.

---

New commits:

​1743f70

'normed' is not quite an alias for 'density'

---

New commits:

​1743f70

'normed' is not quite an alias for 'density'

[Timo Kaufmann]

Replying to gh-bryangingechen:

Replying to jdemeyer:

I'm not convinced about

        if 'normed' in options:
            from sage.misc.superseded import deprecation
            deprecation(25260, "the 'normed' option is deprecated. Use 'density' instead.")

because it's essentially copying a deprecation warning that numpy already gives.

What do you suggest? numpy / matplot (which do indeed also throw deprecation warnings) aren't called until either _render_on_subplot or get_minmax_data are run. I wanted to give the user a notice when they construct the initial Histogram object.

I'm also in favor of throwing our own deprecation warning. That we're using numpy under the hood is an implementation detail. This way we have more control.

[Timo Kaufmann]

This is positive_review from me, pending Jeroen's judgement on the deprecation warning. Jeroen, would you mind commenting on that?

[Jeroen Demeyer]

I don't have a strong opinion on the deprecation warning, I was mostly just commenting.

However, points = list(points) is still wrong! The docs for line2d state

    -  ``points`` - either a single point (as a tuple), a list of
       points, a single complex number, or a list of complex numbers.

It's not at all clear to me that calling list() on a single point or a single complex number is the right thing to do.

[Timo Kaufmann]

If it helps "to properly treat the case when points is a generator or other non-list iterable" (quote gh-bryangingechen) and doesn't make a difference in the case of a single point, why not?

[Jeroen Demeyer]

Replying to gh-timokau:

doesn't make a difference in the case of a single point

Are you sure about this part?

[Timo Kaufmann]

I have to admit I'm not. gh-bryangingechen, do you know more?

[Bryan Gin-ge Chen]

Hi, thanks for pushing back on this. The code from this ticket indeed works fine with a single point (entered as a tuple, list, or a list containing a single tuple or single list, etc.).

However, in the course of testing this, I realized that I did not completely understand the addition of points = list(points) in line2d() and my answer above to Jeroen was incorrect. Note that this line was actually added by gh-timokau ​here, so I should probably have deferred to him to explain it.

Anyways, here's what I think now. The real reason this line was added was to properly treat line(numpy.array([])) (which is one of the doctests). The previous version of line2d used the more pythonic-looking if not points: to detect "no points". As of numpy 1.15, the truth value of numpy.array([]) is still False, but it returns a DeprecationWarning? that this behavior will be removed.

So to get around this, gh-timokau replaced if not points: with the points = list(points) and if len(points) == 0.

Two observations:

• The test if points in CC or points in CDF: now always fails.

• xydata_from_point_list() ​already has this:

  if not isinstance(points, (list, tuple)):
      points = list(points)
  

For the first point, I think we should just remove this test altogether, since it looks like it was originally there to catch some edge cases for if not points: (e.g. points=CC(0)). One somewhat related thing that I noticed was that symbolic complex input is not handled well: line([1+I,I]) doesn't work but line([CC(1+I),CC(I)]) does. Maybe this should be dealt with in another ticket for xydata_from_point_list though.

The second point gives me pause. Is there a cleaner way to detect both an empty list and an empty numpy array in line2d? I'd appreciate suggestions, as I don't see anything nice at the moment. It does seem like ​dealing with empty numpy arrays is a common pain point.

The current code does work so we could also just go with what we have.

[Timo Kaufmann]

Oh I forgot I added that, thanks for investigating :)

I think if not <something that is not a bool> is an antipattern anyways, even if it still worked. If we don't want to change points to a list permanently, maybe we could just do if len(list(points)) == 0 instead.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​99512a8

Remove obsolete check for CC or CDF in line2d()

[Bryan Gin-ge Chen]

Unfortunately, if len(list(points)) == 0 breaks the line(enumerate(range(2)), marker = 'o', legend_label='a') test, since after spitting out all the values once, future calls to list(points) just return [].

The commit I've just pushed simply removes the dead code and leaves points = list(points) and if len(points) == 0: alone. Since this does work and no better alternatives are coming to mind, I propose we just go with this.

[Timo Kaufmann]

Ah right. I now remember that I had the same problem.

Since this does work and no better alternatives are coming to mind, I propose we just go with this.

+1 from me. Jeroen?

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​f8dc1d1	Merge branch 'develop' into 25260_numpy
​45fb0bd	Upgrade numpy to v1.15.2

[Bryan Gin-ge Chen]

I've changed the target to 1.15.2, as that's just been released. Does anyone understand the numpy patches we have? They seem not to apply to this version and the build currently fails:

[numpy-1.15.2] Found local metadata for numpy-1.15.2
[numpy-1.15.2] Using cached file /Applications/SageMath/upstream/numpy-1.15.2.tar.gz
[numpy-1.15.2] numpy-1.15.2
[numpy-1.15.2] ====================================================
[numpy-1.15.2] Setting up build directory for numpy-1.15.2
[numpy-1.15.2] Finished extraction
[numpy-1.15.2] Applying patches from ../patches...
[numpy-1.15.2] Applying ../patches/fix-numpy-1.13.3-for-python3.7.patch
[numpy-1.15.2] patching file setup.py
[numpy-1.15.2] Hunk #1 succeeded at 381 (offset 11 lines).
[numpy-1.15.2] The next patch would create the file tools/cythonize.py,
[numpy-1.15.2] which already exists!  Assume -R? [n]
[numpy-1.15.2] Apply anyway? [n]
[numpy-1.15.2] Skipping patch.
[numpy-1.15.2] 1 out of 1 hunk ignored
[numpy-1.15.2] Error applying '../patches/fix-numpy-1.13.3-for-python3.7.patch'
[numpy-1.15.2] ************************************************************************
[numpy-1.15.2] Error applying patches
[numpy-1.15.2] ************************************************************************
[numpy-1.15.2] Please email sage-devel (http://groups.google.com/group/sage-devel)
[numpy-1.15.2] explaining the problem and including the log file
[numpy-1.15.2]   /Applications/SageMath/logs/pkgs/numpy-1.15.2.log
[numpy-1.15.2] Describe your computer, operating system, etc.
[numpy-1.15.2] ************************************************************************
make[3]: *** [/Applications/SageMath/local/var/lib/sage/installed/numpy-1.15.2] Error 1
make[2]: *** [all-build] Error 2

I have no experience with patching so if someone wants fix this themselves rather than explain to me how to do it they're also welcome to do so.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​839e504

#25260 remove fix-numpy-1.13.3-for-python3.7.patch

[Thierry Monteil]

Replying to gh-bryangingechen:

I have no experience with patching so if someone wants fix this themselves rather than explain to me how to do it they're also welcome to do so.

A you can see in the first line of $SAGE_ROOT/build/pkgs/numpy/patches/fix-numpy-1.13.3-for-python3.7.patch, there is a link to ​https://github.com/numpy/numpy/issues/10500 which is now fixed (the patch or a similar one was applied upstream), so it sufice to remove that patch. I just did that and recompiled numpy and scipy, but did not run all doctests.

[Thierry Monteil]

By the way, both scipy and numpy offer self-tests using pytest), so we should add some spkg-check for them (not in that ticket).

[Bryan Gin-ge Chen]

Thanks! For what it's worth, after your patch, all the tests seem to pass for me.

[Travis Scrimshaw]

They also pass for me. So I am setting this to a positive review.

[Volker Braun]

This fails on kucalc (48-core linux 64-bit):

**********************************************************************
File "src/sage/misc/inline_fortran.py", line 134, in sage.misc.inline_fortran._import_module_from_path._import_module_from_path_impl
Failed example:
    fortran(code, globals())
Exception raised:
    Traceback (most recent call last):
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 659, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 1070, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.misc.inline_fortran._import_module_from_path._import_module_from_path_impl[1]>", line 1, in <module>
        fortran(code, globals())
      File "sage/misc/lazy_import.pyx", line 354, in sage.misc.lazy_import.LazyImport.__call__ (build/cythonized/sage/misc/lazy_import.c:3759)
        return self.get_object()(*args, **kwds)
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/sage/misc/inline_fortran.py", line 96, in __call__
        return self.eval(*args, **kwds)
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/sage/misc/inline_fortran.py", line 200, in eval
        log_string)
    RuntimeError: failed to compile Fortran code:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/numpy/f2py/f2py2e.py", line 648, in main
        run_compile()
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/numpy/f2py/f2py2e.py", line 633, in run_compile
        setup(ext_modules=[ext])
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/numpy/distutils/core.py", line 169, in setup
        return old_setup(**new_attr)
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/distutils/core.py", line 151, in setup
        dist.run_commands()
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/distutils/dist.py", line 953, in run_commands
        self.run_command(cmd)
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/numpy/distutils/command/build.py", line 47, in run
        old_build.run(self)
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/distutils/command/build.py", line 127, in run
        self.run_command(cmd_name)
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/distutils/cmd.py", line 326, in run_command
        self.distribution.run_command(command)
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/numpy/distutils/command/build_ext.py", line 262, in run
        self.build_extensions()
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/distutils/command/build_ext.py", line 449, in build_extensions
        self.build_extension(ext)
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/numpy/distutils/command/build_ext.py", line 380, in build_extension
        **kws)
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/site-packages/numpy/distutils/ccompiler.py", line 337, in CCompiler_compile
        pool = multiprocessing.pool.ThreadPool(jobs)
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/multiprocessing/pool.py", line 732, in __init__
        Pool.__init__(self, processes, initializer, initargs)
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/multiprocessing/pool.py", line 187, in __init__
        self._result_handler.start()
      File "/var/lib/buildbot/slave/sage_git/build/local/lib/python2.7/threading.py", line 736, in start
        _start_new_thread(self.__bootstrap, ())
    thread.error: can't start new thread

Something is wrong on high-cpu count machines, I think. Works on others.

[Julian Rüth]

Any ideas how we could debug this? k8s has 48 cores so we could try to debug this there.

I'll probably start by building there and running just the tests in inline_fortran.py. Maybe that's already enough to trigger the error.

[Jeroen Demeyer]

For Python packages, the sources on PyPI should always be considered the official sources (unless the project specifically points to GitHub for their sources, which is not the case here).

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​78057db

Upgrade numpy to v1.15.2

[Jeroen Demeyer]

I'm getting a timeout instead on that test:

Doctesting 1 file using 8 threads.
sage -t --long src/sage/misc/inline_fortran.py
    Timed out
**********************************************************************
Tests run before process (pid=16924) timed out:
sage: from sage.misc.inline_fortran import _import_module_from_path ## line 27 ##
sage: modname = '___test__import_module_from_path' ## line 28 ##
sage: tmpdir = tmp_dir() ## line 29 ##
sage: filename = os.path.join(tmpdir, modname + '.py') ## line 30 ##
sage: with open(filename, 'w') as fobj:
    _ = fobj.write('foo = "bar"') ## line 31 ##
sage: mod = _import_module_from_path(modname) ## line 33 ##
sage: mod = _import_module_from_path('DoEsNoTeXiSt', path=[tmpdir]) ## line 37 ##
sage: mod = _import_module_from_path(modname, path=[tmpdir]) ## line 41 ##
sage: mod.foo ## line 42 ##   
'bar'
sage: sig_on_count() # check sig_on/off pairings (virtual doctest) ## line 44 ##
0
sage: code = '''
C FILE: FIB1.F
      SUBROUTINE FIB(A,N)
C
C     CALCULATE FIRST N FIBONACCI NUMBERS
C
      INTEGER N
      REAL*8 A(N)
      DO I=1,N
         IF (I.EQ.1) THEN
            A(I) = 0.0D0
         ELSEIF (I.EQ.2) THEN 
            A(I) = 1.0D0
         ELSE
            A(I) = A(I-1) + A(I-2)
         ENDIF
      ENDDO
      END
C END FILE FIB1.F
''' ## line 114 ##
sage: fortran(code, globals()) ## line 134 ##
------------------------------------------------------------------------
/home/jdemeyer/sage-git/local/lib/python2.7/site-packages/cysignals/signals.so(+0x851c)[0x3fff7ea0851c]
/home/jdemeyer/sage-git/local/lib/python2.7/site-packages/cysignals/signals.so(+0x8638)[0x3fff7ea08638]
/home/jdemeyer/sage-git/local/lib/python2.7/site-packages/cysignals/signals.so(+0xd370)[0x3fff7ea0d370]
[0x3fff7ffa04d8]
/lib/powerpc64le-linux-gnu/libc.so.6(__read+0x58)[0x3fff7fc0e02c]
[0x3fffd1c122b0]
/lib/powerpc64le-linux-gnu/libc.so.6(_IO_file_read+0x64)[0x3fff7fb88674]
/lib/powerpc64le-linux-gnu/libc.so.6(+0x8839c)[0x3fff7fb8839c]
/lib/powerpc64le-linux-gnu/libc.so.6(_IO_sgetn+0x28)[0x3fff7fb8bac8]
/lib/powerpc64le-linux-gnu/libc.so.6(fread+0xb4)[0x3fff7fb78a84]
/home/jdemeyer/sage-git/local/lib/libpython2.7.so.1.0(Py_UniversalNewlineFread+0x1e8)[0x3fff7fdc2668]
[...]
----------------------------------------------------------------------
sage -t --long src/sage/misc/inline_fortran.py  # Timed out
----------------------------------------------------------------------
Total time for all tests: 1800.1 seconds
    cpu time: 0.0 seconds
    cumulative wall time: 0.0 seconds

[Jeroen Demeyer]

Running this under strace shows indeed that f2py is creating a huge number of threads which at some points leads to out-of-memory problems. For some reason, this is causing a hang for me instead of an exception.

[Jeroen Demeyer]

I reported the thread issue upstream: ​https://github.com/numpy/numpy/issues/12087

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​40f4318

Use just 1 thread for compiling to prevent bombing the system

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​3508a05

Limit number of threads to prevent bombing the system

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​b151634

Limit number of threads to prevent bombing the system

[Timo Kaufmann]

Just for the record I would prefer to monkey-patch instead (or maybe additionally). It seems to be a clear bug in this case and downstream distributions should probably patch, but I still think sage should do what it can to prevent errors even on an unpatched system.

I know your views on this but still wanted to make that point.

[Julian Rüth]

gh-timokau, I would usually agree that we should monkey-patch. However, in this case, there seems to be nothing Sage specific going on so I do not see a strong reason to monkey-patch. In particular, this does not completely break Sage but just breaks things in some rare cases. As this is a numpy bug, distributions should backport it in their numpy if they care.

Probably more importantly, this would be somewhat hard to monkey-patch. Numpy is not imported in a central spot in Sage as the import takes a while, so it is only loaded on demand. Therefore, there is not a single spot where the monkey-patching could happen.

[Timo Kaufmann]

Yes, I agree that the argument is a bit weak in this case. If it was easy I would still prefer it, but since its not its not that bad.

[Jeroen Demeyer]

The patch was accepted upstream.

[Samuel Lelièvre]

Should we then upgrade to NumPy 1.14.6 now, or wait for NumPy 1.15.3 to be released?

[Timo Kaufmann]

From a packaging perspective its preferable to upgrade to the latest version now. The latest version just requires a patch for some edge case. Without the upgrade sage is completely incompatible with newer numpy versions (which distros probably have by now).

Thank you for your efforts in fixing this upstream Jeroen!

[Erik Bray]

This should be re-targeted for 8.5.

[Samuel Lelièvre]

Next NumPy upgrade at #26643.

[Alex J. Best]

I get two doctest failures on OSX with SageMath version 8.5.beta2, which I believe are related to this ticket, please let me know what other information is helpful (or if I should ignore this).

File "src/sage/schemes/elliptic_curves/height.py", line 1629, in sage.schemes.elliptic_curves.height.EllipticCurveCanonicalHeight.wp_on_grid
Failed example:
    H.wp_on_grid(v,4)
Expected:
    array([[25.43920182,  5.28760943,  5.28760943, 25.43920182],
           [ 6.05099485,  1.83757786,  1.83757786,  6.05099485],
           [ 6.05099485,  1.83757786,  1.83757786,  6.05099485],
           [25.43920182,  5.28760943,  5.28760943, 25.43920182]])
Got:
    array([[ 25.43920182,   5.28760943,   5.28760943,  25.43920182],
           [  6.05099485,   1.83757786,   1.83757786,   6.05099485],
           [  6.05099485,   1.83757786,   1.83757786,   6.05099485],
           [ 25.43920182,   5.28760943,   5.28760943,  25.43920182]])
**********************************************************************
File "src/sage/schemes/elliptic_curves/height.py", line 1637, in sage.schemes.elliptic_curves.height.EllipticCurveCanonicalHeight.wp_on_grid
Failed example:
    H.wp_on_grid(v,4,True)
Expected:
    array([[25.43920182,  5.28760943],
           [ 6.05099485,  1.83757786],
           [ 6.05099485,  1.83757786],
           [25.43920182,  5.28760943]])
Got:
    array([[ 25.43920182,   5.28760943],
           [  6.05099485,   1.83757786],
           [  6.05099485,   1.83757786],
           [ 25.43920182,   5.28760943]])
**********************************************************************

[François Bissey]

I don't think it is the first time we have a formatting issue of that kind but I am not sure how to deal with it. The first thing that comes to mind is: are you using OS X' clang or sage's gcc?

[Alex J. Best]

I'm not 100% sure I'm looking in the right place but looks like homebrew gcc, version 8.2.0. As far as I know I haven't tried to change to clang for my sage install.

I have this line in config.log Configured with: ../configure --build=x86_64-apple-darwin17.7.0 --prefix=/usr/local/Cellar/gcc/8.2.0 --libdir=/usr/local/Cellar/gcc/8.2.0/lib/gcc/8 --enable-languages=c,c++,objc,obj-c++,fortran --program-suffix=-8 --with-gmp=/usr/local/opt/gmp --with-mpfr=/usr/local/opt/mpfr --with-mpc=/usr/local/opt/libmpc --with-isl=/usr/local/opt/isl --with-system-zlib --enable-checking=release --with-pkgversion='Homebrew GCC 8.2.0' --with-bugurl=https://github.com/Homebrew/homebrew-core/issues --disable-nls

[François Bissey]

Hum... Not sure. You should open a new ticket in any case. While numpy could be your culprit if numpy arrays are used it could be something else entirely.