Opened 2 years ago
Last modified 3 months ago
#26231 needs_info defect
Import _tkinter after sage complains about libfontconfig library
Reported by: | mmarco | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | sage-8.4 |
Component: | graphics | Keywords: | |
Cc: | jdemeyer, dunfield, was, chapoton, saraedum, embray, slelievre | Merged in: | |
Authors: | Miguel Marco | Reviewers: | |
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description
I am experiencing the following error:
┌────────────────────────────────────────────────────────────────────┐ │ SageMath version 8.3, Release Date: 2018-08-03 │ │ Type "notebook()" for the browser-based notebook interface. │ │ Type "help()" for help. │ └────────────────────────────────────────────────────────────────────┘ sage: plot(sin) /home/mmarco/sage-8.3/local/lib/python2.7/site-packages/sage/repl/rich_output/display_manager.py:592: RichReprWarning: Exception in _rich_repr_ while displaying object: /usr/lib64/libfontconfig.so.1: undefined symbol: FT_Done_MM_Var RichReprWarning, Graphics object consisting of 1 graphics primitive
I could trace the culprit to
sage: _.show() --------------------------------------------------------------------------- ImportError Traceback (most recent call last) <ipython-input-3-e1aa3d6e2a32> in <module>() ----> 1 _.show() /home/mmarco/sage-8.3/local/lib/python2.7/site-packages/sage/misc/decorators.pyc in wrapper(*args, **kwds) 484 kwds[self.name + "options"] = suboptions 485 --> 486 return func(*args, **kwds) 487 488 #Add the options specified by @options to the signature of the wrapped /home/mmarco/sage-8.3/local/lib/python2.7/site-packages/sage/plot/graphics.pyc in show(self, **kwds) 1984 from sage.repl.rich_output import get_display_manager 1985 dm = get_display_manager() -> 1986 dm.display_immediately(self, **kwds) 1987 1988 def xmin(self, xmin=None): /home/mmarco/sage-8.3/local/lib/python2.7/site-packages/sage/repl/rich_output/display_manager.pyc in display_immediately(self, obj, **rich_repr_kwds) 833 1/2 834 """ --> 835 plain_text, rich_output = self._rich_output_formatter(obj, rich_repr_kwds) 836 self._backend.display_immediately(plain_text, rich_output) 837 /home/mmarco/sage-8.3/local/lib/python2.7/site-packages/sage/repl/rich_output/display_manager.pyc in _rich_output_formatter(self, obj, rich_repr_kwds) 623 has_rich_repr = isinstance(obj, SageObject) and hasattr(obj, '_rich_repr_') 624 if has_rich_repr: --> 625 rich_output = self._call_rich_repr(obj, rich_repr_kwds) 626 if isinstance(rich_output, OutputPlainText): 627 plain_text = rich_output /home/mmarco/sage-8.3/local/lib/python2.7/site-packages/sage/repl/rich_output/display_manager.pyc in _call_rich_repr(self, obj, rich_repr_kwds) 581 if rich_repr_kwds: 582 # do not ignore errors from invalid options --> 583 return obj._rich_repr_(self, **rich_repr_kwds) 584 try: 585 return obj._rich_repr_(self) /home/mmarco/sage-8.3/local/lib/python2.7/site-packages/sage/plot/graphics.pyc in _rich_repr_(self, display_manager, **kwds) 881 if output_container in display_manager.supported_output(): 882 return display_manager.graphics_from_save( --> 883 self.save, kwds, file_ext, output_container) 884 885 def __str__(self): /home/mmarco/sage-8.3/local/lib/python2.7/site-packages/sage/repl/rich_output/display_manager.pyc in graphics_from_save(self, save_function, save_kwds, file_extension, output_container, figsize, dpi) 711 if dpi is not None: 712 kwds['dpi'] = dpi --> 713 save_function(filename, **kwds) 714 from sage.repl.rich_output.buffer import OutputBuffer 715 buf = OutputBuffer.from_file(filename) /home/mmarco/sage-8.3/local/lib/python2.7/site-packages/sage/misc/decorators.pyc in wrapper(*args, **kwds) 484 kwds[self.name + "options"] = suboptions 485 --> 486 return func(*args, **kwds) 487 488 #Add the options specified by @options to the signature of the wrapped /home/mmarco/sage-8.3/local/lib/python2.7/site-packages/sage/plot/graphics.pyc in save(self, filename, **kwds) 3149 rc_backup = (rcParams['ps.useafm'], rcParams['pdf.use14corefonts'], 3150 rcParams['text.usetex']) # save the rcParams -> 3151 figure = self.matplotlib(**options) 3152 # You can output in PNG, PS, EPS, PDF, PGF, or SVG format, depending 3153 # on the file extension. /home/mmarco/sage-8.3/local/lib/python2.7/site-packages/sage/plot/graphics.pyc in matplotlib(self, filename, xmin, xmax, ymin, ymax, figsize, figure, sub, axes, axes_labels, axes_labels_size, fontsize, frame, verify, aspect_ratio, gridlines, gridlinesstyle, vgridlinesstyle, hgridlinesstyle, show_legend, legend_options, axes_pad, ticks_integer, tick_formatter, ticks, title, title_pos, base, scale, stylesheet, typeset) 2526 ticks = (ticks, None) 2527 -> 2528 import matplotlib.pyplot as plt 2529 if stylesheet not in plt.style.available: 2530 stylesheet = 'classic' /home/mmarco/sage-8.3/local/lib/python2.7/site-packages/matplotlib/pyplot.py in <module>() 111 ## Global ## 112 --> 113 _backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup() 114 115 _IP_REGISTERED = None /home/mmarco/sage-8.3/local/lib/python2.7/site-packages/matplotlib/backends/__init__.pyc in pylab_setup(name) 58 # imports. 0 means only perform absolute imports. 59 backend_mod = __import__(backend_name, globals(), locals(), ---> 60 [backend_name], 0) 61 62 # Things we pull in from all backends /home/mmarco/sage-8.3/local/lib/python2.7/site-packages/matplotlib/backends/backend_tkagg.py in <module>() 4 5 import six ----> 6 from six.moves import tkinter as Tk 7 from six.moves import tkinter_filedialog as FileDialog 8 /home/mmarco/sage-8.3/local/lib/python2.7/site-packages/six.pyc in load_module(self, fullname) 201 mod = self.__get_module(fullname) 202 if isinstance(mod, MovedModule): --> 203 mod = mod._resolve() 204 else: 205 mod.__loader__ = self /home/mmarco/sage-8.3/local/lib/python2.7/site-packages/six.pyc in _resolve(self) 113 114 def _resolve(self): --> 115 return _import_module(self.mod) 116 117 def __getattr__(self, attr): /home/mmarco/sage-8.3/local/lib/python2.7/site-packages/six.pyc in _import_module(name) 80 def _import_module(name): 81 """Import module, returning the module after the last dot.""" ---> 82 __import__(name) 83 return sys.modules[name] 84 /home/mmarco/sage-8.3/local/lib/python2.7/lib-tk/Tkinter.py in <module>() 37 # Attempt to configure Tcl/Tk without requiring PATH 38 import FixTk ---> 39 import _tkinter # If this fails your Python may not be configured for Tk 40 tkinter = _tkinter # b/w compat for export 41 TclError = _tkinter.TclError ImportError: /usr/lib64/libfontconfig.so.1: undefined symbol: FT_Done_MM_Var
Indeed, the same kind of error appears just trying to import _tkinter
.
However, in a clean sage -python
session, importing _tkinter
does not give any problem. Furthermore, if in a clean python session I run
from sage.all import * import _tkinter
the error happens, but it doesn't happen if I change the order of imports (that is, importing _tkinter
before the sage library is ok.
I could circumvent the problem by adding a line
import _tkinter
at the file sage/all.py
before the line
from sage.matrix.all import *
So, for some reason, it seems like some code in the matrix module messes in a subtle way with something that tkinter expects.
Change History (21)
comment:1 Changed 2 years ago by
comment:2 Changed 2 years ago by
- Status changed from new to needs_review
comment:3 Changed 2 years ago by
Sorry, I didn't fully read the first part of your message. I thought for some reason you were explicitly trying to use Tkinter directly. Regardless, the solution is still basically the same: It shouldn't be importing _tkinter
at all unless you have some bad PYTHONPATH
or something.
Also maybe a ~/.config/matplotlib
setting the TkAgg backend by default? I feel like Sage shouldn't be loading the default matplotlib config as it could conflict with other matplotlibs on the system.
comment:4 Changed 2 years ago by
Oddly enough, the bug appears without me manually importing tkinter. Just trying to plot a graph, on a clean sage install (compiled from the source tarball).
Maybe at some point matplotlib (or some other component) tries to use tkinter to show plots?
comment:5 Changed 2 years ago by
(In fact, sage-env
does set export MPLCONFIGDIR="$DOT_SAGE/matplotlib-1.5.1"
. Probably the sage.env
module should do the same if MPLCONFIGDIR
is not otherwise set.)
comment:6 Changed 2 years ago by
I have removed the .local/matplotlib directory
(it was empty anyways), and checked that I didn't have the PYTHONPATH
variable set. The problem persists.
I also have the same feeling: for some reason Sage is trying to use tkinter to show the plots, but can't find exactly what is triggering that behaviour.
In any case, I think there are two orthogonal problems:
- Sage trying to use tkinter, when it shouldn't
- Importing the sage library breaks the ability to import other python libraries.
comment:7 Changed 2 years ago by
The latter isn't a real problem; it's expected. The former is strange but I think it's something odd on your system configuration. Starting with the fact that you can apparently import an _tkinter
module at all. If you do:
sage -python -c 'import _tkinter; print(_tkinter)'
what does it show?
comment:8 Changed 2 years ago by
I get this:
mmarco@localhost ~/sage-8.3 $ ./sage -python -c 'import _tkinter; print(_tkinter)' <module '_tkinter' from '/home/mmarco/sage-8.3/local/lib/python2.7/lib-dynload/_tkinter.so'>
comment:9 Changed 2 years ago by
- Status changed from needs_review to needs_info
Perhaps I'm wrong then. It looks like maybe Python will build the _tkinter
module automatically if it happens to find working Tcl/Tk libs somewhere.
Maybe I would err towards not doing that, since it can result in inconsistent builds of Sage (some that have Tkinter, some that don't). Unless we explicitly support it we should disable it. Also matplotlib should be configured by Sage to use the 'agg'
backend for outputting plots.
comment:10 Changed 2 years ago by
That makes sense to me.
comment:11 follow-up: ↓ 15 Changed 2 years ago by
The tkinter
module is part of the standard Python library and has been since the 1990s. Thus Python builds _tkinter
if it can find the Tcl/Tk
library and headers, just as it does the bz2
module or any of its other standard modules that depend on some optional library or another. I would argue very strongly against disabling building tkinter
for Sage. Admittedly, we use it heavily in SnapPy
so I am no doubt biased, but except for servers that run headless tkinter
is usually included with Python or trivial to install.
comment:12 follow-ups: ↓ 14 ↓ 16 Changed 2 years ago by
Some thoughts on the initial report. What happens if you do
sage: %matplotlib
to list the current backend? Historically, I would have expected the response to be Using matplotlib backend: agg
, independent of whether Tkinter is working, but I suspect you're going to get tk
or tkagg
.
I see that Sage recently upgraded to matplotlib 2.1.0
from the 1.*
series. One thing they changed in 2.* is that the TkAgg
backend is always built, even when Tk
is not available or simply not working. This is because they figured out how to build the TkAgg
backend without having the Tk
library installed at build time. See can have the effect of making TkAgg
the default backend, see https://github.com/matplotlib/matplotlib/pull/7530 Now, they claim to have fixed that particular instance, but likely this is some variant.
One solution might be to configure Sage so that it sets the default backend back to Agg
. I'm not expert enough to know the best place to do this, but assuming the Sage interpreter is importing matplotlib
at some stage, then you could probably just do:
matplotlib.use('Agg')
right after that.
comment:13 Changed 2 years ago by
- Cc slelievre added
It is general that when building Python, which Python modules get built depends on ambient functionality:
- the
tkinter
module given ambient Tcl/Tk functionality, as Nathan pointed out, - the
ssl
module given ambient SSL functionality --- crucial to let us "pip install" from PyPI!
Tcl/Tk capability, if present, adds some extra functionality in Sage.
See the references to Tcl/Tk in the Sage documentation:
- https://doc.sagemath.org/html/en/installation/source.html#tcl-tk
- https://doc.sagemath.org/html/en/faq/faq-usage.html#how-to-get-sage-s-python-to-recognize-my-system-s-tcl-tk-install
- https://doc.sagemath.org/html/en/constructions/plotting.html#openmath
It can be more important for some optional or external packages, such as SnapPy.
comment:14 in reply to: ↑ 12 Changed 2 years ago by
Replying to dunfield:
Some thoughts on the initial report. What happens if you do
sage: %matplotlibto list the current backend? Historically, I would have expected the response to be
Using matplotlib backend: agg
, independent of whether Tkinter is working, but I suspect you're going to gettk
ortkagg
.
sage: %matplotlib --------------------------------------------------------------------------- ImportError Traceback (most recent call last) <ipython-input-1-040f64586c53> in <module>() ----> 1 get_ipython().magic(u'matplotlib') /home/mmarco/sage/local/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in magic(self, arg_s) 2158 magic_name, _, magic_arg_s = arg_s.partition(' ') 2159 magic_name = magic_name.lstrip(prefilter.ESC_MAGIC) -> 2160 return self.run_line_magic(magic_name, magic_arg_s) 2161 2162 #------------------------------------------------------------------------- /home/mmarco/sage/local/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in run_line_magic(self, magic_name, line) 2079 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals 2080 with self.builtin_trap: -> 2081 result = fn(*args,**kwargs) 2082 return result 2083 <decorator-gen-105> in matplotlib(self, line) /home/mmarco/sage/local/lib/python2.7/site-packages/IPython/core/magic.pyc in <lambda>(f, *a, **k) 186 # but it's overkill for just that one bit of state. 187 def magic_deco(arg): --> 188 call = lambda f, *a, **k: f(*a, **k) 189 190 if callable(arg): /home/mmarco/sage/local/lib/python2.7/site-packages/IPython/core/magics/pylab.pyc in matplotlib(self, line) 98 print("Available matplotlib backends: %s" % backends_list) 99 else: --> 100 gui, backend = self.shell.enable_matplotlib(args.gui) 101 self._show_matplotlib_backend(args.gui, backend) 102 /home/mmarco/sage/local/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in enable_matplotlib(self, gui) 2948 gui, backend = pt.find_gui_and_backend(self.pylab_gui_select) 2949 -> 2950 pt.activate_matplotlib(backend) 2951 pt.configure_inline_support(self, backend) 2952 /home/mmarco/sage/local/lib/python2.7/site-packages/IPython/core/pylabtools.pyc in activate_matplotlib(backend) 306 matplotlib.rcParams['backend'] = backend 307 --> 308 import matplotlib.pyplot 309 matplotlib.pyplot.switch_backend(backend) 310 /home/mmarco/sage/local/lib/python2.7/site-packages/matplotlib/pyplot.py in <module>() 111 ## Global ## 112 --> 113 _backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup() 114 115 _IP_REGISTERED = None /home/mmarco/sage/local/lib/python2.7/site-packages/matplotlib/backends/__init__.pyc in pylab_setup(name) 58 # imports. 0 means only perform absolute imports. 59 backend_mod = __import__(backend_name, globals(), locals(), ---> 60 [backend_name], 0) 61 62 # Things we pull in from all backends /home/mmarco/sage/local/lib/python2.7/site-packages/matplotlib/backends/backend_tkagg.py in <module>() 4 5 import six ----> 6 from six.moves import tkinter as Tk 7 from six.moves import tkinter_filedialog as FileDialog 8 /home/mmarco/sage/local/lib/python2.7/site-packages/six.pyc in load_module(self, fullname) 201 mod = self.__get_module(fullname) 202 if isinstance(mod, MovedModule): --> 203 mod = mod._resolve() 204 else: 205 mod.__loader__ = self /home/mmarco/sage/local/lib/python2.7/site-packages/six.pyc in _resolve(self) 113 114 def _resolve(self): --> 115 return _import_module(self.mod) 116 117 def __getattr__(self, attr): /home/mmarco/sage/local/lib/python2.7/site-packages/six.pyc in _import_module(name) 80 def _import_module(name): 81 """Import module, returning the module after the last dot.""" ---> 82 __import__(name) 83 return sys.modules[name] 84 /home/mmarco/sage/local/lib/python2.7/lib-tk/Tkinter.py in <module>() 37 # Attempt to configure Tcl/Tk without requiring PATH 38 import FixTk ---> 39 import _tkinter # If this fails your Python may not be configured for Tk 40 tkinter = _tkinter # b/w compat for export 41 TclError = _tkinter.TclError ImportError: /usr/lib64/libfontconfig.so.1: undefined symbol: FT_Done_MM_Var
comment:15 in reply to: ↑ 11 Changed 2 years ago by
Replying to dunfield:
but except for servers that run headless
tkinter
is usually included with Python or trivial to install
That's a big "BUT". Most of our CI infrastructure and servers like SageCell? and its variants are effectively "headless". If Tkinter is to be supported by Sage that means Sage has to include its own Tcl/Tk? as well since balancing the system versions of those libraries with Sage's other bundled dependencies would be very tricky in many cases (not that I think that situation is a good thing, but it it is what it is).
I would argue that Tkinter should not be supported by sage-the-distribution until and unless we have our house in better order w.r.t. other packaging issues. That, or you'll have to package Tcl/Tk? for Sage, which is perhaps not entirely unreasonable, but someone else will have to do that. It should also be optional of course since it would not be useful in most server cases.
comment:16 in reply to: ↑ 12 ; follow-up: ↓ 19 Changed 2 years ago by
Replying to dunfield:
One solution might be to configure Sage so that it sets the default backend back to
Agg
. I'm not expert enough to know the best place to do this, but assuming the Sage interpreter is importingmatplotlib
at some stage, then you could probably just do:matplotlib.use('Agg')right after that.
I generally agree that this should be the case. We already do exactly this when running the doctest suite. However, forcibly doing this is still problematic since it could override possibly intentional user settings. If there's a way to force agg to be the default backend while possibly still respecting a custom mplconfig that would be ideal probably.
comment:17 follow-up: ↓ 18 Changed 2 years ago by
Does anyone know why Sage packages libfreetype in the first place? Because that's probably the source of the problem here.
comment:18 in reply to: ↑ 17 ; follow-up: ↓ 20 Changed 2 years ago by
Replying to embray:
Does anyone know why Sage packages libfreetype in the first place?
Unfortunately, I believe it is a requirement of matplotlib itself. Specifically, a dependency of matplotlib/ft2font.so
. It is also used by pillow/PIL.
comment:19 in reply to: ↑ 16 Changed 2 years ago by
Replying to embray:
If there's a way to force agg to be the default backend while possibly still respecting a custom mplconfig that would be ideal probably.
Yes, that seems like the best solution. I've never had any difficulty overriding the defaults when using matplotlib in Sage, and I do this fairly often. Perhaps we should just try one or two ways of making agg
the default and see if any of them work... ;-)
comment:20 in reply to: ↑ 18 ; follow-up: ↓ 21 Changed 2 years ago by
Replying to dunfield:
Replying to embray:
Does anyone know why Sage packages libfreetype in the first place?
Unfortunately, I believe it is a requirement of matplotlib itself. Specifically, a dependency of
matplotlib/ft2font.so
. It is also used by pillow/PIL.
Probably another good candidate for defaulting to the system lib where possible, rather than using the Sage version.
comment:21 in reply to: ↑ 20 Changed 3 months ago by
Replying to embray:
Replying to dunfield:
Replying to embray:
Does anyone know why Sage packages libfreetype in the first place?
Unfortunately, I believe it is a requirement of matplotlib itself. Specifically, a dependency of
matplotlib/ft2font.so
. It is also used by pillow/PIL.Probably another good candidate for defaulting to the system lib where possible, rather than using the Sage version.
Normally Sage does not even include Tkinter. It is not included by default when building Python, normally, and IIRC you have to explicitly enable it when building Python which we don't for Sage.
If you're doing some
import Tkinter
then you must be pulling it in from some other Python, which is not supported behavior. This is especially problematic if your_tkinter
was linked against system libraries (in this case, indirectly, libfreetype most likely) that conflict with versions included in Sage.I would propose "wontfix" for this since Sage doesn't include Tkinter (perhaps it should be that's another question...)