Monday, February 19, 2007

New blog for my health book

I have started a blog for 'Why Do People Get Ill?'

Monday, September 04, 2006

Migration

I have decided to migrate to somewhere new in the blogosphere. It's the n-category cafe.

Tuesday, August 15, 2006

Hiatus

The family holiday is upon me, taking me away from the blogosphere. I doubt I'll be able to tune in again before September. If you're not a spammer, feel free to add comments while I'm away.

Something to add to your bookmarks in the mean time is this blog by Alexandre Borovik. He's only written an introductory post so far, but without putting too much pressure on him, I expect Mathematics under the Microscope will prove very interesting. I've mentioned Borovik on my blog here (Nov 5 and 12) and here.

Ruminating

If some blog posts record the results of the author's digesting some body of thought, what follows is some, at best, half-chewed reflections on my latest wanderings in machine learning.

First, something which seems inescapable if you're looking to impose a geometry on a statistic manifold is the Fisher information metric. Now, it appears that a good justification for this was given by Censov in 1982. Apparently, this is the only Riemannian metric invariant under congruent embeddings by a Markov morphism. What this amounts to is requiring that the effect of re-partitioning an event space on a probability distribution be sensible. I found this out from Guy Lebanon's very interesting thesis, where he extends the result to conditional spaces (chapter 6). These are useful for modelling the conditional distribution of output data on input data, rather than the joint distribution of this data. Campbell had already extended Censov's results to non-normalized positive measures, on the way dropping the category theoretic apparatus. (It's never too late to reintroduce it.)

Now the distances that fit neatly with the Fisher information metric are the δ-divergences (p. 5 of this), which include the Kullback and reverse Kullback divergences. This opens you to the glorious world of information geometry (see this list), convex optimization, Legendre transforms between δ-coordinates and (1 - δ)-coordinates, etc. The Zhu and Rohwer articles argue for the advantages of working within the space of all positive measures, rather than of normalized probability distributions, which is δ-flat for all δ, i.e., Christoffel symbols vanish.

All is going swimmingly, except that with some spaces of model you're interested in, like multi-layered neural nets and other graphical models, there's no one-one mapping between the model parameters and the space of distributions, which messes up the geometry in parameter space. Now, there was a trend to move away from neural nets, but they have never quite disappeared. Some, like Geoffrey Hinton, still hope that we can learn something about the brain from studying plausible neural net algorithms, see What kind of a Graphical Model is the Brain?, perhaps discovering some conceptual representations in the higher layers of a trained net.

This runs against the idea that we'd be better off simplifying our task by producing a machine which can merely discriminate between inputs, such as images of 4s and images of 8s, rather than a model which aims to generate the data. But Hinton claims to be able to produce more accurate generative models than the best discriminative classifiers.

A second trend, especially if you were a Bayesian neural net person, was to notice that in some kind of limit of the number of hidden nodes in a layer, what emerged was a Gaussian process. (For the life of me I can't see why information geometry hasn't invaded Gaussian process theory.)

Perhaps, then, layered models are worth sticking with. So is there anything we can do with the non-smooth mapping between parameter space and distribution space. Yes, we turn to algebraic geometry. First, we can follow Watanabe and use Hironaka's resolution of singularities. Second, we follow Pachter and Sturmfels, and say that


(a) Statistical models are algebraic varieties.
(b) Every algebraic variety can be tropicalized.
(c) Tropicalized statistical models are fundamental for parametric inference.
An easy example of (a), concerning a distribution of two binary variables, expresses the independence of these variables as requiring the distribution to satisfy an equation in R4, namely, p00.p11 - p01.p10 = 0. But what are the tropics doing here? Well tropical maths is what John Baez and I were discussing here, and Sturmfels has a gentle introduction here. I have a sneaking feeling it would be worth trying to understand whether the tropical/ordinary = Legendre/Laplace transform analogy has anything to do with the appearance of the Legendre transform earlier.

Well, I did say it was half-chewed.

Monday, August 14, 2006

MacIntyre and the state of philosophy

Alasdair MacIntyre is concerned that philosophy has come to play such a minor role in modern society, specifically with regard to moral philosophy in Moral Philosophy and contemporary social practice: what holds them apart?, and more generally in Philosophy recalled to its tasks: a Thomistic reading of Fides et Ratio. (Both articles in The Tasks of Philosophy, CUP, 2006, references below from this book unless otherwise stated.) 'Fides et Ratio' is a papal encyclical which sees a central and autonomous role for philosophy in the search to better understand 'truth' and what kind of 'good' it constitutes in our lives. Here is one part of MacIntyre's diagnosis:
Philosophers do in fact become irrelevant to others not only by making their utterances inaccessible, but also by losing sight of the often complex and indirect connections between their own specialized, detailed and piecemeal enquiries and those larger questions which give point and purpose to the philosophical enterprise, which rescue it from being no more than a set of intellectually engaging puzzles. Part of what is needed to remedy this is to call to mind a third salient characteristic of philosophy identified in the encyclical, its systematic character. Philosophy does not consist of a set of independent and heterogeneous enquiries into distinct and unconnected problems: the characterization of space and time, the nature of the human good, the relationship of perceived qualities to the causes of perception, how referring expressions function, what standards govern aesthetic judgment, the nature of causality, and so on. For the answers that we give to each of these questions impose constraints upon what answers we can defensibly give to some at least of the others. And when from collaborative work in a number of areas the logical, conceptual, empirical, and metaphysical relationships between each of these sets of answers begin to emerge, we commonly find that we have at least an outline of a system, a system that will inescapably have implications for how the philosophical questions posed by plain persons are to be answered. We will have reached a point at which we are able to recognize the need for a comprehensive vision of the human good and of the order of things (30, 46). System-building however can itself degenerate into a form of philosophical vice against which the encyclical warns us (4). Philosophers who are aware of the systematic character of their enterprise may always fall in love with their own system to such an extent that they gloss over what they ought to recognize as intractable difficulties or unanswerable questions. Love of that particular system displaces the love of truth. If the vice of reducing philosophy to a set of piecemeal, apparently unconnected set of enquiries is the characteristic analytical vice, this vice of system-lovers may perhaps be called the idealist vice. (p. 181)
One would imagine, then, that MacIntyre would be pleased by efforts on the part of analytic philosophers to link virtue ethics to epistemology, see, e.g., here and here. After all, he's famed for his revival of the virtue-based ethics of Aristotle and Aquinas. However, epistemology is not the proper study of our quest for understanding,

For if the Thomist is faithful to the intentions of Aristotle and Aquinas, he or she will not be engaged, except perhaps incidently, in an epistemological enterprise...

The epistemological enterprise is by its nature a first-person project. How can I, so the epistemologist enquires, be assured that my beliefs, my perceptions, my judgments connect with reality external to them, so that I can have justified certitude regarding their error and truth? ...But the thomist, if he or she follows Aristotle or Aquinas, constructs an account both of approaches to and of the achievements of knowledge from a third-person point of view. My mind or rather my soul is only one among many and its knowledge of my self qua soul has to be integrated into the general account of souls and their teleology. Insofar as a given soul moves successfully towards its successive intellectual goals in a teleologically ordered way, it moves towards completing itself by becoming formally identical with the objects of its knowledge, so that it is adequate to those objects, objects that are then no longer external to it, but rather complete it. (pp. 148-149)
It seems that I should be reading Jonathan Kvanvig as a virtue epistemologist who explores the social and genetic aspects of enquiry.

Now, this linking of what others might consider disjoint areas of philosophy continues. Part and parcel of MacIntyre's position, is the inextricable unity of ethics and politics. If ethics is being related to a theory of enquiry, then so must politics. And this should hardly surprise us given what I mentioned before about MacIntyre learning from philosophers of science such as Kuhn, Lakatos, Popper and Feyerabend. What is very striking about these philosophers is how they understand aspects of science in political terms.

This now raises a further issue. In The Essential Contestability of Some Social Concepts, Ethics 84(1) 1-9, 1973 (available on JSTOR), MacIntyre remarks:

Consider...the continuing argument between Kuhn, Lakatos, Polanyi, and Feyerbend, an argument in which what is at stake includes both our ability to draw a line between authentic sciences and degenerative or imitative sciences, such as astrology or phrenology, and our ability to explain why "German physics" and Lysenko biology are not to be included in science. A crucial feature of these arguments is the way in which dispute over the norms which govern scientific practice interlocks with debate over how the history of science is to be written. What identity and continuity are recognized will of course depend on what side is taken in these latter debates but since these debates are so intimately related to the arguments about the norms governing practice, it turns out that the dispute over norms and the dispute over continuity and identity cannot be separated. (p. 7)

A theory of intellectual enquiry must, then, include a theory of the writing of the narrative history of a tradition of enquiry.

A particular way of writing the history of science, the history of philosophy and intellectual history in general willbe the counterpart of a Thomistic conception of rational enquiry, and insofar as that history makes the course of actual enquiry more intelligible than do rival conceptions, the Thomistic conception will have been further vindicated. (167-168)

Of every particular enquiry there is a narrative to be written, and being able to understand that enquiry is inseperable from being able to understand that enquiry is inseperable from being able to identify and follow that narrative. (p. 168)

...from an Aristotelian standpoint it is only in the context of a particularly socially organized and morally informed way of conducting enquiry that the central concepts crucial to a view of enquiry as truth-seeking , engaged in rational justification, and realistic in its selfunderstanding, can intelligibly be put to work. (p. 169)
For some reflections on writing such a history for mathematics, see here and here.

So we have ethics, politics, philosophy of history and the theory of enquiry inextricably linked. But then,
...we need to learn from Aquinas that any such account of truth is incomplete, and therefore more questionable than it needs to be, until it is situated within a larger teleological view of human nature, according to which truth, understood as adaequatio, is also understood as constitutive of the human good. (p. 215)
If we follow MacIntyre, we can hardly avoid, then, encountering debates in the philosophy of mind concerning the relationship between the physical workings of a body and its directed activity. How far are we here?
We are able to say what the body is made of and this in reasonable detail. And we are able to identify the ends to which the activity of bodies are directed. But what we do not know how to answer is the question of how something of this kind of material composition could have this kind of finality. Medieval philosophers were not sufficiently puzzled by this question, because they knew too little about the materials of which the human body is composed. Modern philosophers have not been sufficiently puzzled by this question, because, from La Mettrie to AI programs to the theorizingof philosophers recently engrossed by the findings of neurophysiology and biochemistry, they have tended to suppose that, if only we knew enough about the materials of which the body is composed, the problem of how we find application for teleological concepts would somehow be solved or disappear. But perhaps the time has now come when we should recognize that progress in understanding the material composition of human bodies has brought us no nearer and shows no sign of bringing us any nearer to an answer to this question. So where do we go from here? The point of this essay is to identify just where it is that we now are and by doing so to suggest that we need to begin all over again. (pp. 102-103)
Enough for one post. What, I hope, is becoming very evident is that the interconnectedness of MacIntyre's philosophy. For me the question becomes, in light of my support for his theory of enquiry, how far must I follow him, and consequently Aquinas and Aristotle, in their teleological metaphysics, which, as MacIntyre points out, were indispensable parts of their respective systems.

Saturday, August 12, 2006

Emulating Hilbert

Dennis Lomas has pointed out to me that various translations of Paul Bernays' writings are available on-line. Go to The Bernays Project and click on translations. Bernays is perhaps best known for his collaboration with David Hilbert in their studies of the foundations of mathematics. The paper Die Bedeutung Hilberts für die Philosophie der Mathematik (1922) gives an interesting snapshot of Bernays' views on the significance of Hilbert's work as philosophy, long before the shadow of Gödel fell over the programme. Not only is Hilbert's axiomatic method praised for its importance to mathematics, but at the end of the piece it is promoted as important to physics too, providing the simplest presentation of relativity theory, and pointing Hilbert to a way to unify this theory with electrodynamics, carried further by Weyl.

At the beginning of the article Bernays expresses his delight that mathematical thought had at least regained influence over philosophical speculation. I wonder what he would make of the current situation. The really curious thing is that so few of those philosophers who would want to emulate Hilbert have turned to category theory. Not only is it evidently important for the axiomatic formulation of mathematics, but it is looking very likely that it will play a critical role in whichever reconciliation of general relativity and quantum field theory wins out.

For some of the latest research in category theory, you can take a look at the slides from the CT2006 conference, including one by Makkai, whose radical idea is to remove equality from mathematics (further papers here).

Friday, August 11, 2006

Klein 2-Geometry IV

Can we sustain our momentum for the categorification of the Erlangen Program into its fourth month? At least now it is clear that what we need is a good account of how to quotient a 2-group by one its sub-2-groups. I've been messing around a little with some baby 2-groups and think I see how they work. I now think that the categorified Euclidean geometry that cropped up early on, i.e., the one that spoke of weak points and weak lines, arises from a discrete categorification of the Euclidean group. This has Euclidean transformations as 1-morphisms, and only trivial 2-morphisms. We may expect the geometry from more general 2-groups to look quite different.

Update: Things are hotting up. For the first time in my life (to my face at least) I've been called 'evil'. What can be achieved before the hiatus of a sojourn in France?

Monday, August 07, 2006

There is No Wealth but Life

A Victorian version of "They paved paradise and put up a parking lot" from Fors Clavigera: Letters to the Workmen and Labourers of Great Britain, written by Ruskin during the period 1871-1884:

You think it a great triumph to make the sun draw brown landscapes for you! That was also a discovery, and some day may be useful. But the sun had drawn landscapes before for you, not in brown, but in green, and blue, and all imaginable colours, here in England. Not one of you ever looked at them; not one of you cares for the loss of them, now, when you have shut the sun out with smoke, so that he can draw nothing more, except brown blots through a hole in a box. There was a rocky valley between Buxton and Bakewell, once upon a time, divine as the vale of Tempe; you might have seen the gods there morning and evening, - Apollo and all the sweet Muses of the Light, walking in fair procession on the lawns of it, and to and fro among the pinnacles of its crags. You cared neither for gods nor grass, but for cash (which you did not know the way to get). You thought you could get it by what the Times calls 'Railroad Enterprise.' You enterprised a railroad through the valley, you blasted its rocks away, heaped thousands of tons of shale into its lovely stream. The valley is gone, and the gods with it; and now, every fool in Buxton can be at Bakewell in half-an-hour, and every fool in Bakewell at Buxton; which you think a lucrative process of exchange, you Fools everywhere!"
In Praeterita III, he explains his sense of the word 'gods', and comments:

...and myself knowing for an indisputable fact, that no true happiness exists, nor is any good work ever done by human creatures, but in the sense or imagination of such presences. (p. 500)