Philosophy of Real Mathematics

New blog for my health book

noreply@blogger.com (david) — Mon, 19 Feb 2007 09:30:00 +0000

I have started a blog for 'Why Do People Get Ill?'

Migration

noreply@blogger.com (david) — Mon, 04 Sep 2006 10:46:00 +0000

I have decided to migrate to somewhere new in the blogosphere. It's the n-category cafe.

Hiatus

noreply@blogger.com (david) — Tue, 15 Aug 2006 15:47:00 +0000

The family holiday is upon me, taking me away from the blogosphere. I doubt I'll be able to tune in again before September. If you're not a spammer, feel free to add comments while I'm away.

Something to add to your bookmarks in the mean time is this blog by Alexandre Borovik. He's only written an introductory post so far, but without putting too much pressure on him, I expect Mathematics under the Microscope will prove very interesting. I've mentioned Borovik on my blog here (Nov 5 and 12) and here.

Ruminating

noreply@blogger.com (david) — Tue, 15 Aug 2006 15:46:00 +0000

If some blog posts record the results of the author's digesting some body of thought, what follows is some, at best, half-chewed reflections on my latest wanderings in machine learning.

First, something which seems inescapable if you're looking to impose a geometry on a statistic manifold is the Fisher information metric. Now, it appears that a good justification for this was given by Censov in 1982. Apparently, this is the only Riemannian metric invariant under congruent embeddings by a Markov morphism. What this amounts to is requiring that the effect of re-partitioning an event space on a probability distribution be sensible. I found this out from Guy Lebanon's very interesting thesis, where he extends the result to conditional spaces (chapter 6). These are useful for modelling the conditional distribution of output data on input data, rather than the joint distribution of this data. Campbell had already extended Censov's results to non-normalized positive measures, on the way dropping the category theoretic apparatus. (It's never too late to reintroduce it.)

Now the distances that fit neatly with the Fisher information metric are the δ-divergences (p. 5 of this), which include the Kullback and reverse Kullback divergences. This opens you to the glorious world of information geometry (see this list), convex optimization, Legendre transforms between δ-coordinates and (1 - δ)-coordinates, etc. The Zhu and Rohwer articles argue for the advantages of working within the space of all positive measures, rather than of normalized probability distributions, which is δ-flat for all δ, i.e., Christoffel symbols vanish.

All is going swimmingly, except that with some spaces of model you're interested in, like multi-layered neural nets and other graphical models, there's no one-one mapping between the model parameters and the space of distributions, which messes up the geometry in parameter space. Now, there was a trend to move away from neural nets, but they have never quite disappeared. Some, like Geoffrey Hinton, still hope that we can learn something about the brain from studying plausible neural net algorithms, see What kind of a Graphical Model is the Brain?, perhaps discovering some conceptual representations in the higher layers of a trained net.

This runs against the idea that we'd be better off simplifying our task by producing a machine which can merely discriminate between inputs, such as images of 4s and images of 8s, rather than a model which aims to generate the data. But Hinton claims to be able to produce more accurate generative models than the best discriminative classifiers.

A second trend, especially if you were a Bayesian neural net person, was to notice that in some kind of limit of the number of hidden nodes in a layer, what emerged was a Gaussian process. (For the life of me I can't see why information geometry hasn't invaded Gaussian process theory.)

Perhaps, then, layered models are worth sticking with. So is there anything we can do with the non-smooth mapping between parameter space and distribution space. Yes, we turn to algebraic geometry. First, we can follow Watanabe and use Hironaka's resolution of singularities. Second, we follow Pachter and Sturmfels, and say that

(a) Statistical models are algebraic varieties.
(b) Every algebraic variety can be tropicalized.
(c) Tropicalized statistical models are fundamental for parametric inference.

An easy example of (a), concerning a distribution of two binary variables, expresses the independence of these variables as requiring the distribution to satisfy an equation in R⁴, namely, p₀₀.p₁₁ - p₀₁.p₁₀ = 0. But what are the tropics doing here? Well tropical maths is what John Baez and I were discussing here, and Sturmfels has a gentle introduction here. I have a sneaking feeling it would be worth trying to understand whether the tropical/ordinary = Legendre/Laplace transform analogy has anything to do with the appearance of the Legendre transform earlier.

Well, I did say it was half-chewed.

MacIntyre and the state of philosophy

noreply@blogger.com (david) — Mon, 14 Aug 2006 00:53:00 +0000

Alasdair MacIntyre is concerned that philosophy has come to play such a minor role in modern society, specifically with regard to moral philosophy in Moral Philosophy and contemporary social practice: what holds them apart?, and more generally in Philosophy recalled to its tasks: a Thomistic reading of Fides et Ratio. (Both articles in The Tasks of Philosophy, CUP, 2006, references below from this book unless otherwise stated.) 'Fides et Ratio' is a papal encyclical which sees a central and autonomous role for philosophy in the search to better understand 'truth' and what kind of 'good' it constitutes in our lives. Here is one part of MacIntyre's diagnosis:

Philosophers do in fact become irrelevant to others not only by making their utterances inaccessible, but also by losing sight of the often complex and indirect connections between their own specialized, detailed and piecemeal enquiries and those larger questions which give point and purpose to the philosophical enterprise, which rescue it from being no more than a set of intellectually engaging puzzles. Part of what is needed to remedy this is to call to mind a third salient characteristic of philosophy identified in the encyclical, its systematic character. Philosophy does not consist of a set of independent and heterogeneous enquiries into distinct and unconnected problems: the characterization of space and time, the nature of the human good, the relationship of perceived qualities to the causes of perception, how referring expressions function, what standards govern aesthetic judgment, the nature of causality, and so on. For the answers that we give to each of these questions impose constraints upon what answers we can defensibly give to some at least of the others. And when from collaborative work in a number of areas the logical, conceptual, empirical, and metaphysical relationships between each of these sets of answers begin to emerge, we commonly find that we have at least an outline of a system, a system that will inescapably have implications for how the philosophical questions posed by plain persons are to be answered. We will have reached a point at which we are able to recognize the need for a comprehensive vision of the human good and of the order of things (30, 46). System-building however can itself degenerate into a form of philosophical vice against which the encyclical warns us (4). Philosophers who are aware of the systematic character of their enterprise may always fall in love with their own system to such an extent that they gloss over what they ought to recognize as intractable difficulties or unanswerable questions. Love of that particular system displaces the love of truth. If the vice of reducing philosophy to a set of piecemeal, apparently unconnected set of enquiries is the characteristic analytical vice, this vice of system-lovers may perhaps be called the idealist vice. (p. 181)

One would imagine, then, that MacIntyre would be pleased by efforts on the part of analytic philosophers to link virtue ethics to epistemology, see, e.g., here and here. After all, he's famed for his revival of the virtue-based ethics of Aristotle and Aquinas. However, epistemology is not the proper study of our quest for understanding,

For if the Thomist is faithful to the intentions of Aristotle and Aquinas, he or she will not be engaged, except perhaps incidently, in an epistemological enterprise...

The epistemological enterprise is by its nature a first-person project. How can I, so the epistemologist enquires, be assured that my beliefs, my perceptions, my judgments connect with reality external to them, so that I can have justified certitude regarding their error and truth? ...But the thomist, if he or she follows Aristotle or Aquinas, constructs an account both of approaches to and of the achievements of knowledge from a third-person point of view. My mind or rather my soul is only one among many and its knowledge of my self qua soul has to be integrated into the general account of souls and their teleology. Insofar as a given soul moves successfully towards its successive intellectual goals in a teleologically ordered way, it moves towards completing itself by becoming formally identical with the objects of its knowledge, so that it is adequate to those objects, objects that are then no longer external to it, but rather complete it. (pp. 148-149)

It seems that I should be reading Jonathan Kvanvig as a virtue epistemologist who explores the social and genetic aspects of enquiry.

Now, this linking of what others might consider disjoint areas of philosophy continues. Part and parcel of MacIntyre's position, is the inextricable unity of ethics and politics. If ethics is being related to a theory of enquiry, then so must politics. And this should hardly surprise us given what I mentioned before about MacIntyre learning from philosophers of science such as Kuhn, Lakatos, Popper and Feyerabend. What is very striking about these philosophers is how they understand aspects of science in political terms.

This now raises a further issue. In The Essential Contestability of Some Social Concepts, Ethics 84(1) 1-9, 1973 (available on JSTOR), MacIntyre remarks:

Consider...the continuing argument between Kuhn, Lakatos, Polanyi, and Feyerbend, an argument in which what is at stake includes both our ability to draw a line between authentic sciences and degenerative or imitative sciences, such as astrology or phrenology, and our ability to explain why "German physics" and Lysenko biology are not to be included in science. A crucial feature of these arguments is the way in which dispute over the norms which govern scientific practice interlocks with debate over how the history of science is to be written. What identity and continuity are recognized will of course depend on what side is taken in these latter debates but since these debates are so intimately related to the arguments about the norms governing practice, it turns out that the dispute over norms and the dispute over continuity and identity cannot be separated. (p. 7)

A theory of intellectual enquiry must, then, include a theory of the writing of the narrative history of a tradition of enquiry.

A particular way of writing the history of science, the history of philosophy and intellectual history in general willbe the counterpart of a Thomistic conception of rational enquiry, and insofar as that history makes the course of actual enquiry more intelligible than do rival conceptions, the Thomistic conception will have been further vindicated. (167-168)

Of every particular enquiry there is a narrative to be written, and being able to understand that enquiry is inseperable from being able to understand that enquiry is inseperable from being able to identify and follow that narrative. (p. 168)

...from an Aristotelian standpoint it is only in the context of a particularly socially organized and morally informed way of conducting enquiry that the central concepts crucial to a view of enquiry as truth-seeking , engaged in rational justification, and realistic in its selfunderstanding, can intelligibly be put to work. (p. 169)

For some reflections on writing such a history for mathematics, see here and here.

So we have ethics, politics, philosophy of history and the theory of enquiry inextricably linked. But then,

...we need to learn from Aquinas that any such account of truth is incomplete, and therefore more questionable than it needs to be, until it is situated within a larger teleological view of human nature, according to which truth, understood as adaequatio, is also understood as constitutive of the human good. (p. 215)

If we follow MacIntyre, we can hardly avoid, then, encountering debates in the philosophy of mind concerning the relationship between the physical workings of a body and its directed activity. How far are we here?

We are able to say what the body is made of and this in reasonable detail. And we are able to identify the ends to which the activity of bodies are directed. But what we do not know how to answer is the question of how something of this kind of material composition could have this kind of finality. Medieval philosophers were not sufficiently puzzled by this question, because they knew too little about the materials of which the human body is composed. Modern philosophers have not been sufficiently puzzled by this question, because, from La Mettrie to AI programs to the theorizingof philosophers recently engrossed by the findings of neurophysiology and biochemistry, they have tended to suppose that, if only we knew enough about the materials of which the body is composed, the problem of how we find application for teleological concepts would somehow be solved or disappear. But perhaps the time has now come when we should recognize that progress in understanding the material composition of human bodies has brought us no nearer and shows no sign of bringing us any nearer to an answer to this question. So where do we go from here? The point of this essay is to identify just where it is that we now are and by doing so to suggest that we need to begin all over again. (pp. 102-103)

Enough for one post. What, I hope, is becoming very evident is that the interconnectedness of MacIntyre's philosophy. For me the question becomes, in light of my support for his theory of enquiry, how far must I follow him, and consequently Aquinas and Aristotle, in their teleological metaphysics, which, as MacIntyre points out, were indispensable parts of their respective systems.

Emulating Hilbert

noreply@blogger.com (david) — Sat, 12 Aug 2006 21:12:00 +0000

Dennis Lomas has pointed out to me that various translations of Paul Bernays' writings are available on-line. Go to The Bernays Project and click on translations. Bernays is perhaps best known for his collaboration with David Hilbert in their studies of the foundations of mathematics. The paper Die Bedeutung Hilberts für die Philosophie der Mathematik (1922) gives an interesting snapshot of Bernays' views on the significance of Hilbert's work as philosophy, long before the shadow of Gödel fell over the programme. Not only is Hilbert's axiomatic method praised for its importance to mathematics, but at the end of the piece it is promoted as important to physics too, providing the simplest presentation of relativity theory, and pointing Hilbert to a way to unify this theory with electrodynamics, carried further by Weyl.

At the beginning of the article Bernays expresses his delight that mathematical thought had at least regained influence over philosophical speculation. I wonder what he would make of the current situation. The really curious thing is that so few of those philosophers who would want to emulate Hilbert have turned to category theory. Not only is it evidently important for the axiomatic formulation of mathematics, but it is looking very likely that it will play a critical role in whichever reconciliation of general relativity and quantum field theory wins out.

For some of the latest research in category theory, you can take a look at the slides from the CT2006 conference, including one by Makkai, whose radical idea is to remove equality from mathematics (further papers here).

Klein 2-Geometry IV

noreply@blogger.com (david) — Fri, 11 Aug 2006 10:15:00 +0000

Can we sustain our momentum for the categorification of the Erlangen Program into its fourth month? At least now it is clear that what we need is a good account of how to quotient a 2-group by one its sub-2-groups. I've been messing around a little with some baby 2-groups and think I see how they work. I now think that the categorified Euclidean geometry that cropped up early on, i.e., the one that spoke of weak points and weak lines, arises from a discrete categorification of the Euclidean group. This has Euclidean transformations as 1-morphisms, and only trivial 2-morphisms. We may expect the geometry from more general 2-groups to look quite different.

Update: Things are hotting up. For the first time in my life (to my face at least) I've been called 'evil'. What can be achieved before the hiatus of a sojourn in France?

There is No Wealth but Life

noreply@blogger.com (david) — Mon, 07 Aug 2006 15:06:00 +0000

A Victorian version of "They paved paradise and put up a parking lot" from Fors Clavigera: Letters to the Workmen and Labourers of Great Britain, written by Ruskin during the period 1871-1884:

You think it a great triumph to make the sun draw brown landscapes for you! That was also a discovery, and some day may be useful. But the sun had drawn landscapes before for you, not in brown, but in green, and blue, and all imaginable colours, here in England. Not one of you ever looked at them; not one of you cares for the loss of them, now, when you have shut the sun out with smoke, so that he can draw nothing more, except brown blots through a hole in a box. There was a rocky valley between Buxton and Bakewell, once upon a time, divine as the vale of Tempe; you might have seen the gods there morning and evening, - Apollo and all the sweet Muses of the Light, walking in fair procession on the lawns of it, and to and fro among the pinnacles of its crags. You cared neither for gods nor grass, but for cash (which you did not know the way to get). You thought you could get it by what the Times calls 'Railroad Enterprise.' You enterprised a railroad through the valley, you blasted its rocks away, heaped thousands of tons of shale into its lovely stream. The valley is gone, and the gods with it; and now, every fool in Buxton can be at Bakewell in half-an-hour, and every fool in Bakewell at Buxton; which you think a lucrative process of exchange, you Fools everywhere!"

In Praeterita III, he explains his sense of the word 'gods', and comments:

...and myself knowing for an indisputable fact, that no true happiness exists, nor is any good work ever done by human creatures, but in the sense or imagination of such presences. (p. 500)

Summer reading

noreply@blogger.com (david) — Thu, 03 Aug 2006 18:58:00 +0000

A new blog is born - Modulo Truth - to a Harvard student John Cobb. Although still only an undergraduate he is already reading proofs that "left invariant vector fields always generate a global flow on the Lie group", while rereading Alasdair MacIntyre's After Virtue. If only I had advanced so far down my own path at his age.

Perhaps he would enjoy my Summer reading, which arrived in the post today - MacIntyre's The Tasks of Philosophy, a collection of essays published by Cambridge University Press. Now, I already described how I admired the connectivity of MacIntyre's philosophy in this post:

This is the kind of connectivity that interests me. Not that science through its theories – genetics, cosmology, etc. – has a bearing on philosophical theses, but that a moral philosopher may learn from philosophers of science, as may a philosopher of mathematics from moral philosophers.

(See also this post.) I suspected that he had learned a very important lesson from Kuhn, Lakatos, Feyerabend and others, but now it is official. In the Preface to the collection we read about how the first essay, which marked a "major turning-point" in his thinking in the 1970s,

...was elicited by my reading of and encounters with Imre Lakatos and Thomas Kuhn and what was transformed by that reading was my conception of what it was to make progress in philosophy or indeed in systematic thought more generally. Up to that time, although I should have learned otherwise from the histories of Christian theology and of Marxism, I had assumed that my enquiries would and should move forward in a piecemeal way, focusing first on this problem and then on that, in a mode characteristic of much analytic philosophy. So I had worked away at a number of issues that I had treated as separate and distinct without sufficient reflection upon the larger conceptual framework within which and by reference to which I and others formulated those issues. What I learned from Kuhn, or rather from Kuhn and Lakatos read together, was the need first to identify and then to break free from that framework and to enquire whether the various problems on which I had made so little progress had baffled me not or not only because of their difficulty, but because they were bound to remain intractable so long as they were understood in the terms dictated by those larger assumptions which I shared with many of my contemporaries. And I was to find that, by rejecting the conception of progress in philosophy that I had hitherto taken for granted, I had already taken a first step towards viewing the issues in which I was entangled in a new light. (vii-viii)

Perhaps, once I have worked through all of the essays, I'll be in a position to finish this essay off.

A tale from the book of mathematics

noreply@blogger.com (david) — Tue, 01 Aug 2006 16:25:00 +0000

Back in this post I discussed the importance of the story-like aspect of mathematics. Here's a simply told tale of the classification of Three-manifolds by Shing-Tung Yau:

Ladies and gentlemen, Today I am going to tell you the story of how a chapter of mathematics has been closed and a new chapter is beginning.

Feeling the master's superiority

noreply@blogger.com (david) — Mon, 31 Jul 2006 03:22:00 +0000

There can be but two reasons for a philosopher to spend time, as I have, rethinking fundamental concepts of geometry. The first of these derives from the thesis that philosophy may legitimately contribute to the formulation of certain fundamental concepts, which include those of geometry. The second derives from the Collingwoodian thesis that to do the philosophy of a discipline well one must be 'thoroughly at home' with it. As for the first, the list of such legitimate concepts, as conceived by contemporary analytic philosophy, includes causation, probability, necessity, time, consciousness, identity, number, set, and yet would appear to exclude mathematical concepts such as (mathematical) space, point, dimension. I have no objection to the drawing of a distinction - it is hardly within the philosopher's brief to explore the nature of being igneous or of being crystalline - , but I have yet to see a principled way of separating the legitimate from the illegitimate. Let me, here, pursue the second reason.

As I observe in my book,

For Collingwood, ... a capacity to experience the force of the absolute presuppositions of the contemporary form of the discipline about which one is philosophising is vital. While describing which qualities someone should possess to be able to answer the questions of philosophy of history, he remarks acidly that:
No one, for example, is likely to answer them worse than an Oxford philosopher, who, having read Greats in his youth, was once a student of history and thinks that this youthful experience of historical thinking entitles him to say what history is, what it is about, how it proceeds, and what it is for. (Collingwood 1946: 8)
A similar conclusion could be formulated for philosophy of mathematics, and indeed Kant is praised for dealing with the presuppositions of mathematics 'rather briefly' for 'he was not very much of a mathematician; and no philosopher can acquit himself with credit in philosophizing at length about a region of experience in which he is not very thoroughly at home' (Collingwood 1940: 240). Returning to history, he continues:
An historian who has never worked much at philosophy will probably answer our four questions in a more intelligent and valuable way than a philosopher who has never worked much at history. (Collingwood 1946: 9)
Evidence for the equivalent statement about mathematics is provided by the very many important contributions made by mathematicians thinking about their discipline, several of which I shall lean on in the course of this book. (pp. 17-18)

This Collingwood, Robin George, Waynflete Professor of Metaphysics at Oxford, was the son of William Gershom Collingwood, who worked with John Ruskin at Brantwood, the latter's home on the shores of Coniston Water. There could have been little direct personal influence on the young Collingwood, Ruskin dying in 1900 shortly before he was 11, and suffering greatly from mental illness in his final years, but most likely his father, who completed a biography of Ruskin in 1893, provided the necessary immersion in Ruskinian principles.

In earlier posts (2 May & 5 May), I mentioned Ruskin's Unto This Last. I am currently reading Ruskin's autobiography Praeterita, where we read:

"Mostly a quiet stream there, through the bogs, with only a bit of step or tumble a foot or two high on occasion; above which I was able practically to ascertain for myself the exact power of level water in a current at the top of a fall. I need not say that on the Cumberland and Swiss lakes, and within and without the Lido, I had learned by this time how to manage a boat - an extremely different thing, be it observed, from steering one in a race; and the little two-foot steps of Tummel were, for scientific purposes, as good as falls twenty or two hundred feet high. I found that I could put the stern of my boat full six inches into the air over the top of one of these little falls, and hold it there, with very short sculls, against the level [Distinguish carefully between this and a sloping rapid.] stream, with perfect ease for any time I liked; and any child of ten years old may do the same. The nonsense written about the terror of feeling streams quicken as they approach a mill weir is in a high degree dangerous, in making giddy water-parties lose their presence of mind if any such chance take them unawares. And (to get this needful bit of brag, and others connected with it, out of the way at once), I have to say that half my power of ascertaining facts of any kind connected with the arts, is in my stern habit of doing the thing with my own hands till I know its difficulty; and though I have no time nor wish to acquire showy skill in anything, I make myself clear as to what the skill means, and is. Thus, when I had to direct road-making at Oxford, I sate, myself,with an iron-masked stone-breaker, on his heap, to break stones beside the London road, just under Iffley Hill, till I knew how to advise my too impetuous pupils to effect their purposes in that matter, instead of breaking the heads of their hammers off, (a serious item in our daily expenses). I learned from an Irish street crossing-sweeper what he could teach me of sweeping; but found myself in that matter nearly his match, from my boy-gardening; and again and again I swept bits of St Giles' foot-pavements, showing my corps of subordinates how to finish into depths of gutter. I worked with a carpenter until I could take an even shaving six feet long off a board; and painted enough with properly and delightfully soppy green paint to feel the master's superiority in the use of a blunt brush."

How much more important for Ruskin, then, to devote years to drawing, so as to be able to write intelligently about art.

Gambling on the Riemann Hypothesis

noreply@blogger.com (david) — Sun, 30 Jul 2006 16:16:00 +0000

I discussed the idea of Bayesianism and gambling in mathematics back on 3 November. Now, from sigpe, I see that you can trade in futures for mathematical results such as the Riemann Hypothesis. Of course, to put a finite time limit on a mathematical gamble, it has to be of the form, 'By 20XX, the Y conjecture will have been proved.' In the case of the Riemann hypothesis, it's 2020.

But, clearly your degree of belief in RH's being proved by 2020 is upper bounded by your degree of belief in it's truth. Put the other way around, if you happened to believe that it is 70% likely that RH is false, you would be very happy to sell options at its current price. This price has fluctuated significantly over the decade of the future's existence. Presumably most of the fluctuation comes from evidence that someone is closing in on a proof. It would be interesting to see how, say, analogical evidence for or against its truth, such as Deninger provides, played out.

Information Geometry and Machine Learning

noreply@blogger.com (david) — Wed, 26 Jul 2006 11:32:00 +0000

I'm drawing up a list of papers which formulate machine learning algorithms as maximum entropy or minimum relative entropy solutions, or more broadly are written in the general framework of information geometry. The list isn't aiming for completeness, rather coverage. If anyone knows of any obvious omissions, I'd be grateful to hear.

I'm interested in the moment by the topic at the end of the list - Bayesian Information Geometry, which might just be what a huge number of machine learing algorithms are approximating. The idea is simple enough. Starting out form a prior distribution in the space of distributions, for any data the decision rule minimises the divergence between the true distribution and its estimate. For a given data set this is equivalent to finding the distribution at the smallest mean distance from the true distribution, the mean being taken with respect to the posterior distribution. Snoussi's paper shows how broad this framework is, with the flexibility to choose your distance function and the weight you give to your choice of prior.

At a sociological level, it's interesting to speculate why information geometry has been somewhat reluctantly taken up. I'm sure a large part of this is due to there being no straightforward introduction to the subject. Someone should write an exposition of its key successes without the usual huge dollop of differential geometry in the opening section.

Something I'm also curious to know is why there's not greater use of information geometry by the Gaussian process machine learning theorists. With Gaussian processes as maximum entropy solutions, you'd have thought they'd be tailor-made for the IG treatment, perhaps even to help in the choice of covariance function.

Philosophy's foreign relations

noreply@blogger.com (david) — Mon, 24 Jul 2006 13:20:00 +0000

Jon Williamson has just paid me a visit here in Tubingen. Jon and I go back quite a way to the time when we were engaged on parallel projects, studying the interaction betweeen AI and philosophy, at King's College London. We hope to work together in the future on a topic I've addressed in recent posts, namely, maximum entropy, perhaps in the context of information geometry. Over a wheat beer, we mulled over our thoughts on the proper relationship between philosophy and neighbouring disciplines. While I doubt Jon would wholeheartedly support my historical stance, we both are of the opinion that it is important for philosophers to leave their comfort zone periodically to engage with these disciplines.

A good way to catch a glimpse of how practitioners of these disciplines view philosophy comes in the introductory spiel to their invited contributions to philosophical collections. A case in point is the Handbook on the Philosophy of Information. Here are two such views:

The philosophy of X, where X is a science, often involves philosophers analyzing the concepts of X and commenting on what concepts are or are not likely to be coherent. AI necessarily shares many concepts with philosophy, e.g. action, consciousness, epistemology (what it is sensible to say about the world), and even free will. This article treats the philosophy of AI but also reverses the usual course and analyzes some basic concepts of philosophy from the standpoint of AI. The philosophy of X often involves advice to practioners of X about what they can and cannot do. We reverse the usual course and offer advice to philosopers, especially philosophers of mind. The point is that philosophical theories can make sense only if they don’t preclude human-level artificial systems, and this fact has further consequences.
Information in Artificial Intelligence by J. McCarthy

Philosophers of science are concerned with explaining various aspects of science, and often, moreover, with viewing science as a kind of gold-mine of philosophical opportunity. The direction in both cases is philosophy from science. For a theoretical scientist, the primary inclination is often to see conceptual analysis as a preliminary to a more technical investigation, which may lead to a new theoretical development. In short: science from philosophy. This article is written mainly in the latter spirit, from the stand-point of Theoretical Computer Science, or perhaps more broadly “Theoretical Informatics”: a — still largely putative — general science of information. That being said, we hope that our conceptual discussions may also provide some useful grist to the philosopher’s mill.
Information, Processes and Games by Samson Abramsky

Both, then, are seemingly open to dialogue with philosophy. Of course, we should not underestimate the preparation necessary to be able to engage fruitfully with practioners. The main danger of their breaking off contact comes from their perception that you have a peculiarly lop-sided view of their field, driven by some quixotic philosophical position.

Renyi entropy

noreply@blogger.com (david) — Thu, 20 Jul 2006 10:23:00 +0000

Following the discussion we had here about the merits of Tsallis and Renyi entropies, here's an interesting paper by Peter Harremöes - Interpretations of Renyi Entropies And Divergences. Harremöes is looking for an information theoretic interpretation of the Renyi entropies in terms of what he calls an operational definition:

To us an operational definition of a quantity means that the quantity is the natural way to answer a natural question and that the quantity can be estimated by feasible measurements combined with a reasonable number of computations. In this sense the Shannon entropy has an operational definition as a compression rate and the Kolmogorov entropy has an operational definition as shortest program describing data. (p. 2)

Via an introductory account of codes, we learn that "the Renyi divergence measures how much a probabilistic mixture of two codes can be compressed".

Like the Kullback-Leibler divergence (Shannon relative entropy), the Renyi divergence is addititive or extensive in the sense that

D_q(P₁ x P₂//Q₁ x Q₂) = D_q(P₁//Q₁) + D_q(P₂//Q₂).

[KL-divergence equals the Renyi divergence for q = 1.] So too is the corresponding Renyi entropy. But for q > 1 it lacks a property possessed by the Shannon entropy, and also by all Renyi entropies with q in [0,1], namely concavity. The Tsallis entropy chooses the other option, and so while concave for q > 1, it is no longer additive/extensive.

Klein 2-Geometry III

noreply@blogger.com (david) — Wed, 19 Jul 2006 08:37:00 +0000

Update: I'm floating this post to the top again so that we don't lose it.

Time to begin the new month's posting on categorified geometry, continuing May and June. Fortunately John Baez, although now in Shanghai, is on broadband. I wouldn't fancy solo Kleincategorification (second and final comments). It's like when you're learning to ski, you can manage much trickier slopes with an expert to follow.

I guess the biggest worry in a venture of this kind is that all you achieve is a repackaging of what's already known. There's a discussion here, involving John, about whether Lie 2-algebras bring into the light anything new (cf. posts 9, 13 and 14). (The archives of sci.physics.research is full of delights. Here's another thread on 2-groups.)

In this discussion, John mentions his reasons for quitting his role as moderator of sci.physics.research. I'm not sure I've characterised all that well there what it is I'm looking for beyond individuals exposing their ideas in a free and informal way. John can see I want something a little more agonistic, but fears the tendency towards antagonism. I see agonism is a political position. One of its advocates has this to say:

Agonism implies a deep respect and concern for the other; indeed, the Greek agon refers most directly to an athletic contest oriented not merely toward victory or defeat, but emphasizing the importance of the struggle itself-a struggle that cannot exist without the opponent. Victory through forfeit or default, or over an unworthy opponent, comes up short compared to a defeat at the hands of a worthy opponent-a defeat that still brings honor. An agonistic discourse will therefore be one marked not merely by conflict but just as importantly, by mutual admiration. (Samuel Chambers)

Bloggers of the world, forego antagonism, choose agonism.

Of course, you may be able to internalise the agon, by taking also the part of the opponent. Indeed, this is how I arrived at the idea behind this series on the categorification of Kleinian geometry. I imagined what someone highly dubious about the scope of worthwhile categorification might say. "Let us for the moment accept that the 'categorification' (such an ugly name) of arithmetic and combinatorial identities via groupoids and species has been worthwhile, what do you have to say about Euclidean geometry, the jewel of Greek mathematics. If you have nothing new to tell me about points, lines and circles, I shall remain unconvinced."

We've hardly begun

noreply@blogger.com (david) — Tue, 18 Jul 2006 19:21:00 +0000

From Barry Mazur's Foreword to Fearless Symmetry: Exposing the Hidden Patterns of Numbers by Avner Ash & Robert Gross (to be published by Princeton University Press):

At some point in his or her life every working mathematician has to explain to someone, usually a relative, that mathematics is hardly a finished project. The mathematicians know, of course, that it is far too early to put the glorious achievements of their trade into a big museum and become happy curators. Our subject has, in certain respects, hardly begun. But, at least in the past, this seems not to have been universally acknowledged.

Yes, forget the big museum, it's the seed bank we need.

Conceptual essentialism

noreply@blogger.com (david) — Fri, 14 Jul 2006 00:39:00 +0000

John Baez recently added a comment to this post, which is too old now for comments to appear in 'recent comments'. I had remarked that something he had said earlier sounded like it came straight from the Jaffe-Quinn debate. For those of you who don't remember it, these two mathematical physicists launched a passionate attack on slipping standards in mathematics, brought about by an imitation of the sloppier ways of physicists. Many very interesting responses were made, not least William Thurston's wonderful On proof and progress in mathematics.

Anyway, John replied:

I hope it's clear that I'm not complaining about the lack of rigor. I'm complaining about a swarm of people writing hundreds of short papers on the same subject in a short time, each referring to many of the previous ones, nobody taking the time to distill the matter to its essence. Even if all the papers contained nothing but rigorous theorems, I would still find this annoying. It's fine if you wish to devote yourself to one specialized subject, rapidly master the literature, and compete with the crowd to extract some big nuggets before this vein of ore looks exhausted and it's time to move on. I'm sure this is fun for people with a competitive streak. But there are other people who like to slowly mull over one topic and nurse it to perfection - or like me, mull over lots of topics and gradually form a web of connections until something interesting emerges. And, you know, it's just possible that some of the people in that Jaffe-Quinn dispute were secretly annoyed about the fast-paced "swarming" style of theoretical physics more than any lack of rigor. I forget if any of them came out and said this.

I think the phrase 'nobody taking the time to distill the matter to its essence' is the key one here. Remember, two posts ago we had Borovik saying "The work of three generations of mathematicians confirmed that matroids, indeed, capture the essence of linear dependence" (my emphasis).

I've done my damnedest to get the idea of mathematical activity at its highest level aiming to extract the essence of a situation to be the principal topic of philosophy of mathematics, but with little success. It's not that I'm the only philosopher thinking about such things. For instance, Kenny Easwaran posted Do Mathematical Concepts Have Essences? on his blog, where you can follow up the reference to a paper I wrote on the subject. But it never stays on the agenda for long.

I think what is needed is a name. Essentialism is overused. Conceptualism is also already taken. It concerns the kind of problem faced when wondering what the tallness is shared by, say, a 2 metre man, a 30 metre tree, and a 300 metre building. As this paper explains:

Conceptualism, along with nominalism and realism, is one of three traditional families of views about universals. There are many species of each family, but the story line goes like this. Realists hold that there are universal properties and that these solve the problems of universals. Conceptualists deny this, arguing that concepts can do most of the work realists invoke properties to do. And nominalists, at least traditional ones, spurn both universals and concepts, arguing that words alone can do all the legitimate aspects of this work.

Blending the two, conceptual essentialism has been used in philosophy of science to designate a similar position. But is it snappy enough? How much of Kuhn's success was down to his choice of the word revolution?

The prevalence of Kullback-Leibler

noreply@blogger.com (david) — Tue, 11 Jul 2006 11:52:00 +0000

How's this for an explanation of the prevalence of the Kullback-Leibler divergence:

Much statistical inference takes the form of finding an optimal distribution satisfying some set of constraints. Very often these constraints are such that for any two distributions, P and Q, satisfying them, so do all mixtures of the form bP + (1 - b)Q. This is what Amari calls m-flatness (m for mixture), i.e., these paths of mixtures are geodesics with respect to the m-connection. Now, the dual affine connection to the m-connection is the e-connection (e for exponential), and e-flat manifolds of distributions are the ubiquitous exponential families. (To see e-flatness is not the same as m-flatness, consider that mixtures of Gaussians are not generally Gaussian.) Minimizing the relative KL-entropy of distributions satisfying the constraint is equivalent to finding where the exponential family meets the space of constrained distributions.

So my question is whether presenting the m-flatness idea first, as arising out of common-or-garden linear constraints, such as fixing the values of moments, is a good way to motivate the KL-divergence. Then it would be interesting to think about other types of constraints which would lead to flatness with other connections, and what the generalized exponential families, flat according to the dual affine connection, would look like.

Discrimination against -oids

noreply@blogger.com (david) — Mon, 10 Jul 2006 13:38:00 +0000

From Not Even Wrong, I see that the Institut Henri Poincare in Paris is holding a 3 month program on Groupoids and Stacks in Physics and Geometry. They have included an interesting overview of the subject to motivate the program. I have a particular soft spot for groupoids having studied the case for their admission into the paradise of mathematics in chapter 9 of my book. Groupoids had a strangely difficult childhood, finding acceptance surprisingly late, the oddest explanation for which is Connes' claim that:

...it is fashionable among mathematicians to despise groupoids and to consider that only groups have authentic mathematical status, probably because of the pejorative suffix 'oid'. (Noncommutative Geometry, 6-7)

Rather than this persecution of suffixes, a more common sentiment is that they are really just dressed up groups. In an old e-mail I have from Saunders Mac Lane he adopts just this line, perhaps surprisingly for the co-inventor of category theory, groupoids being categories with invertible morphisms, and very present in Mac Lane's home of algebraic topology.

Someone who had read my book made the good suggestion that I subject matroids to a similar treatment. Now I hear (penultimate comment by Srandby) that Mac Lane didn't like these either.

Once, Mac Lane came to give a talk. During the talk, in front of a packed audience, he stated that matroid theory wasn’t good or important mathematics, pissing off several faculty who worked in matroid theory. I found this comment to be very bizarre. Here was an advocate of a vast generalization of dubious importance dismissing a generalization of vector spaces that has tremendous importance.

Is there something to Connes' anti-oid theory?

Anyone who wants to take up the challenge of assessing matroids should take a look at Coxeter Theory: The Cognitive Aspects, an article by Alexandre Borovik. In section 13 - Combinatorics as non-parametric mathematics - Borovik claims:

The work of three generations of mathematicians confirmed that matroids, indeed, capture the essence of linear dependence. Since linear dependence is a ubiquitous and really basic concept of mathematics, it is not surprising that the concept of matroid has proven to be one of the most pervasive and versatile in modern combinatorics. (p. 23)

This book with Gelfand should no doubt be consulted too: Coxeter Matroids, Birkhauser, xiv+264 pp., ISBN 0-8176-3764-8 (with I. M. Gelfand and N. White), 2003.

I believe that Borovik will include the Coxeter Theory article in a book to appear with Springer. I read a draft of this book a couple of years back and found it wonderfully rich.

Conditionalization as I-projection

noreply@blogger.com (david) — Thu, 06 Jul 2006 11:19:00 +0000

First Greenspan,

“In essence, the risk management approach to monetary policy-making is an application of Bayesian decision-making.” (p. 37)
“Our problem is not, as is sometimes alleged, the complexity of our policy-making process, but the far greater complexity of a world economy whose underlying linkages appear to be continuously evolving. Our response to that continuous evolution has been disciplined by the Bayesian type decision-making in which we have engaged.” (p. 39)

“Risk and Uncertainty in Monetary Policy,” American Economic Review, May, 2004, 33-40.

Now it's the turn of the US Food and Drug Administration to come out in favour of Bayesianism.

Thanks to Yet Another Machine Learning Blog for this. An earlier post - Maximum entropy and bayesian updating - on this interesting blog presents the following example from Kass of a possible clash between maximising entropy and conditionalization:

Consider a Die (6 sides), consider prior knowledge E[X]=3.5.

Maximum entropy leads to P(X)= (1/6, 1/6, 1/6, 1/6, 1/6, 1/6).

Now consider a new piece of evidence A="X is an odd number"

Bayesian posterior P(X/A)= P(A/X) P(X) = (1/3, 0, 1/3, 0, 1/3, 0).

But MaxEnt with the constraints E[X]=3.5 and E[Indicator function of A]=1 leads to (.22, 0, .32, 0, .47, 0) !! (note that E[Indicator function of A]=P(A))

Indeed, for MaxEnt, because there is no more '6', big numbers must be more probable to ensure an average of 3.5. For bayesian updating, P(X/A) doesn’t have to have a 3.5 expectation. P(X) and P(X/A) are different distributions. Conclusion ? MaxEnt and bayesian updating are two different principles leading to different belief distributions. Am I right ?

Example 3 on p. 4 of Information topologies with applications by Peter Harremoes provides the answer here. Passing from a distribution P(X) to P(X/A) is just one simple case of a general process of projection from a point to a subspace of a space of distributions. Let P(X) be a distribution and A an event such that P(A)>0. Let C(A) be the set of distibutions Q, with Q(A) = 1. Then P(./A) is the closest element of C(A) to P in the sense of Kullback-Leibler distance (relative entropy). It is a robust bayes act to update thus.

More technically, C(A) is 'm-flat' in the sense of Amari, i.e., if Q and R are in C(A) then so is b.Q + (1 - b).R. The projection of P onto C(A) along the dual e-connection is P(./A). Forming the conditional distribution is but one small example of Csiszar's I-projection, which may use divergences other than the Kullback-Leibler.

Back to Kass' example, the MaxEnt formulation is projecting to the manifold of distributions satisfying both of the constraints, rather than just one as in the case of conditionalization.

Links

noreply@blogger.com (david) — Fri, 30 Jun 2006 07:41:00 +0000

Over at Ars Mathematica there's a discussion about the merits of category theory. I mentioned in a comment there this site of preprints, of which my favourites are Lawvere's 1 and 8. I most enjoy the way category theory suggests that you transcribe pieces of reasoning into different 'keys', sometimes just to recover something you already knew, but hadn't viewed in this way, preferably to perform a new piece.

I'd like to use this kind of thinking in coming to understand information geometry. I should be able to learn from a series of papers by Chris Hillman, especially the ones on entropy and information. Another idea worth exploring is one which says that probability and optimization are in some sense dual. This is related to what I have posted about tropical or idempotent mathematics.

Dawid on probabilities

noreply@blogger.com (david) — Sun, 25 Jun 2006 18:46:00 +0000

A few days ago our reading group ran through Phil Dawid's Probability, Causality and the Empirical World: A Bayes-de Finetti-Popper-Borel Synthesis and learned that not only does probability not exist for him, like for de Finetti, nor does causality. A common characterisation of positions concerning probabilities split them into four groups:

1) Frequentist: probabilities are limiting frequencies of outcomes in sequences of events.
2) Propensity theorist: an individual event has a propensity to display a certain outcome.
3) Subjective Bayesian: a subjective degree of belief in an outcome.
4) Objective Bayesian: relative to specific background knowledge, there is an objective value an agent should accord to their degree of belief in an outcome.

Dawid (pronounced 'David') holds a Bayesian position, made evident in his involvement in the Sally Clark case, in which a mother was unjustly jailed after her two children died. But for Dawid, while the position that probabilities are consistent assignments of degrees of belief is all well and good, and it avoids the problems of taking probabilities to exist out there in the world, at some point you should want to calibrate the probabilities that an individual is spewing out. I could lock myself in a dark room keeping my degrees of belief consistent, but if I have England as 99% likely to win the World Cup, and other oddities, you will want to have a framework in which you can criticise me. This idea of calibration takes place in weather forecasting. You might score the forecaster's rain predicitions by forming the sum of (x_i - y_i)², where x_i = 0 if it is dry on the i^th day, and 1 if it rains, and y_i is the forecasted percentage chance of rain - the Brier score.

Intuitively, if a forecaster believes in their forecasts for rain they ought to be happy to accept bets made either for or against it raining at the odds they give. You'd think then that we could winkle out the bad from the good forecaster by noting which ones would go broke within a short space of time. The trouble is in telling when a 'good' forecaster's being very unlucky, and when a 'bad' forecaster's being very lucky. You are forced to appeal to an infinitely long series of predictions. And this is Dawid's position. Probabilities don't exist, they're theoretical tools that mediate between our theories and the world. The only way they hook onto the world is by what they rule out as impossible (this is the Borel part of the synthesis). For example, among other things, a forecaster who says 30% chance of rain every day is ruling out the possibility that in the long run it will rain on 40% of the days. This has the paradoxical consequence that if, say, two weather forecasters make predictions for rain over the next century and agree on every day except tomorrow when one says 10% and the other say 90%, there is no way you can say whether one was right. They've both ruled out various sets of sequences of outcomes, like those for which the average differs from the limit of the average of their probabilities. But neither rules out anything that the other doesn't.

Leaving aside the problem that runs aren't infinite, this game theoretic interpretation of probability theory is certainly very interesting. Shafer and Vovk have written a book-length treatment of the idea in their Probability and Finance: It's Only a Game!. Game theoretic ideas are also used to understand maximum entropy distributions. They correspond to stable points in zero-sum games played between a decision maker and nature. Flemming Topsoe has many good papers on this, and there's also one by Dawid with Peter Grunwald. Different entropies match up with different loss functions, such as the Brier score above. Something to add to the pot is Dawid's The geometry of proper scoring rules, a longer version of a paper written with Steffen Lauritzen. Now we have game theory blending with the differential geometric approach of Amari known as Information Geometry, discussed in earlier posts. I wish I could understand all this properly.

Old correspondence

noreply@blogger.com (david) — Thu, 22 Jun 2006 11:18:00 +0000

My old laptop returned to life today. It had refused to boot up a while ago, and so was left to gather dust on a shelf. While I was passing this morning I gave its ON button a small tap and with that it sprang into action. So I've had the chance to look into my correspondence of the period 1999-2001. It's intriguing to see nascent ideas, some since turned into papers, others since abandoned.

At the time, I had contacted several mathematicians for the writing of my chapter on groupoids, including Saunders Mac Lane, who told me that they didn't feature at all in the early days of category theory. I also corresponded with the late George Mackey. During my time as a PhD student I had loved the story of maths he had told about in The scope and history of commutative and noncommutative harmonic analysis, so I asked him how he saw groupoids fitting into the picture. His reply included the remark:

At the moment I am occupied with developing some recent ideas I have had on a possible extensive development of my methods to apply to a much larger part of mathematics and produce more unification. I will explain more fully when I have made a bit more progress in seeking the proper formulation.

He spoke of some manuscripts he had written along these lines. Someone could do us a great service by digging these out.

I was also interested in diagrammatic reasoning at the time, so contacted Todd Trimble about his category theoretic reconstruction of the American philosopher Charles Peirce's existential graphs. He said:

One thing I would have emphasized, had I addressed your group, is Lawvere's revolutionary insight that the connectives and quantifiers in logic are controlled by *adjoint functors*. I think this is the key to further progress in geometrizing logic: higher-dimensional adjunctions are intimately connected with Morse theory, esp. the calculus of cancelling and rearranging critical points of Morse functions. (I don't think Gerry Brady and I fully connected the Beta graphs with this geometric aspect of adjunctions -- it ought to be done.) It is interesting to me that Peirce perceived, at a pre-formal level, the structure of connectives via adjunctions.

This connection of logic and singularity theory really needs exploring.

To end this short stroll down memory lane, I had forgotten that I had developed an interest in Dudley Shapere, a philosopher of science. He seemed to my mind to be asking the kinds of question about science that I wanted to ask about mathematics. I had included this in an e-mail:

Shapere suggests that one should be answering the following questions:
(1) What considerations (or, better, types of considerations, if such types can be found) lead scientists to regard a body of information as a body of information - that is, as constituting a unified subject matter or domain tobe examined or dealt with?
(2) How is description of the items of the domain achieved and modified at sophisticated stages of scientific development?
(3) What sort of inadequacies, leading to the need for further work, are found in the bodies of information, and what are the grounds for considering these to be inadequacies or problems requiring further research? (Included here are questions not only regarding the generation of scientific problems about domains, but also scientific priorities - the questions of importance of the problems and of the "readiness" of science to deal with them.)
(4) What considerations lead to the generation of specific lines of research, and what are the reasons (or types of reasons) for considering some lines of research to be more promising than others in the attempt to resolve problems about the domain?
(5) What are the reasons for expecting (sometimes to the extent of demanding) that answers of certain sorts, having certain characteristics, besought for those problems?
(6) What are the reasons (or types of reasons) for accepting a certain solution of a scientific problem regarding a domain as adequate. (Shapere1984, 277-8).
He reckons that only the last has been seriously examined by philosophers of science.

These still strike me as very good questions.

More on information geometry

noreply@blogger.com (david) — Wed, 21 Jun 2006 12:53:00 +0000

Some blogs use different categories to sort out their posts. However, in my experience, things that interest me enough tend to show themselves to be related somewhere down the line. A while ago I was pondering the question -

How much of the mathematics used in physics is describing our knowledge and ways of observing and intervening, and how much the physical world itself?

- in the context of Caves and Fuchs' interpretation of quantum theory. Now I see Ariel Caticha has an article trying to understand general relativity in terms of information geometry:

The point of view that has been prevalent among scientists is that the laws of physics mirror the laws of nature. The reflection might be imperfect, a mere approximation to the real thing, but it is a reflection nonetheless. The connection between physics and nature could, however, be less direct. The laws of physics could be mere rules for processing information about nature. If this second point of view turns out to be correct one would expect many aspects of physics to mirror the structure of theories of inference. Indeed, it should be possible to derive the “laws of physics” appropriate to a certain problem by applying standard rules of inference to the information that happens to be relevant to the problem at hand.

Elsewhere, work is underway to generalise information geometry to the infinite dimensional spaces used in nonparametric statistics. There are important papers at Jun Zhang's site, including 'Nonparametric information geometry: Referential duality and representational duality on statistical manifolds.' Zhang is a psychologist who uses this mathematics to model psychometric testing.