My first day cycling to work almost ended prematurely at the hands of the Metropolitan Filth.
As I was crossing the normally stately Eaton Square I was distracted by the sight of an elderly gentlemen on a bicycle of unimaginable antiquity which he must have nicked from the Science Museum back in South Ken, or indeed from the Egyptology section of the V&A. It is but a small exaggeration to say that his palaeovehicular contraption was a marvel of High Victorian overengineering: marble and hand-burnished mahogany surfaces held together by wrought iron and discredited physics, doubtless requiring frequent tyre rephlogistication.
And that's how I was almost hit by a pair of speeding police cars hooning at me the wrong way 'round an intersection like Ayrton Senna on a neckload of blow. Having kept safely to the curb, the elderly cyclist just looked at me plaintively and uttered an absurd "Poop! Poop!", though he must have been about fifty when the film of Wind in the Willows came out.
I try to restrict this 'blog to subject matter which annoys me, avoiding work, personal reminiscences and my over-indulged computing hobby.
Today I shall have been mainly annoyed about database reflection, in the sense of "automatically give me a web app which lets me view and update the contents of an SQL database". There are lots of solutions to this problem, but they partake of some of the following deficiencies:
- they also solve the problem of editing the database schema
- they are written in languages of exclusively epidemiological interest (e.g., PHP and other disease vectors such as Perl)
- they require a separate copy of the schema to reside outside the database
- they are written with the assumption that the database schema will be instantiated by the web app
Now leaving programmers in charge of the database schema does not scale, and many good programmers are afflicted by fear, ignorance and prejudice where SQL is concerned. In any case, larger organisations employ sysadmins to run production database servers and these DBAs are just not going to let some random Perl script scribble all over their database, if indeed it's allowed to run with any privileges at all.
So any off-the-shelf solution whose preference is to issue all those CREATE TABLE statements itself must be approached cautiously. The bad news is that for small-scale systems this is an acceptable means of prototyping, so this behaviour is endemic in web app frameworks (and indeed is regarded as so normal that some of the documentation refers to "legacy" databases as though it were undesirable that they should not be created by some web app). If you want to re-use any of these components, you may well find yourself stuck in the "update table requires alter table privilege" ghetto. The competent people who are writing the Create-Read-Update-Delete ("CRUD") frontend don't want to have to reimplement the whole stack, so they just give you access to whatever database tables the rest of the framework presents them.
... and these frameworks, being written in dynamic languages, instantiate database entities as objects or classes in their own right. Where the web app contains the authoritative copy of the schema, doubtless in some pidgin DDL, it's obviously trivial for a cargo cult programmer to cut-n-paste himself a few more database tables by imitating the the ritual incantations elsewhere in the models file.
I reproduce an example of this stuff below out of sheer sadism:
class UnbearableMovie(DeclarativeBase):
__tablename__ = 'unbearable_movie'
#{ Columns
id = Column(Integer, primary_key=True)
title = Column(Unicode(255), nullable=False)
director = Column(Unicode(255), nullable=False)
#}
So once you've included that in your code, you do m = UnbearableMovie(name="Sex and the City") or whatever. This is no good if you don't know what the tables are called because someone else is deciding that or it might change, but to their credit the SQLAlchemy people have an autoLoad feature which inspects a database table for its column names. I couldn't find anything on the web which ran this on all database tables in Python, but did find people with the same problem. So here is a rough and very brittle solution:
class ReflectedTable(object):
def __init__(self, **kwargs):
for k, v in self.iteritems():
setattr(self, k, v)
def init_model(engine):
"""Call me before using any of the tables or classes in the model."""
global metadata
DBSession.configure(bind=engine)
def table_names():
introspect = '''SELECT name FROM sqlite_master
WHERE type = 'table';'''
for row in engine.execute(introspect):
yield str(row[0])
def do_reflect(name):
class_name = name.title()
reflected_table = Table(name, metadata, autoload=True,
autoload_with=engine)
cls = type(class_name, (ReflectedTable,), {})
mapper(cls, reflected_table)
setattr(MY_APP_NAME.model.reflected, class_name, cls)
for name in table_names():
do_reflect(name)
Combined with the catwalk CRUD frontend, this is enough to get the necessary: a database whose schema changes are reflected in a web app without further hassle.
UPDATE: Someone may well have got there before me, and written SqlSoup, though I've not checked that that does what I want. Interestingly, the chap who wrote has written a "blow off steam" blog post just like this one to vent his frustrations.
Chinese professor attacks Labor PM from the Right ...
Later this year there will be another referendum in the Republic of Ireland about the Lisbon Treaty. The previous one was defeated so the executive branch of the government has negotiated a set of arrangements with other EU member state governments' executive branches. Whereas polling suggests that public has changed its mind on the Lisbon Treaty for reasons other than these measures, assuming that it were these arrangements which will persuade people to vote differently, how will they make such a difference in practice as to persuade people to vote differently?
The EU exercises powers conferred on it by member states. These powers and the procedures for their exercise are defined in treaties between the member states. Each state has provision for the EU law enacted via these conferred powers to have effect in its legal system. Often EU law will differ from what a member state would enact left to its own devices: there'd be no point having the EU if the outcome of its legislative processes were identical to what member states would otherwise do. As the EU's processes are less democratic than the member states, there is a systematic bias in EU law towards inefficient regulation and the favouring of special interests.
The Irish are concerned about abortion, tax and defence neutrality; the constitution was amended by referendum in 1983 to ban abortion, and this has conflicted with the fundamental EU free trade rules: in the 1990s the Gardai interned a 14 year old rape victim to stop her travelling to Britain, where abortion is a inter alia service provided for profit, for what in Ireland is regarded primarily as a moral issue; EU law would have had to have favoured the British medical industry or the Irish electorate.
The previous solution was to add a protocol to the Maastricht Treaty purporting to exempt Ireland's abortion policies from the general rules about free movement. All solutions have to look something like this. The options are either
- to prevent the EU from having a particular power conferred upon it,
- conditioning the exercise of powers on the consent of the Irish government,
- or to obtain a derogation such that the powers are not exercised against the Republic of Ireland
To make a difference, any new compromise measures for the Lisbon Treaty need to be respected by Irish courts when interpreting EU law. Conventionally, EU law has meant "the EU treaties in force as a result of ratification according to the procedures of each member state, plus the much larger body of laws made via processes defined in those treaties". If the Republic of Ireland is to have an effective opt-out or guarantee, somehow that needs to be included in the treaties, but treaties require ratification. The Lisbon Treaty stands ratified in the generality of EU member states, and in member states such as the UK, this ratification was in the teeth of public opposition and manifesto commitments to predicate ratification on approval at referendum.
Should the Republic ratify the Lisbon treaty once a protocol with its guarantees is added, other countries' interest groups might demand similar concessions, re-opening the debate on ratification of the original treaty. How "ratified" does such a protocol need to be for it to have effect in the Irish legal system? Does each member state need to go through its ratification procedure again in relation to the protocol (or the treaty plus the protocol)? If the guarantees in the protocol can become effective without unanimous ratification, why is that necessary for the treaty itself?
This blog post was occasioned by a curious report on the front page of Thursday's Irish Times, which stated "One alternative that member states may offer the [Irish] Government is a legal decision on the guarantees issued by the European Council. This would have legal standing but would enshrine the guarantees in the EU treaties (sic)." It's not clear whether there's supposed to be a "not" in either clause of the second sentence. What on earth is going on here? The European Council is effectively the prime ministers and presidents of the EU member states. Has it suddenly become a legislative/judicial body?
I think what is being proposed here is a formal agreement similar to the Luxembourg Accord whereby de Gaulle obtained a veto on all EEC legislation exercisable by any member state's executive. This was never justiciable and was done away with by Thatcher when it proved a bulwark against market liberalisation. It isn't, or at least shouldn't be something you can argue before the courts, so wouldn't override any EU law adversely affecting Ireland's position on taxation or abortion et c.
But if it were justiciable, would this not constitute an alternative to treaty ratification? Should not member states be concerned that elaborate treaty ratification procedures can be supplanted by the say-so of the head of government, even to the extent of overriding provisions of the national constitution? Can one really envision the Bundesverfassungsgericht permitting the rights of Germans to turn on an agreement between the heads of the executive branches of foreign governments? Against the national constitution? When those governments might depend on extremists for their majorities?
I should note that I make no apologies for the cumbersome wording of sentences above, as the common synecdoche conflating a country ("Republic of Ireland"), its government as a whole ("the Irish government") and those who negotiate on its behalf ("the executive branch of the government") is particularly apt to mislead when EU negotiations are taking place: the executive branch routinely uses its monopoly over external negotiations to usurp the rôle of the legislative branch.
The actual reason the Irish have changed their minds is the economic downturn: somehow it is felt that either the Lisbon Treaty's ratification will be of economic benefit, or will lead to more sympathy and generosity from other EU member states.
Who benefited from the secrecy of MPs' expenses? In 2007 members of the House of Commons Commission and their allies attempted to get the Freedom of Information Act amended to prevent the disclosure of this information. One might speculate that such people had something to gain from doing so.
Commons expenses are managed by a body called the Fees Office, which is effectively subordinate to this Commission. It has now been revealed that the Office, and senior MPs, literally encouraged some MPs to maximise their use of the allowances and expenses system. This money was characterised as being money that the MPs ought to receive but could not as the public was opposed to greater direct salaries. As such, its receipt was conditional on MPs' behaviour. MPs thus compromised would be less likely to oppose those who ran and knew about the expenses system.
It has been known for centuries that financial dependency assorts ill with decision-making independence. This is the justification for paying MPs (and judges) such high salaries in the first place. This is why the practice of having too much of the legislature on the Crown's payroll was banned in the Act of Settlement 1701, and we still regulate the number of ministers. Indeed the very mechanism for resigning from the Commons consists in applying for a Crown office reserved for that purpose. The expenses system thus achieves an effect known to be constitutionally improper and illegal if achieved by more straightforward means. Fewer actual taxpaying constituents' votes and views matter when the behaviour of their MPs can be influenced in this way.
So who is the House of Commons Commission? It is, roughly speaking, the whips of the major parties and the Speaker, whose election in turn used to depend on the whips (though no longer).
Publishing the expense information reduces the power of the whips over the backbenchers, in favour of the power of their constituents, and this is a good thing.
The usual malcontents are trying to use the current crisis as an excuse to push for their favourite constitutional "reform" schemes such as an elected Upper House, electoral "reform", fixed-term parliaments and other cryptorepublican schemes, rather than measures targeted at the actual abuses taking place at present. The only one of these off-topic proposals for change which attracts my sympathy is primary elections for the House of Commons.
Primary elections were previously problematic in the UK because its liberal (in the sense of non-corporatist) constitution succeeded in failing to recognise political parties as such. Following an election whose outcome was influenced by a candidate whose party affiliation was contrived to mislead, we now have state recognition of political parties. We do not, however, have fixed-term parliaments, and thus is it not obvious when to hold a primary election.
Now we could adopt fixed-term parliaments, but that no longer has much to do with redressing the balance between taxpayers and party whips. It involves changing the conditions under which Parliament may be dissolved. These are currently not under the control of politicians, but that would have to change to move to fixed-term parliaments, as the choice currently rests with the Head of State and is exercised according to the Lascelles Principles. The most recent occasion on which this made a difference was after the resignation of Tony Blair as leader of Labour. Had he called a general election once it looked like Brown would win the party leadership, the Queen would not have granted one. The subtleties involved here would have to be enacted in statute, and thus become subject to the whims of future parliamentary majorities, and one might ask again ... who benefits?
As a compromise, primary elections could be held three years after the most recent general election, and constitute a mandate for the next five years (being the maximum life of the House of Commons). This would reduce the power of party hierarchies to threaten MPs with deselection, and independence of MPs is paramount.
Join me, comrades, in the distributed nanoclassification of human intellectual failure!
For too long have readers laboured under the weight of a thousand species of false infererence. In too many places, words written or uttered knowingly by persons in public life have been the unsafe scaffolding of arguments which should never have been constructed. Scarcely is it possible to read a single paragraph of a newspaper but one encounter an error of fact or inference fatal to whatever point was being advanced.
What is said and written in public life informs and affects the attitudes of voters. Such influence must be used responsibly. However remote the causal chain between utterance and enactment, however small the responsibility borne by those misquoted in the press or constructing op-ed pieces, it exists. Everyone in our society would benefit if public debate were conducted more clearly and accurately: the effect of regulations would be better predicted. This would happen if there were greater incentives for those engaged in public debate to communicate responsibly, if individuals could vindicate their interest in accurate public communication, if it were cheaper to rebut all the rubbish hack journalism out there, if it didn't cost the budget of Malawi to disprove all the lies told on a single episode of the Today programme. "If" ...
The regulatory cost of bad journalism is externalised from participants in public debate to the victims of inefficient regulation. Public debate is subject to insufficient discipline: little beyond the traditional "whatever the advertisers will bear" and letters to the editor. The publication of a reader's rebuttal is entirely at the discretion of the newspaper editor, and, though the norms and costs differ, the same applies for blogs.
When a newspaper publishes information which is false, the fact of this publication, and the sense in which the original information is false, are themselves a piece of information: a fact about an assertion, or at least, an assertion about an assrtion. Let us call this "accuracy metadata". This metadata obviously may influence reasoning about the original facts, so its availability will affect the ultimate policy decisions taken in response to the perceived preferences of voters. Currently the accuracy metadata for a newspaper consists in errata and letters to the editor published in the newspaper itself, and critical commentary elsewhere, e.g., in others newspapers and blogs. The scarcity of accuracy metadata about their articles reduces the incentives for accuracy, and permits some columnists to make a living pandering to the hatreds and insecurities of readers with little regard for the truth.
Editorial control over accuracy metadata promotes inaccuracy.
On the web, it is broadly feasible to use search engines to find articles referencing a given article automatically; this includes accuracy metadata hosted on independent websites outside the publisher's editorial control. The cost of assessing the accuracy of a piece of information contains the cost of locating its accuracy metadata and evaluating it, and that of publishing this metadata and getting it noticed. Technological change has made some of these processes affordable for a much broader section of society.
People might have the ability to pay for accuracy metadata, but have they the willingness to do so? Can anyone be bothered, except where he's seriously affected? Polities permitting free association of private individuals are increasingly common, and have even included France since 1901. Free societies abound in organisations dedicated to scrutinising public authority and contributing to public debate. These are the agents of the vast multitude who can't be bothered to follow public debate in detail: political parties, trades unions, professional associations, watchdogs, NGOs, single-issues campaigns, for-profit lobbying firms, et c, and are either freeloaded upon, or paid for voluntarily (or compulsorily in the case of some quangos).
Whoever bears the cost of improving public debate, technology has lowered it and will continue to. It is already possible for editorially indepepent accuracy metadata to be published and discovered. Currently this is deficient in three ways. Accuracy metadata tends to cover a whole article or document rather than a given paragraph or sentence; it may not identify what precisely is wrong with a statement (as opposed to indicating its truth value); and it is expensive to process, on account of textual inconsistencies. Technology can improve this granularity, specificity and consistency.
Granularity is the hardest to fix: web documents can be referenced as a whole by means of URLs, but the existing mechanism for identifying material within a document relies on its publisher incorporating anchors within his own text, and this is unconventional for electronic copies of newspaper articles. For the time being, a heuristic way of identifying sentences will have to do: the URL, plus a rough guide to the desired paragraph and some of the text of the sentence, sufficient for a computer to be able to identify the sentence referred to and robust against small revisions of the original document.
Specificity and consistency are two aspects of the same problem, and have a single solution: some means of encouraging people to denote like things alike. There must be a list of ways that a sentence can be wrong, with each having an identifier which will be standard across all the sentences discussed. In practice, what this would look like is the ability to highlight a sentence in the browser, and then select what is wrong with it from a list, with the software publishing this somewhere.
This already exists in a limited fashion on Wikipedia: there is a set of tags which may be added to any article which assert that the preceding text is not from a neutral point of view, or requires a citation, or whatever. Each of these is consistent across Wikipedia: this means that it's theoretically possible to see cheaply which editors are the worst neutrality offenders, incorporators of uncited material, and so on. I learnt from David Stove's article, "What is Wrong With Our Thoughts", that a list of diseases is called a "nosology", and he proposed to establish one for human thought, but averred that it might need billions of entries. Stove was not concerned with User Interface design, but a drop-down list of this length will never do, particularly in the mobile and netbook segment. Crucially, moreover, some ways of being wrong are much more frequent or important than others.
Now our friends on the radical Left, of whom we have none, will immediately protest that this system will entrench (or is it "perpetuate") the hegemony of a single arbiter of what is true. Partially, this criticism will be motivated by relativism. In its extreme form, relativism is the notion that it is morally permissible to mutilate a girl's clitoris if you're black.
Less polemically, it is the claim that truth is not universal, but depends on local conditions. I propose to surrender entirely to this claim. There are similar claims, such as that what is true is only what it is useful to regard as true. I ask those holding this view whether it is only true that Hitler caused the death of millions of Jews insofar as that fact is useful, and who might benefit from this sort of idea? Whatever the philosophical claims which might motivate one to oppose a single nosology of human thought, one need agree with none of them but still acknowledge that the establishment of a single list of ways a sentence can go wrong is not a good idea. Just as there should be a plurality of institutions participating in public debate, let a hundred nosologies bloom, and let a hundred thousand asserters use them.
What I propose is that web-browsing software gain the ability to select text from articles and publish a designation of this text as being defective according to some nosology. So, a Nosology Service Provider (NSP) could describe, collate and publish the types of inferential errors which it is concerned to see rebutted. A citizen, acting on his own or as part of an organisation dedicated to advancing public debate, which we shall call Nosological Assertion Service Providers (NASPs), could then independently publish accuracy metadata asserting that such and such a sentence was false in a particular way, according to his (the NASP's) understanding of the NSP's nosology.
This more than accommodates the relativists. It is perfectly possible that reasonable people may disagree in good faith about what is true, so one needs a plurality of NSPs. In mathematics this is completely formalised, e.g., one simple acknowledges whether one is assuming the Axiom of Choice is true or false. Reasonable people are generally well capable of honestly assessing what someone with different beliefs about what is true may think. They are also capable of starting with the same set of beliefs but assessing ambiguous or complex evidence differently, so NASPs may differ in their use of the same NSP's nosology.
The cost of NASPing will probably be pretty lower than the cost of writing the original sentence. This is an important precondition for what Yochai Benkler in "Coase's Penguin" terms "peer-based commons production". It may be easily possible in the future for anyone viewing the website of some news source to see elsewhere in the window a set of counterassertions about parts of the text, indicating that everyone from the Greens to the Tory Right wants to point out, in relation to some statement by the minister, that "Correlation Does Not Imply Causation". Users could opt not to be presented with text condemned by too broad a selection of NASPs, helping reduce the advertising revenue brought in by atrocious journalism. Statistical analysis of the types of error favoured by particular participants in public debate could be compiled. The more one tried to mislead the public, the less one would be listened to.
I hope one day that the above Ad Hominem Tu Quoque about Hitler is one day noted as such by some NASP, and we'll finally get to see just how much a NASP knows about English literature.
The Principle of International Law is that one should not act justly now, unless in concurrence with the representatives of the executive branches of the governments of the five permament members of the UN Security Council, who are disposed to agree on no matter other than the maintainance of this privileged position. All wars of choice either are wrong, or, if they are right, contravene International Law. It follows, then, that no man must ever be defended by armed force against the aggression of his own government, but must instead have recourse to a process known as Reforming the Dictatorship From Within.