My first challenge is to get to grips with the Java Persistence API (JPA), which provides a standardised object-relational mapping (ORM) for Java applications. Basically, it takes a bunch of annotated Java classes and determines how best to map these to a relational database, taking care of foreign key constraints, inheritance, joins, and so forth. On the surface, this is all very good. Current best practice in developing these kinds of apps is to separate out the data model of the domain into a layer of simple Java beans (i.e., objects with properties and little custom behaviour), so something like JPA makes this much easier to knock together than using JDBC directly. (Although, it’s not that hard to use JDBC).

My beef is not with JPA, then, which does a reasonable job. My beef is with some of the documentation that surrounds it. In particular, most of the docs seem to be written by people with a strong OO background, but a weak logic/relational background. For example, I’m currently working through Java EE 6: From Novice to Professional by Antonio Goncalves. This is largely an excellent book, but even this author makes the same basic mistakes that are common to this literature, and indicate a lack of understanding of basic relational concepts. For example, here is a representative quote from page 105 (discussing inheritance mapping in JPA):

“How do you organize a hierarchical model into a flat relational one?”

The mistake here is in believing that relational models are in any way “flat”. I can only assume this comes from a rather literal interpretation of the usual representation as relations as tables on a page, which are (given the 2D nature of paper) naturally flat. However, any high school student will have had experience of drawing graphs and charts from data in tables, and so will know that the relations they describe are often anything but flat.

In mathematical terms, a relational database describes a hypergraph. This is a very general structure that very much subsumes simple hierarchies. If you imagine any collection of nodes in an N-dimensional space, and then connect them up with edges in almost any way you want, such a model can be described relationally in a straight-forward fashion. The same is true for hierarchies. Of course, OO models don’t just model hierarchies, but are also capable of modelling general graphs. The difference with OO is that it can only directly describe normal graphs, where each edge connects only at most two vertices (entities) – in other words, OO is limited to describing at most binary relations, much like RDF. Of course, this is not a problem as you can always reify N-ary relations into a separate entity and a bunch of binary relations.

So, neither description is correct: relations aren’t flat, and OO models are rarely strictly hierarhical. But let’s get back to the issue of inheritance. The problems, as far as I can see, of mapping inheritance hierarchies to relations come ultimately from an initial mis-design of how to map classes to relations. The common approach is to map each object to a row of a database table. Each property of the object then maps to a column in the table. This seems natural enough, but soon runs into problems. Firstly, it is difficult to represent collection-valued attributes this way. Secondly, missing attributes lead to NULL entries in the table (with all the problems they bring). Thirdly, it becomes difficult to know how best to incorporate attributes from super classes – do you collapse them all into one table, or distribute the data across multiple tables. To my mind, this mapping is just plain wrong from a logical point of view. Fixing the logical view should be the first step. After that, then you can worry about optimising access.

So, what is the correct logical view of OO and inheritance? Here, I believe OWL (the Web Ontology Language) has shown the correct interpretation. Each attribute of an OO class describes a binary relation, and so each attribute should be a separate relation (table). That may initially seem like madness, but it is the correct logical way of viewing the situation. Consider an Order class like the following:

class Order {
    Customer customer;
    Set<Product> productsOrdered;
    Address shippingAddress;
}

In the logical view I am recommending, you would have 4 relations to describe each order:

1. order: a unary relation containing the IDs (primary keys) of all orders. Thus order(x) is true if x is an order.
2. customer: a binary many-t0-one (functional) relation mapping order ids to customer ids.
3. productsOrdered: a binary many-to-many relation mapping order ids to product ids.
4. shippingAddress: a binary many-to-one relation mapping order ids to address ids.

The customer, address, and product descriptions would be similarly broken down into multiple relations.

So, how do we represent inheritance in this scenario? The answer is very simple: we simply add more relations. For example, suppose we add a new class of customers who are eligible for a discount on their orders. In Java, we might describe this as follows:

class VipCustomer extends Customer {
    double discountPercentage;
}

If the customer is a VIP customer, then we can discount their order according to the discount they are eligible for. To add this extended class, we merely have to add two new relations: VipCustomer (unary) to record the IDs of those eligible for discounts, and discountPercentage to record the discount allowed. We can then further constrain the database to ensure that all VIP customers are also customers (using a foreign key constraint). Alternatively, we can in fact infer this information using view definitions. In Datalog, we would write this as follows:

VipCustomer(X) :- discountPercentage(X,_).

This states that a VipCustomer is someone who has an associated discount percentage recorded. We can go further and infer that all VipCustomers are also customers:

Customer(X) :- VipCustomer(X).

In SQL terms, these can be written as view queries:

CREATE VIEW VipCustomer(ID) AS SELECT CustID AS ID FROM discountPercentage;
CREATE VIEW Customer(ID) AS SELECT ID FROM VipCustomer UNION ...;

Here you can start to leverage the power of the relational database to infer information for you. I recommend anyone wanting to learn about relational database design to spend some time playing with Prolog/Datalog and OWL to learn about logical modelling. Relational databases are more than just simple persistent stores: they are essentially inference engines for a restricted form of first-order logic, and you can really leverage them much better once you understand the implications of that.

Of course, you are probably screaming at me that this is terribly inefficient due to the number of joins that are required, and you are probably correct (although you should alway measure first). Once you have the logical model correct, you can then consider de-normalising the model to avoid common joins. Typically, you’d do this by finding functional dependencies, as described in most database textbooks. For example, there is only one customer and one delivery address in our order, so we can merge these columns directly into the base Order relation. However, I’d still leave the original logical design around as a set of views that map onto this base relation: the query optimiser should be able to eliminate them:

CREATE TABLE BaseOrder(Id ..., CustID ..., AddressID ...);
CREATE VIEW Order AS SELECT Id FROM BaseOrder;
CREATE VIEW customer AS SELECT Id, CustID FROM BaseOrder;
CREATE VIEW shippingAddress AS SELECT Id, AddressID FROM BaseOrder;
...
SELECT c.CustID, a.AddressID FROM customer c, shippingAddress a where c.id = a.id;

This last query should be optimised into the following by any decent query optimiser:

SELECT CustID, AddressID FROM BaseOrder;

Nevertheless, I do believe there is still much value in this analogy. After all, as Kay would point out, biological systems are much more sophisticated* than anything we can yet build, and generally much more resilient too. But we must be careful in pursuing such analogies, as the design goals of evolution are different to our own. While evolution can dispense with the intelligent designer, we cannot! And those intelligent designers have their limits. They need to be able to revisit design decisions and update running systems to correct flaws, optimise performance, adjust to new challenges, etc. Modularity is crucial to these tasks. If there is a God, He is not a software engineer.

PS – I recognise that Kay himself is well aware of these limitations of the analogy, as is abundantly evident in the wonderful design of Smalltalk.

* NB: I much prefer the term “sophisticated” to “complex”: the former is perhaps a goal worth pursuing, the latter should be minimised or eliminated.

In my field of research, I often run into material on the “Semantic Web”. This is supposed to be the vision of where the Web is going: more structured mark-up of information using tags from well-defined, domain-specific vocabularies (so-called “ontologies”). On top of this, we can then build intelligent applications (“agents”) that can draw together information from diverse sources, integrate it, make inferences from it, and generally use this knowledge to act on our behalf. This is a vision I have some sympathy with, although there are clearly huge obstacles to overcome before it becomes anything like a reality. We’re still not sure what logic dialect(s) to base it all on, and that’s just the first step!

But the problems I see with the Semantic Web are not primarily technical. The main problem I see is that the Web is actively evolving in the opposite direction. Just take a look at the HTML 5 (draft) specification. While some lip-service is paid to “semantics” in the introduction, in actual fact the spec moves further away from a declarative, logical language, and instead embraces vacuous tags such as <canvas>. This tag exists solely to provide a context into which to render, procedurally in JavaScript, arbitrary 2D graphics (or 3D graphics with the emerging WebGL standards). While I welcome the attempt to provide an open alternative to technologies such as Flash, I’m not sure that <canvas> really delivers. Indeed, it seems that it suffers many of the same drawbacks of Flash. If the goal was to continue to evolve the more meaningful distinction between different tags, why didn’t they develop a set of specific-purpose tags rather than developing another procedural API? For general graphics, the SVG standard already exists. For charts and graphs, why not introduce a more meaningful chart element, with declarative descriptions of axes, data points, formulae, etc? For example, suppose we wish to plot a 2D chart of the formula y = x². In the current HTML 5 specification we could “mark-up” this formula as follows:

<!DOCTYPE html>
<html>
<head><title>Test of HTML 5 Canvas Tag</title>
<script>
function point(x, y) { this.x = x; this.y = y; }
function axis(min, max, scale) {
    this.min = min; this.max = max; this.scale = scale;
    this.start = min * scale;
    this.end   = max * scale;
}
function xSquaredChart() {
    var chart = {
        origin:     new point(100, 0),
        xAxis:      new axis(-10, 10, 10),
        yAxis:      new axis(0, 100, 1),
        interval:   0.1
    };
    chart.f = function(x) { return Math.pow(x,2); }
    return chart;
}
function drawChart(ctx, chart) {
    var H = 100;
    var o = chart.origin;

    // Draw the axes
    ctx.fillStyle = "rgb(0,0,200)";
    ctx.fillRect(chart.xAxis.start+o.x, H-o.y, chart.xAxis.end+o.x, -1);
    ctx.fillRect(o.x, chart.yAxis.start, 1, chart.yAxis.end);

    // Plot the data
    ctx.fillStyle = "rgb(200,0,0)";

    for (x = chart.xAxis.min; x <= chart.xAxis.max; x += chart.interval) {
        var y = chart.f(x);
        ctx.fillRect(x * chart.xAxis.scale + chart.origin.x,
                     H - (y * chart.yAxis.scale + chart.origin.y),
                     1, 1);
    }
}
function draw() {
    var chart = xSquaredChart();
    var canvas = document.getElementById("canvas");
    if (canvas.getContext) {
        var ctx = canvas.getContext('2d');
        drawChart(ctx, chart);
    }
}
</script>
</head>
<body onload="draw();">
    <canvas id="canvas" width="200" height="100">A canvas should be here.</canvas>
</body>
</html>

If you try this out in a modern browser such as Firefox or Safari, that supports the canvas tag, you should indeed see a plot of the formula for values of x between -10 and 10. My point is not so much about the length of code needed to plot this simple function—indeed, I have deliberately made the code longer than it needs to be for generality—but more the total opacity of its purpose or meaning. The best we can hope for is that libraries become available for such common tasks, and that authors take the time to also supply marked-up comments and captions that identify that it is a chart and what it is a chart of. Consider a hypothetical alternative if HTML 5 directly supported tags for marking up charts:

<chart id="x-squared" type="line">
  <axis id="x">
    <range from="-10" to="10" step="0.1"/>
    <legend>x</legend>
  </axis>
  <axis id="y">
    <formula>y = x^2</formula>
    <legend>x<sup>2</sup></legend>
  </axis>
  <caption>Plot of x<sup>2</sup>.</caption>
</chart>

The formula element could be specified in MathML content tags or even in JavaScript. Explicit data points could be given with a datum tag supporting error bars and other features. The browser can add all kinds of nifty features, such as the ability to edit data or change ranges on the chart and automatically display the new values, like a spreadsheet. However, the main point is that the mark-up is much clearer, and more importantly, its purpose is easily extracted by automated tools. Other uses of the canvas tag could also be handled by special-purpose tags, such as animation, presentation slides, logos, simulations and even games.

At this point you may be thinking to yourself: OK, but surely you can’t cover every possible use of the canvas tag with special purpose tags? Indeed, you probably can’t. That doesn’t mean that we should try, though. By including special-purpose tags alongside the canvas tag we can encourage people to use them when they fit, and only resort to procedural APIs when all else fails. After all, custom tags should be much simpler to use, more maintainable and readable, and offer the possibility of much more efficient implementations within the browser rather than in JavaScript. Another much more powerful alternative also exists: an ability to define your own custom tags and tag-views. A further post will expand on this topic soon!

Does anyone know how to disable it? The “FAQ” says nothing about opting-out.

The DNS acts as the internet’s address books and helps computers translate the website names people prefer (such as bbc.co.uk) into the numbers computers use (212.58.224.131).

What this quote means to say is “DNS transforms textual domain names (such as bbc.co.uk) into the numeric dotted-form (212.58.224.131) that is used by the Internet Protocol (IP)”. To the actual computer, both of these are just patterns of bits that represent symbols. A numeric interpretation is no more or less convenient for the computer than any other symbolic representation: it’s all just patterns of electrical charge as far as the machine is concerned. The inventors of IP could just as easily have chosen to use a DNS-style address representation. It happens that the sort of symbol shifting that computers do is very good for numerical tasks, and it also happens that we know a lot about how to do things with numbers, and so numbers are useful in a lot of applications of computers. In this case, the numeric form can be compactly represented, and efficiently manipulated. That was an engineering decision, not a fundamental limitation of computers.

My love of computers stems from a fascination with language, logic, and fundamental notions of representation and interpretation. Diving into computing has brought me into contact with all sorts of ideas from philosophy, physics, mathematics, linguistics, psychology, cognitive science, and AI. It saddens me that people miss out on these extraordinarily beautiful ideas because of a persistent belief that its all really about numbers, which puts a lot of people off.