Blog

Model railroad signal decoder

In the past few weeks I’ve built a model railroad signal for garden railways with a custom DCC decdoder. Here’s what it looks like:

I’ve heard from a number of people that they thought this was interesting, but they didn’t really understand what I was doing. So today I’ll try to explain the core of it all, this circuit board:

A complex electronic circuit board.

And here’s the associated circuit diagram:

A complex electronic circuit diagram.

I’ll try to explain why this is the way it is and what it does.

(This whole explanation is aimed at people who have never done anything with electronics before. If that’s not you, then this may be a bit boring. Also, I didn’t come up with any of the parts of this. Most of this is based on things I learned by reading OpenDCC and Mikrocontroller.net. I’m sure I still made a lot of mistakes, though, and they’re definitely all mine and not the fault of anyone on these sites.)

The goal

First let’s talk about requirements. My goal was to build an american signal type “Searchlight”. Such a signal has between one and three lamps. Thanks to a clever electromechanic design that moves different color filters around, each lamp can show different colors - up to three from a total selection of four.

Replicating this system for a model railroad is not practical. I need something else. Having multiple colored LEDs next to each other wouldn’t work; they’re too big and I want it to all look like one light source. There are LEDs that contain red, blue and green in one housing, but that would require a lot of wires quickly that all have to be put in the mast. The solution is this:

Two wired LEDs with four pins sticking out of each.

This is an “adressable” LED, better known under the name “Neopixel” used by a large american online store. There are many variations from different manufacturers. The key thing is that each LED has a tiny control circuit built right in. It takes four wires: Plus five volts, minus, data in and data out. If you have more than one, you can connect the data out of the first directly to the data in of the second and so on. Connect the plus and minus as well, and you can control almost unlimited amounts of LEDs with just three wires.

The data line has a special protocol that you need to generate. Basically you need to switch it from 0 to 5 to 0 volts again and again at a certain rate; the time it stays at 5 volts (“high”) determines whether you’re sending a 0 or a 1. From these bits you form bytes, which tell each LED what specific color value to send.

Due to this dataformat, you definitely need some electronic circuit controlling the signal, and the first requirements for this are:

  • Provide five volts DC power
  • Generate the data for the LEDs in the correct format

The Input

There are a lot of options for designing the input side of things. In my case, I’m assuming the signal is electrically connected to the rails of a model railroad that is controlled digitally. With digital command control (DCC), the voltage at the rails has a constant value of about 15 to 25 volts, larger for larger scales. This voltage constantly flips polarity; first plus is on the left rail, then it goes to the right rail (and minus vice versa), and then back. It’s like AC in normal wall outlets, but with very abrupt changes instead of a smooth sine wave.

An irregular square wave shown on an oscilloscope.

This voltage has two tasks. First it supplies the locomotives with power, but it also transmits information. If one of these change-and-back sequences is long, it transmits a “0”; if it’s short, it transmits a “1”. These bits together then form the bytes that form the messages that say things like, “Locomotive three run at speed step 64” or “switch 10 switch to direction left”.

This decoder uses both features. The digital voltage provides both the data and the power. For a locomotive, that is required since the only conductors you have are the rails. This is a stationary decoder, so I could have designed it so that it only uses digital commands, and gets the power from an external power supply. However, I wanted to use the least amount of cables, so I’m using the simple version.

With that, the requirements are fixed. The circuit has to:

  • Turn the digital power (15-25 Volts, AC-ish) into 5 Volts DC
  • Read and understand the digital data signal (decode it, hence the name “decoder”) and calculate the colors for the LEDs.

Computation

This calculation is the real key here. The digital signal has a completely different fromat than what the LEDs expect. It’s slower, but also has completely different meaning. At best it transmits “set switch or signal 10 to state 0”. Which color values are associated with that, let alone any blending to make it look nice, are things the signal has to decide for itself. There is no way to build a simple stupid adapter here; I need a complete computer.

Luckily, you can get those for cheap and in really tiny.

A single small computer chip.

The ATtiny85 costs about 1€ depending on how many you order, and it’s smaller than one cent coin (I think in basically any currency), but from a technical point of view, it is essentially a full computer. It has all the important parts anyway. There is a CPU that can run at (depending on the version) up to 20 MHz; half a kilobyte of RAM and eight kilobyte of internal storage for the program. Multiple programs is a bit of a challenge. If you know Arduinos, the ATtiny85 is related to the ATmega328p in the Arduino Uno and Nano. Far less powerful, but cheaper and significantly smaller.

What it lacks are all the surroundings like keyboard and screen for input and output. The chip is designed for applications where this isn’t needed, or at least only minimal things. The software that you write can assign each pin (okay, five out of eight) freely for different tasks: The pin can work as an input, telling the software whether there’s a low or high level of voltage at it (meaning 0 or 5 Volts), or it can work as an output and write high or low values, meaning setting the pin explicitly to 0 or 5 Volts.

There are other options for the Pins as well; among other things it can also read analog voltages and generate them to some extent. But for this task I only need the simple digital high-low inputs and outputs.

These types of chips, known as microcontrollers, exist in thousands of variations by different manufacturers with very different performance characteristics. They are the key part of basically everything that’s digitally controlled these days. Washing machines, everything that plugs into a computer including every single Apple lightning cable, TVs, TV remotes, amazing amounts of parts in cars and so on are all the realm of microcontrollers. The ATtiny85 is, as the name implies, very much at the low end of the scale (though there are smaller ones), and even here, it is a bit out of date. But it is very easy to program and very forgiving of mistakes, which makes it great in hobby situations.

To run, this chip needs around 3-5 Volts DC (some versions like the one here can also run on a bit less) and exactly one capacitor. I’m already generating 5 Volts DC for the LEDs anyway, so this chip will get them as well. That means for all the calculation, only two pieces of hardware are required.

Excerpt from the circuit diagram. Shows a microcontroller and a capacitor.

Excerpt from the photograph of the circuit board. The microcontroller and its capacitor are highlighted.

There is some more associated hardware, though, for getting the program (which I’ve written myself) on the chip. For that you need a programmer, a device that you can buy for some money, or make yourself astonishingly easily from an Arduino. It needs to be connected with six wires to the chip. The standard for this is with a six-pin plug, which I’ve thus included here as well. There are standard six-wire cables for this.

Excerpt from the photograph of the circuit board. The six pin plug is highlighted.

You could connect the cables differently, for example with some sort of spring-loaded contacts on some programming circuit board you’d have to build for that, or in the worst case, just temporarily solder the cables in there. But the plug version is both simple and convenient, with the only downside that it makes the circuit a bit more pointy.

5 Volt DC

The 5V DC is a fairly boring topic compared to the slightly complex software. But on the other hand, a lot of the circuit board is just concerned with it, and it is really the basis of the decoder. It fundamentally consists of two parts:

DC

To generate DC I’m using a bridge rectifier, a simple standard setup of four diodes.

Excerpt from the circuit diagram. Shows a bridge rectifier.

The diodes are arranged such that no matter what, the rail that is currently “plus” is connected with the internal “plus”, and the rail that is currently “minus” is connected to the internal “minus” (also known as ground, shortened to GND).

To simplify the circuit board I’m using double diodes. These are parts that contain two diodes in one housing. The diodes are connected with each other and the three pins.

Excerpt from the photograph of the circuit board. Two identical parts with three pins each are highlighted.

The first (here lowest) pin is connected to the cathode of the first diode. The middle pin is connected to the anode of the first diode and the cathode of the second. The top pin is connected to the anode of the first pin. That means I need only two parts to have four diodes. There are integrated bridge rectifiers you can buy, but those are optimized for normal AC and not ideal here.

5 Volt

From the rectifier we get between 15 and 25 volts DC, depending on what the model railroad is set to. Honestly, 25 is probably excessive, but I preferred to err on the side of safety. Either way, even the low end of the scale will immediately fry the small LEDs and the microcontroller. That means I need a voltage regulator.

Excerpt from the circuit diagram. Shows a voltage regulator with a lot of capacitors, and a diode connecting the output to the input.

Excerpt from the photograph of the circuit board. A large piece with three pins on one side and a metal tab on the other is highlighted together with some capacitors.

The voltage regulator, here an UA78M05, is a fairly complex integrated circuit that contains a big output transistor that conducts current. Part of the circuit checks whether the output voltage matches the expected 5 Volts; it opens or closes the transistor more or less to ensure it does. For stable working it needs a few more capacitors at both input and ouptut side.

An oddity in the design I’ve chosen is the diode.

Excerpt from the last picture. Only a single part with two pins is highlighted.

During normal operations it is blocking, meaning it does nothing. You could totally get rid of it. The problem is when I program the microcontroller. In that case the 5V for the whole design comes from the programmer device. In that case the voltage regulator has 0 Volts on the input side and 5 Volts on the right. It does not like that and could be destroyed from that, since current is going the wrong way round in there. In this case the diode conducts the 5V (minus 0.7) to the left side. That is not enough for the regulator to do anything, but it is enough to ensure that it doesn’t break.

The specific diode, type 1N4148, is one I picked because I had some of those laying around.

The whole voltage regulator is a very simple solution, but not a great one. The very high-level view is that it works like a self-regulating resistor. That means the current going through it is the same on the input and output side, and only the voltage changes. If you get 5 Volts with 20 milliampere out, then you need to put 20 milliampere at, say, 15 Volt in. The difference of 10 Volts times 0.02 Ampere means you loose 0.2 Watts, which get turned into heat. That may not sound like much, but it already gets warmer than I expected. I think with up to three LEDs it should be alright, but there’s definitely room for improvement.

The preferred solution would be a so-called buck converter, a significantly more complex part that needs more additional components. I’m going to try that out for the next version of this decoder.

Data Input

To parse the DCC signal, it’s sufficient to read the data from a single rail. If the rail is the “plus” rail, it has a high voltage (compared to the internal Minus or 0 Volt level); if it is the minus rail, then it is connected via the rectifier to the internal 0 Volt level. Using just one input means I can’t tell the difference between “track is minus” and “there’s no power from the track”, but since the decoder is always wired in, I don’t think that’s important.

The problem is that when the track is positive, it has a voltage of 15 Volt compared to ground. That is way too much for the microcontroller. To solve this I’m using a small design with a Z-diode.

Excerpt from the circuit diagram. Shows a resistor in series with a zener diode. Another wire branches off from the connection between the resistor and the zener diode.

The special thing of a Zener diode is that if you put it in reverse, where normal diodes block, it will start conducting anyway if the voltage exceeds a certain level (and stop once it doesn’t anymore).Here I’m using a 4.7 Volt one. If the input has 4.7 volts or more, it starts letting so much current through that the voltage drop over it is pretty much exactly 4.7 Volts, which is enough to be read as “High” by the microcontroller.

The amount of current that the Zener diode lets through can quickly grow very high, which in turn can destroy the diode. That’s why it needs to have a fairly large resistor in the line leading up to it, to ensure that there is never a lot of current going through it. The microcontroller itself doesn’t need any current from this pin, it just measures whether the voltage is above or below a certain threshold. That way it can read the DCC signal.

Excerpt from the photograph of the circuit board. A small piece with three pins highlighted together with a resistor.

A weird specialty here: Like all diodes, a Zener diode has only two ends that need to be connected. For some reason, the one I have here comes in a package that has three leads. (Yes, that is the same package as for the rectifier diodes.) The third lead isn’t connected to anything in the diode, and it isn’t connected to anything on the circuit board either. It’s just soldered in to hold the Zener diode safely in place. I’m not sure why it was designed this way, but at least it makes it impossible to solder it in the wrong way round.

Bonus: The decoder also works flawlessly if I just leave out the Zener diode and use only the resister. I’ve tested this. The excessive voltage gets transferred away via built-in protection dioes on the microcontroller which connect it to the 5V rail… somehow. I haven’t honestly really understood what’s happening in that case, and in particular how I calculate it. I know from the internet that somewhere between 22 and 33 kiloohms works well here, but I can’t actually show you the maths that prove why. That’s why I preferred to use the version with the Zener-Diode here, which is much more easily understandable. The next version of the decoder, which will get more components elswhere, will probably not have the Zener-diode though, just to save space.

Output

The output hardware for the microcontroller is simple because there is none. One pin of the controller is directly connected to the data in pin of the first LED.

There are Best-Practices documents for these LEDs that recommend adding a resistor, ideally close to the LED. Since I don’t have long leads to the LED and only use relatively few, I haven’t needed that this far. If I do, I’d probably add the resistor to the LED instead of the decoder.

That means the decoder is done. Well, almost.

Output part 2: Acknowledgement

A problem with classic digital model railroads is that the protocol is strictly one-directional. There is no way for the decoder to send any information back. No part of the circuit I’ve designed here can provide any voltage back to the tracks or similar.

Most of the time that’s no problem. But if a digitalsystem wants to configure the settings that the decoder has, for example setting the address it responds to, then it would be nice if there was a way to read the existing value, and get a signal that a new value has been written successfully. For that, there is a small hack: The decoder in a locomotive simply briefly turns on the locomotive’s motor. This increases the current drawn, and that can be measured.

The problem for my decoder is that there is no motor connected. So I need to increase the power draw some other way. In the first version I did that by simply turning the LEDs on fully white, which actually works. But there are scenarios where this might not be sufficient, for example if I build signals for N gauge with smaller LEDs that draw less power.

So that means I need to increase power by letting current flow through a resistor. The resistor needs a larger housing because it needs to withstand more power that gets dissipated in it. I guess one in a smaller housing would work just as well, since the power is applied only very briefly (six milliseconds), so it doesn’t really have the time to get hot, but better to err on the side of caution.

Excerpt from the circuit diagram. Shows a resistor in As a load for a MOSFET.

Excerpt from the photograph of the circuit board. A different small piece with three pins highlighted together with a different resistor.

If you look closely you may see that the circuit diagram and the board both say 47 Ohm, but the resistor is actually 68 Ohm. I did the maths again and changed my opinion after I had alreaady ordered the circuit board here.

The resistor cannot be connected directly to the pin of the microcontroller, because the microcontroller can’t deal with all the current that would flow through it. Instead I’m using a small transistor, specifically a MOSFET of type 2N7002. This thing is nice and easy: If the microcontroller output pin is at 5 Volt, then it is open, if it’s at 0 Volt, then the transistor is closed. There is no additional hardware required.

The only real problem with this transistor is that it is the third different component in the same exact package. It is the same package, known as SOT-23, that is used by the Zener diode and the double diodes for the rectifier.

Excerpt from the photograph of the circuit board. Highlighted are four different components that all look alike and all have three pins each.

If you shine a lot of light on it, then you can maybe see that each of them has some markings. These markings are difficult to read under the best of circumstances, and they’re only a code for the component; you still have to look up which thing it actually is in a table somewhere. Really, what you should do is keep these pieces in the labelled plastic bags they came in until you need them, and then immediately put them back before you open the next bag.

(I deliberately ignored something here: There is a fairly complex extension of the DCC standard known as Railcom which decoders can use to answer. This requires additional hardware on the decoder, which I didn’t add, and also a more powerful microcontroller.)

Conclusion

The photograph of the circuit board again. All parts are highlighted in different colors.

That’s all there really is to this decoder. I’ve designed the circuit diagram and the board layout with Eagle. I wouldn’t say I really like the program, but it can do everything you need, and it’s free for hobby purposes. I did not make the circuit boards myself, but instead had them made at Aisler. They charge a little over six euros for three of these boards (size about 2 by 2.5 cm). I’ve soldered it together myself, and indeed, it happens to work.

Alfa Romeo Giulietta Racing in 2018

With the first few races including Alfa Romeo Giuliettas already completed, it is about time to update the old post for things going on in 2018. There’s actually a lot!

TCR

A quick recap: TCR is a specification for C-segment touring car racers that are comparatively cheap but quick and look dramatic. Many different series use them. The TCR version of the Alfa Romeo Giulietta is built by independent tuner Romeo Ferraris (no relation) and raced in various series, either by Romeo Ferraris themselves, some front company, or their customers. And there are more of them every year!

The big news is that the headlining TCR World series is changing. Previously the WTCC (world touring car championship), officially the higher-ranking series, was in a slow but steady decline because nobody was making the TC1 cars used there. For this season, the series has adopted the TCR regulations, and also the staff who used to manage the old TCR World series. The TCR World series is officially cancelled, and the WTCC is renamed to WTCR, officially expanded to World Touring Car Cup. Apparently it’s no longer allowed to be a championship because the TCR regulations do not allow full works teams.

The Giulietta will be there again. This time it will run under the team name “Mulsanne Racing”, which was also already used for the TCR Middle East 2017 (the Giulietta did not participate in the TCR Middle East 2018). You can actually see that they changed the front bumper a little bit compared to the 2018 car, presumably to improve airflow into the central cooler. I still have the impression that Mulsanne Racing is not really an independent entity, but just some shell that Romeo Ferraris use, since they have absolutely no internet presence. The closest thing I could find is a “Mulsanne Racing” in Dallas, Texas, a company that offers fractional ownership in go carts and which is presumably no relation. But I could be wrong here.

The drivers this year will be italian veterans: Fabrizio Giovanardi, who was champion in various touring car series, and Gabriele Tarquini, who has a history in Formula 1 and was very successful in various touring car championships. Both of them have driven Alfas before. Overall, this is the main Alfa Giulietta effort in the TCR.

But the customers are also still around! First, the simple story: In Italy, V-Action, who ran two Giuliettas last year, will continue to do so. This time they will hopefully be able to run them the entire season instead of only from race five.

The very first race with a Giulietta this year took place in Austin, Texas of all places. This race was part of the Pirelli World Challenge, despite the name a US-only race series with many categories (that usually run independently) that added a TCR division this year. Romeo Ferraris together with one Tim Munday of BAM! Motorsports actually set up a whole import and support business for the Giulietta, to sell it to teams racing there. This makes it the first time that the Giulietta is sold in the US at all. So far they seem to be running just one car, on their own, but it did at least get a fourth place in the first round (but did not finish in the second). The homepage also lists events from the “IMSA Continental Tire SportsCar Challenge”, another sports-car racing series whose name also starts with a tire manufacturer and ends with Challenge. The difference, apart from the tire manufacturer, appears to be that this one is an endurance race series. That series also added a TCR division for this year, but at least for the first race this year, there was no Giulietta there.

Then we got the UK, which went from no Giuliettas racing in 2017 to three in two different series. This country got its own TCR series this year, creatively titled TCR UK. The intention is that this will be the junior series to the more higher-spec BTCC (see below), but some drivers will compete in both. The Giulietta is run by Laser Tools Racing, featuring Aiden Moffat and Derek Palmer. In the first round, the first race had a place 2 for Aiden while Derek did not finish. In the second round, Aiden did not finish (ran out of fuel in the penultimate lap while on second place) and Derek Palmer, who was battling technical issues the entire weekend, finished last. So far a mixed but not entirely hopeless start.

There’s no word on TCR Giuliettas elsewhere, but the thing with TCR is that it could always appear as a guest star somewhere. The TCR Europe, this time a full series, looks like it might be a nice spot. I don’t have high hopes for the TCR Germany as a regular, but I wouldn’t be surprised to see it as a guest starter once more. And there’s still that one Giulietta owned by Hungarian team Unicorse that doesn’t seem to be doing anything.

(By the way: English-language Wikipedia currently claims that Unicorse raced the Giulietta in the ETCC in 2016 and 2017. That is incorrect. They didn’t get the car until 2017 and it does not show up in any result lists for the ETCC 2017. Unicorse would have liked to run it there, according to some press releases, but that hasn’t happened yet. The ETCC is now dead, but TCR Europe is still around.)

Note that while TCR regulations are also used for endurance series, I have not heard of anyone who wants to run the TCR Giulietta in an endurance series, and Romeo Ferraris don’t seem to be marketing it there either.

BTCC

The British Touring Car Championship is the “main” touring car series in Britain, featuring cars that are basically silhouette racers: While the body shell is from the factory (and then heavily modified of course), much of suspension, brakes and drivetrain, including optionally the engine, are standard parts. As already announced in 2017, they now feature their own Giulietta as well. Built by Handy Motorsport and driven by Rob Austin, the car is now finished and has done first tests. The team has said that they are very excited about its potential. Apparently the great thing about the Giulietta, in this series that removes almost all technical aspects that make the car unique, is the wheelbase and the short overhang.

Will we ever see the BTCC and TCR Giuliettas head to head to see which is the fastest Giulietta in all of Britain? It seems unlikely, but maybe we can compare lap times once they’ve run on the same courses. An initial look shows that TCR and BTCC are roughly in the same ballpark, at least on Silverstone (based on the one race the TCR UK had so far in 2018 and the BTCC race in 2017), with maybe a slight edge for BTCC.

(Note that Aiden Moffat, from the TCR UK, is running with Laser Tools Racing in the BTCC as well, but here with a Mercedes Benz A-Class. Traitor.)

SA Endurance Series

Can you believe I missed a racing Giulietta the last time around? Even though I found no less than six major efforts? Well, I did, and I want to apologise. The one I missed (and found out about) runs in the Mopar South African Endurance Series, and the Giulietta is run by Arnold Chatz Cars Racing. I haven’t been able to find any technical details about this one, but it looks like one of the more modified ones. Lower ride height, big front and rear wing, but it does not have the fender extensions that make the TCR and BTCC Giuliettas almost square (as if the existing Giulietta wasn’t annoying enough to park in old parking garages already). Once more, it is fascinating how many different ways people find to reinterpret the Giulietta’s front bumper to make it look race-worthy… and I like almost all of them (with the obvious exception of the Mosca/Etruria TCT one) more than the stock version.

The car got an overall fourth place last season in its class and hopes to get even better this year. In the first race, it easily won its category (as one of three) and managed to beat the odd GT4 car, going for seventh place overall. That does not sound all bad.

BRSCC Alfa Romeo Challenge - Giulietta 8C

On the less serious but amazingly still british end of things, we get the BRSCC Alfa Romeo Challenge, a series exclusively for Alfa Romeos, primarily aimed at hobby drivers. It’s far from the only of its kind, and you will generally not find the Giulietta here because they’re still too new. The standard cars here are 156, 147 (both without facelift), and the odd 155 and even 75. Also some Fiat Puntos, which offends me on a personal level.

But they do allow odd cars in an “invitation” category, and being the home of Doctor Who, Top Gear, The Slow Mo Guys and James Bond, you can bet someone has come up with something odd. One Roger Evans is building a Giulietta with the drivetrain from a fire-damaged 8C Competizione he found on Ebay. Apparently the wheelbase is almost the same, so it’s at least in the generally possible category, but it’s still a project that has been going on for years now. The 2018 season preview on the series home page expresses hope that maybe it’ll run this year. Is that realistic? Your guess is as good as mine, but there’s no way I’m skipping a story like that.

Historic note: FIA Alternative Energies Cup

I’m just going to do this quickly because the official website is a nightmare, all the information I can find is italian, and frankly, I just don’t care that much. The FIA Alternative Energies Cup is a rally series for cars with alternative energy, meaning anything but gasoline or diesel. Team “Montecarlo Engineering Racing Team” has run a Giulietta converted to LPG equipment there, as well as a 1990s era GTV. Apparently they had some success in the “other” category. From what I can tell, it seems like the car is stock except for the LPG conversion and the paint scheme. But it has sponsor stickers, a number on the door and it appears in official result lists (I think), so it deserves to be on this list. From what I can tell on Wikipedia, the same series has also seen a number of wins by at least one LPG converted Mito.

Bonus!

Mitjet is a specification for small, low-weight, simple race cars for beginners, with standard technology, standard motor, but slightly different bodies that can, with stickers, be made to vaguely look like a Mercedes, Audi or BMW. In the italian Mitjet series, someone put stickers on their car that made it look vaguely like the current Giulia. It’s obviously not licensed, there are no logos and the four-leaf clover is on close inspection some other shape, but still, this is the closest that car has come to actual racing as far as I can tell. Still more than the Stelvio.

In news that is almost as exciting, long-time bottom-tier Formula 1 team Sauber is getting Ferrari motors this season, but thanks to a sponsorship deal, they’re putting Alfa Romeo stickers on the car and pretending Alfa Romeo has something to do with this. In the first race they had absolutely no success with this.

Speaking of TCR and italian touring cars, remember Tecnodom? They unsuccessfully ran a Giulietta in the TCS spec a while back, and with slightly more success two Mitos in the same class. Apparently they now also want in to the TCR business, specifically TCR Italy, and they want to run with an Italian car they built themselves. But since the Giulietta is taken (TCR regulations allow only one manufacturer for each TCR-spec car), they are using the new Fiat Tipo. I am obviously rooting for the Giulietta over the Tipo, but I guess I am rooting for the Tipo over all non-italian cars.

This weekend

This weekend (April 7th/8th) will actually feature a number of series that I’ll be looking at:

  • The first race of the BTCC
  • The first race of the WTCR
  • Less interesting but I’ll still look at the results, the second race of the Formula 1.

Alfa Romeo Giulietta in Racing

This is a best effort at a list of all the ongoing, finished or upcoming projects where the Alfa Romeo Giulietta (type 940, the current one) is used as a race car.

Why am I doing that? Well, mostly because I drive that car myself (not in racing) and I was curious. But also because I can already see that history won’t be kind to this car, and will probably be cited as one of the low points in this storied italian brand. This has nothing to do with the car itself: It’s fun to drive, practical, has great styling and good marks on stuff like reliability and safety. The problem is that in 2011, the Alfa 159 got cancelled so Fiat could build more of their Panda. This made the Giulietta, a fancy-looking but ultimately harmless VW Golf rival that never sold all that well, the flagship model of one of the greatest brands in automotive history. And that was just sad.

And then Alfa Romeo is one of the great motorsport brands, winning for example the first ever Formula one championship. Alfa Romeo is literally where Enzo Ferrari got his start, and they built legendary race cars ever since. Until about 2004, when the 147 Cup ended, and there has been no works-supported racing by them ever since. Now we get rumors that Alfa might join the DTM (Alfa clearly doesn’t want to), or that Alfa might join the Formula 1 again… by relabelling last year’s Ferrari engines that some smaller teams buy. That sounds reeeeeally exciting. But the thing is: If you want Alfa Romeo racing, you can already get it, and it’s great fun to watch. And they’re racing the Giulietta! In at least seven different projects! Here’s the ones I found:

TCR

TCR is a fairly new race car specification designed to be cheap but fun, with cars that look really dramatic but are accessible to a wide variety of teams. These cars are used in a number of different championships, some for one or a few countries and a flagship TCR International series. It is also used for a growing number of endurance race series.

The car specifications are based directly on the SEAT Leon Cup, and the cars are in practice mostly designed by independent tuners with sometimes more and sometimes less support from the manufacturer. Big wings and engines tuned to go up to 350 HP are also part of the concept. Currently eleven manufacturers have cars there, with more on the way. The Giulietta is one of them, with the race version designed by tuner Romeo Ferraris (despite the name no direct relation to either manufacturer). Alfa Romeo’s support seems to have started and ended at them saying „good luck, guys!“, while e.g. Volkswagen designs the complete car and sells it to others. Despite that, it’s proven capable of winning races.

Its first use was in the 2016 TCR International championship, with the Romeo Ferraris team, where it proved underpowered and in need of major work. It skipped a few races, then ended in a new but still not that competitive version. Over the winter, a lot more work happened, and it appeared again in the TCR Middle East, a very small series that was mostly european teams testing their new cars and drivers. Here it proved competitive, getting a number of second places. Now it’s again in the international series, and it managed to win the very first race of the season outright. Since then there have been more victories and podium positions. Performance has not been quite consistent enough to be a serious championship contender, but it’s getting there.

A note on team names: In 2016 it was „Romeo Ferraris“, then in the middle-east series it was „Mulsanne Racing“, and now it’s „GE-Force racing“. As far as I can tell most operation is handled by Romeo Ferraris either way, with key personnel such as Michele Cerruti (driver, team manager and one of the people who developed this car) appearing in all these places. However, Romeo Ferraris will also sell this car and so far managed two sales to other teams.

The first is Hungarian team Unicorse, who got one car, in red. It has participated in the Hungarian round of the international series in 2017 (no interesting placement). There were announcements that it would appear in the ETCC, an earlier touring-car championship that also allows TCR cars, but so far that does not appear to have happened.

Two more Giuliettas are run by V-Action in the TCR Italy championship, starting in the second half of 2017. One of those was also entered for the final round of the TCR Germany 2017 as a guest starter. Keep the TCR Italy in mind, we’ll keep coming back to that.

In short, as of right now, I’d say TCR has the queen of the Giuliettas, and if you’re interested enough to have read through here then you might enjoy following it. You can watch it all for free on Youtube.

TCT

Here it starts getting complicated. TCR Italy also allows a second type of car called TCT, which uses the same technical specifications as TCR, but only has a national instead of international type approval. I think this means that these are cheaper. In this context, TCT does not stand for twin clutch transmission, a feature you can get for normal road-going Giuliettas, which makes googling the whole thing a bit annoying.

A team called Etruria has developed a version of the Giulietta for this and raced two of them. The overall technical data seems very similar to what Romeo Ferraris did for TCR (the specification is the same, after all), but the resulting car looks quite different, with different body kit and air inlets. Personally, I don’t like it.

These cars ran in 2016 and some races in 2017 and were never all that competitive with the TCR cars. These cars have not raced at the same events as the TCR Giuliettas of V-Action, so sadly we don’t know which team did a better job. I’m not sure they will either, since at least one of the TCT Giulietta drivers switched to the TCR Giulietta.

Apart from that, TCT also featured the car driven by Gianni Giudici, a guy who will appear a lot on this list. If you look at the picture you’ll see that this car does not look like a TCR or TCT car at all, really. I don’t know what’s up with that, but if I had to guess, I’d say this: Giudici also ran a Giulietta in the VLN before that (we’ll get to that in a bit). What if he simply re-used that car here? Since it was built for a different series, it might not fit into the lower-spec TCS category (see below), so it ends up in TCT by default. The new rear wing is actually required for TCR; I’m not sure if it’s also required for TCT or Giudici simply decided it was a good idea. Either way the car didn’t prove very competitive with the cars of any category, and it hasn’t been seen since.

TCS

TCS is yet another specification for race cars found only in Italy. In 2017, the TCS is run as a separate (and mostly Seat-only) series, while in 2016 TCS cars took part in the TCR championship, classified separately. Before that, the story gets even more complicated; at one point, the TCS was actually the only type of car in the Italian Touring Car Championship.

The fundamentals of the car remain fairly similar, though: Relatively little changes; Cars get cars fitted with roll cage, safety equipment and some tuning to engine and suspension, but no massive wings or much bigger wheels or similar. There are further sub-divisions based on displacement, but the starter field tends to be small, so e.g. TCS 1.4 has generally only had one or maybe two entries these season.

This series has featured several Giuliettas. Sadly the sources for this are all in Italian, and they’re full of people announcing things that then never happen, so this part took a bit of digging and I don’t guarantee that my information is correct. That said:

This year there are no Giuliettas that actually ran. Leone Motorsport did announce that they wanted to run a car and got it featured on websites and on the March 2017 issue of an italian motor sports magazine, but the results show that it never actually took part in any TCS race.

The team (or its driver anyway) did run before that, though, in the 2016 series, where it managed a „not classified“ in the first free practice of the first race. Later they got two „not classified“ in actual races, with different drivers as well, and rather low placements in qualifying.

Distinct from that (I think), in 2015, Gianni Giudici also raced a Giulietta in the TCS series, in Monza. I don’t know whether it has anything to do with either his TCT or VLN efforts, and he didn’t do particularly well (9th place out of ten finishers after not classifying in the first race).

For fun: In 2016, the series also had some Alfa Romeo Mitos, a car that actually deserves all the dismissal as „not a true Alfa Romeo“. I’m exaggerating a bit, but a high seat position and steering with almost no feedback make for a boring drive and it’s amazing how little interior space it has compared to how large it is on the outside. Anyway, the race version, built by Tecnodom, comes in orange. Very orange. They often won the TCS 1.4 category even when they weren’t the only ones in it (meaning they placed in front of the one Fiat 500), and were at times even competitive with some TCS 1.8 racers. They did not run in the same race as any of the Giuliettas, though.

VLN

I’ve mentioned Gianni Giudici a few times before, but this seems to be his main effort regarding Giuliettas. The VLN is an endurance racing series, meaning races that take at least six hours, organised by the group of organisers that host the 24 hours race at the Nürburgring (in german: Veranstaltergemeinschaft Langstrecke Nürburgring; this catchy title produces the abbreviation VLN).

Annoyingly, this series does actually speak my language (i.e. german), but it doesn’t provide any useful way to find historic information about placements. From what I could gather from their news and fan sites, they ran at least one race where they managed a third place in their category - out of four. In presumably a different race, the car had quite a crash. It’s still notable because one of their drivers is none other than Nicola Larini, a legend among Alfa Romeo race car drivers from the good old DTM days.

It’s also important to note that many people, including Giudici, announce a lot of stuff that never ends up happening, and I tried my best to exclude that. If you look up Giudici you will find a lot of that. Among other things, apparently he wanted to build his own TCR Giulietta at some point, and besides Larini, he also wanted to hire Nannini for the VLN series. Also, he has some good graphic designers working for him, so there are a lot of pictures of Giudici Giuliettas that never actually existed in real life.

Meanwhile, this isn’t the only place Larini has turned up. The TCR series features a balance of performance system, where all cars get evaluated and then get ballast weight added or removed to bring them all to the same level. Larini is one of the two evaluation race drivers for this purpose, and used to be the only one last year.

BTCC

This is only an announcement at this point and thus something I’m a bit wary of, but I figured I should add it for completeness anyway. The British Touring Car Championship is a series that uses its own regulations, which are somewhere between TCR and the Silhouette racers of the current boring Alfa-less DTM. The shell of the car is fundamentally from the production vehicle, but anything underneath gets ripped out and replaced by standardised sub-frames holding standardised suspension equipment, and there is even a standard motor teams can (but don’t have to) use.

In 2018, team Handy Motorsport (who are currently running some Toyota) plans to run a newly built Giulietta in this series. The high standardisation should make this relatively straight-forward, and they have announced that they will use the standard engine instead of trying to tune the one in the production Giulietta. They already have a demo car, but that is just a standard production Giulietta with stickers.

British Giulietta fans may also want to look to the upcoming TCR UK, a TCR series in the UK that positions itself as an entry-level series below the BTCC. There has been no announcement yet (not that announcements mean anything), but at least some british drivers have test-driven TCR Giuliettas already.

Targa West

So here’s a weird thing that came up on Google: Apparently a Giulietta was used on an Australian rally, and did pretty well in the „show room“ class. This answers the age-old question of „Are there Alfa Romeos in Australia?“. The one and only source I could find was the racer bragging on an internet forum, so I’m reluctant to call this the most important thing ever, but it definitely counts for this list.

Die Ostdeutschlanderklärer

Es gibt eine neue Art von Lieblingsartikel in der deutschen Zeitungslandschaft: Wessis machen eine Rundreise durch die neuen Bundesländer und berichten dann den anderen Wessis, wie „die da drüben“ so ticken und wieso „der Osten“ immer noch nicht im vereinten Deutschland angekommen ist. Es gibt viele dieser Artikel, und der Fokus variiert - mal sind drei Statistiken dabei, mal dürfen ein paar „echte Ossis“ ihre Meinung kundtun, aber das Grundprinzip ist immer gleich. (Der, der mich gerade aufgeregt hat ist hier zu finden - hinter einer Paywall, aber letztlich nicht wirklich Geld wert). Man schreibt über gute Straßen auf denen schlecht gelaunte Leute laufen, über sozialistische Sozialstrukturen die jetzt fehlen, und natürlich immer über die AfD, ohne die man sich ja die ganze Mühe, mal „rüber“ zu fahren, nie machen würde. Nur für eine Erkenntnis ist nie genug Platz: Das solche Artikel ein massives Teil des Problems sind.

Eines der grundlegenden Probleme des vereinten Deutschlands ist, dass die Einheit nie in den Köpfen der Westdeutschen angekommen ist, und dass man auch nie den Versuch unternommen hat. Diese Artikel sind dafür ein wunderbarer Beleg. Bis heute gilt in den deutschen Medien die Westdeutschlandsvermutung: Die Geschichte und Erfahrung der BRD ist der Standard; die ehemalige DDR ist eine Anomalie die man extra erklären muss. Wenn man zum Beispiel auf der „einstiges“-Seite von Spiegel Online die wichtigsten oder interessantesten Punkte der deutsche Nachkriegsgeschichte ansieht, dann sieht man Bundesliga, RAF, Privatfernsehen, derzeit sehr viel über Seenotretter, und natürlich den ersten Supermarkt Deutschlands in Köln. Immerhin wird da kurz erwähnt dass es so was auch in der DDR gab.

Wenn die DDR vorkommt, dann als die Ausnahmeerscheinung. Beispielsweise gibt es in der Serie über Seenotrettung genau einen Artikel der sich damit beschäftigt und klar macht dass es heldenhafte Rettungen nur im Westen gab, im Osten war natürliches alles Stasi. Denn wenn die DDR erwähnt wird, dann kann das nur im Kontext der Stasi passieren.

Bundesdeutsche Kultur ist und bleibt westdeutsche Kultur, egal woher die Kanzlerin kommt und wie viel in den Filmstudios Babelsberg produziert wird. Geschichten wie die DEFA-Winnetou-Filme oder Urlaub am Plattensee sind es nicht wert erzählt zu werden, so lange man noch nicht den ersten Versprecher im ZDF erwähnt hat. Das westdeutsche Wirtschaftswunder wird gefeiert; die ostdeutsche Wirtschaft ist stets ein großer Haufen von Ineffizienz in denen allen Arbeitern alles egal ist. Dass die DDR in vielen Bereichen im Ostblock technologisch führend war, die zweitgrößte Wirtschaftsmacht und den höchsten Lebensstandard pro Kopf hatte, wird ignoriert. Dass die meisten Leute dort hart gearbeitet hatten und Stolz auf das Erreichte waren passt nichts ins Bild oder ist vielleicht auch nicht bekannt. Und das ganze Themengebiet Treuhand sieht man anscheinend nicht als relevant, obwohl man doch angeblich verstehen will, wie es den neuen Bundesländern heute geht.

Hier wäre mal eine These: Könnte es sein, dass die Menschen in den neuen Bundesländern, die sich so abgehängt fühlen, vielleicht auch in mancher Hinsicht abgehängt sind? Dass die Ostalgie lebt und blüht, weil es immer noch keine gesamtdeutsche Geschichtsschreibung und damit kein gesamtdeutsches Identitätsgefühl gibt? Und könnte es vielleicht sein, dass man über solche Themen reden muss, nicht nur wenn die AfD in den Nachrichten ist oder am dritten Oktober? Ich persönlich denke, dass so was sinnvoller sein könnte, als der nächste Artikel mit dem Grundauftrag: „Echte deutsche (Wessis) erklären dem echten Deutschland (Westdeutschland) wie die Ossis so ticken“.

Eclipse E4: A Critique

At work, I’m developing on a large internal application based on the Eclipse Rich Client Platform, which is basically the foundations of the Eclipse IDE made available for everyone else, too. As platforms go, it’s alright, I guess, but it does have its share of weirdness. For example, why do I have to dispose colors once I’m done with them!?[^1](#) But with a healthy amount of helper functions, it’s all manageable.

[^1](#): For compatibility with 8-bit indexed displays. Since this is nowhere near a target for our app, I generally don’t dispose colors.

We’ve recently finished porting the application from Eclipse 3.7 to the 4.x line (specifically 4.5, planning to upgrade to 4.6 once it’s out). This may seem a bit late, considering that the first official release of that line, Eclipse 4.2, came out four years ago. After working with the new code, though, I’m starting to think that maybe we’re still too early. Eclipse 4, or e4 for friends, features a massive rewrite of some of the core parts of the platform. And I think that it’s unfinished and in parts rather misguided.

Unfinished Business

Editors are gone, as are views, replaced with generic parts. I could get behind that, but gone as well are editor inputs objects. So telling an editor what file to open and making that decision persistent requires custom logic - and possibly a lot of that if you relied on the flexibility editor inputs gave you. You might see the MInputPart in the documentation, but apparently that was never implemented and is now deprecated. I think the only reason it’s still in the documentation is to taunt people. Of course, even if it did work, an URI based scheme only brings you so far if you have an editor that is supposed to receive objects from many different sources.

Many classic services have no e3 equivalent. For example, the IFocusService lingers around, but it’s very unclear whether I can rely on it once we’re e4 only.

No documentation

The documentation for new e4 specific stuff is largely missing or automatically generated and not helpful at all. I dare you to find any information in the documentation of MToolItem that isn’t obvious from the method names. What’s particularly missing is overview information: What do the different parts in the application model actually mean? What’s the difference between a MPart and a MPartDescriptor, and which one do I need? What’s the lifecycle of a MPart exactly? What is the point of context activation?

The best source of information are the rather terse Vogella tutorials, but these are task-oriented, not concept-oriented. If you want to do something that no Vogella employee wanted to do yet, you’re largely on your own.

Loose coupling. Like Spaghetti

I like the idea of dependency injection in theory, but I’m not sure about the practice. And I have no idea whether that is an Eclipse thing or not, but it still irks me.

The basic idea suffers from two problems: You want all dependencies passed in externally instead of getting them from some global nebulous set of singletons and what not. That’s valid. But you also don’t want gigantic constructors with 200+ arguments for complex classes. Not that you should have classes like that anyway, but a complex editor implementation can reasonably require access to most services Eclipse provides, even if it is just passes them on to helper objects.

The solution is more or less to put all that global state in a map, then pass that map as the argument to every constructor, and allow it to create new versions of the same map internally. Only they chose a more complex version that also has a lot of syntactic sugar, to make sure it runs really slowly. Eclipse has a reputation to maintain, after all.

As a result, you still have no clear idea what part of the application needs access to what unless you search for type names, and now you also have no idea where it’s coming from. It’s not so much that it’s bad per se, but it definitely does not realize the advantages that dependency injection was meant to provide.

And while we’re here already: The way event handling is done via dependency injection is just insane.

Horribly bad ideas

I honestly don’t get the application model, and not just because there is no real documentation on it. I have no idea why it’s here at all, what problems it’s meant to solve, and how it thinks it has solved them. And most of all, I don’t get why so many Eclipse developers are so damn happy about it. “Look at this”, they go, “I change the window’s title in the application model, and the window’s title actually changes! And I can even do that… at runtime!” And then they’re surprised when I don’t applaud.

But hey, whatever, I’ve been able to write helper methods and helper classes to get around any problem in e4 yet. The real problem of the application model is that it seamlessly merges user state and declarative application code into one giant ball of mud. And then it makes that mud persistent, which is just as unpleasant as it sounds.

Because suddenly, changes in user state get the opportunity to conflict with changes to declarative application code. Will my new context menu show up if the user has moved any editors around? For that matter, will my old context menu continue to show up? Surprisingly often, the answer is no, and never for any good reason I could discern. By merging what absolutely had to stay separate, they built a giant bug-producing machine.

And they know it. The closest thing to an official recommendation is to simply disable persistent user state. Only during development, in theory, but they sure don’t provide a better way to deal with the problem once I release an update of my app. In my opinion, this wins the very highly contested price for least user-friendly thing ever done in Eclipse.

Oh, there’s a workaround, but it’s one that nobody will tell you about, and it’s the most WTFy thing I ever implemented, to the point that I still can’t quite believe that this is what it takes. But I’m doing it anyway: I wrote my own code for serializing the application model to an XML file and back, making 100% sure that I only touch the parts that the user can change (i.e. what editor is where), but not the parts that are declarative program code. It’s been a few months since I implemented this thing, and I still sometimes wake up in the night and shout “There must be something better! Some simple flag I missed somewhere!” Do drop me a line if you found it.

Handlers

Finally, command handlers. The new way of writing them is good. Having the validation logic in the handler itself as opposed to an XML file somewhere is better. Good choices all around.

Less good: Why can I have only one handler per command per part? I kind of need several different “delete” handlers for different parts of some more complex editors, thank you very much, and I have only limited interest in creating one giant monster handler that contains the delete logic for absolutely everything. Of course it’s possible to work around that; I’ve managed to work around every little thing in e4 except the application model weirdness so far. In this case, write one handler that creates an internal list of other handlers, then asks each whether it can execute and the first one to shout yes gets to do the job. But I don’t get the logic that says I have to write stuff like that.

I also don’t quite get why some handlers exist and others don’t, but that’s less of an issue.

No tool support

Why is it that so many new things are released without good tool support? Do people somehow think that’s optional? I don’t see it. For example, to port from e3 to e4, there is an awful lot of getting information out of one XML file, translating it, and putting it into a different XML file. It’s boring, tedious and straightforward. And the one and only tool I found for that is one I wrote myself (No; you can’t have it. Nothing personal, but I have no idea what hoops I’d have to jump through to get something developed internally released as Open Source here).

The tools that do exist regularly stop accepting copy and paste and have no useful search. Most of them aren’t part of the platform proper, but require a separate download. It all feels very experimental. I went back to editing the XML files by hand.

Overall

Overall, e4 is still very far from done, and that’s frustrating. But in a few places, even the core is wrong or at least questionable. And while it looks like Eclipse 4.6 will solve a few issues I’ve been having (Generics for Databindings! Hooray!), they don’t seem even interested in solving the big ones.

Agents of SHIELD retrospective

So I wrote this a while back, posted it elsewhere, then forgot about it, until I was reminded the day before yesterday that this wouldn’t be all out of place here, either. So let’s talk about a TV show that I found really disappointing once more: Agents of SHIELD.

At the start, I was really excited about it. Then I was disappointed. Then I got angry. Then it got better again, so by now my feeling is mostly “meh”. And what a big “meh” it is; I’ve been putting this off for more than a week after the final episode of season 1 aired and I still barely have the energy to get through this post.

The show certainly means well, although it really tries more to be a Joss Whedon show than a Marvel property. I guess the writers had prepared a checklist pretty much like the one I posted above and were trying their best to fast-track through it. This became very apparent later on when the characters basically outright stated “That bonding over a common enemy thing? Might as well do that now, nothing else interesting going on this episode”. And never forget that “the Bus”, their plane, is for all points and purposes the Serenity from Firefly. Similar looks, capabilities, identical internal layout including which character owns what space.

What it lacks is the depth and complexity that real Whedon shows have. For all of the characters, their motivations, strengths and flaws can be described in two sentences at most. Instead the show tries to keep our interest with layers of mystery and question after question. Lost has been getting a lot of shit for this, but there can be no doubt that Lost did it way better. The story of Coulson’s resurrection, for example, is in the end neither particularly complicated, nor does it mean much for his character both in how it happened or in how he reacts to it. It’s also kind of stupid. But in Agents of SHIELD, kind of stupid is probably the biggest compliment you can make, considering that the show is so very frequently extremely stupid.

For example, take the episode 0-8-4. How many trained Agents of SHIELD and peruvian military police does it take to stop one Land Rover filled with I guess about three or four insurgents? I don’t know, but it’s gotta be more than 6 and about 15, respectively, because they try and fail. Then there are things like where the “adorable quirky” (annoying) scientist guy gets to go out in the field with the actually competent person. Yeah, Fitz isn’t field-rated, I get it. But are you seriously telling me he’s never even seen a spy movie before?

It all works out for the heroes because the villains are just as stupid. Take the excavator thing: The bad guys didn’t need an excavator to open the secret SHIELD truck; it had normal doors (with some boxes stashed behind to throw anyone who opened the doors of track, so clearly they weren’t welded shut). They don’t use the excavator to open the secret compartment within the truck either; that job goes to a blowtorch. And they paid for the excavator in gold bars, which seems hardly reasonable. Of course the gold bars can be traced back to the main villain’s gold mine, something that a gold mine owner probably should have known. Oh, and the entire plot hinges on Malta not being a member of the UN or EU (it is, of course, a member of both organizations). This is some serious multi-level idiocy going on here.

Later episodes pick up the slack quite a lot, and at turns even make fun of themselves, for example when Skye reveals that her name (given to her at the orphanage) was Mary Sue, or my personal favorite: When Agent Hand sees all of Team Coulson’s incompetence and deduces they must have been Hydra all along.

But at that point the show was doing exactly the same thing as „Captain America: The Winter Soldier“, and boy does it not hold up. I get that a TV show with a limited budget can’t look as good as the Hollywood blockbuster, but the longer running time should allow it to have better human drama. It doesn’t. Captain America wins on all fronts here with Cap as an all-around good guy who never actually becomes boring, an interesting friendship with Black Widow, inspirational speeches that actually work on me (and I’m a long-standing cynic) and real pain over the thing with Bucky. Even Falcon with his amazing lack of screen time has more personality than most of the Agents of SHIELD group.

Also, somewhere before there, I think I’ve kind of stopped caring. I mean, the final resolution to “how and why did Coulson get resurrected?” was “Nick Fury did it because Coulson is a nice guy whom everyone likes”. Seriously. And my reaction to that was “eh, could have been worse”.

Finally, to wrap this up: The show doesn’t look that good. Don’t get me wrong, it looks perfectly okay in a late 1990s kind of way. But we live in an age where Heroes, Lost, Breaking Bad or further out of the field Gossip Girl have established strong visual identities on TV. Whether it is through the choice of color, editing, or other things, these shows have character. SHIELD is, visually, less exciting than Bones or Castle.

In total: Agents of SHIELD is really not worth the bother. The later episodes are at least not offensively stupid anymore, but there isn’t really anything else to get passionate about either. Season 2 has been announced, but I’ll probably skip it. Here’s hoping that these Netflix shows or Agent Carter do better.

DAG vs. Tree

I am wondering whether the data structure “tree” might be a giant mistake in the history of computer science. Almost everything that can be saved in a tree is really a DAG.

Some background: A tree is a data structure consisting of nodes, where each node has one or no parent (this is technically a forest; a tree has only one node with no parent). Each node has children; specifically the nodes that have that node as its parent. Circles are forbidden.

The best-known example for this are computer file systems. Each file is in a folder. That folder is in another folder and so on, until you reach the drive, which forms the root of the tree. Mail lists and some kinds of forums and comment systems also work like that: Each post is a reply to exactly one previous post (at most) and gets shown a layer below. However, tree structures predate computer science. The whole field of Taxonomy as well as linguistics, both with their habit of grouping things in families and sub-families and so on, do exactly the same thing.

It’s easy to process trees automatically, for example to find all children. You can also identify each child uniquely through a given path.

There is just one problem with them: Reality does not work like that. In most real-world cases things, can have multiple parents. A classic example: What folder does a file fit in? Likewise, in discussions, sometimes you want to reply to several posts that make similar points at once. In linguistics, languages don’t really belong to one family exclusively, they have influence from lost of different ones.

To get around that, one either has to pick a parent explicitly, or split the object. Splitting works for discussions well, for files sometimes, for e.g. classification of car types not at all. In either case, information about the semantic relationships is lost.

The answer to that is called DAG, for Directed Acyclic Graph. The rules are almost the same as a tree, only a node can have more than one parent as well. An example: Media library apps like iTunes allow a song to appear in several playlists at once.

DAGs represent the examples here naturally. A file can be in two folders, for example. Since a tree is a special case of a DAG, it’s also easily possible to represent the cases that actually are tree-like.

In reality many systems already include hacks to create DAG structures. For example, thanks to Hard links, most file systems actually are DAGs, they just don’t show it that often. Symlinks, Aliases and Windows shortcut files all create a “second-class parent”, which explicitly stores the removed relationships.

Another approach is through tag systems, where a file, blog post or similar can have an arbitrary number of tags. This structure is actually just a two-level DAG. That may seem like a solution, but since it isn’t possible to form relationships between tags, and because many use cases of trees can’t be modeled that way (for example discussions), it isn’t a full replacement.

Admittedly, from a technical point of view, DAGs are much harder. For example: A tree is always planar, which means you can easily show it on a screen without crossing lines, something that is not guaranteed for DAGs. Enumerating all elements without repeating any is a lot harder. Objects can be reachable through two completely distinct paths - which is of course the goal, but it means you have to look at the graph to answer the question “do these two paths end at the same object”, which is trivial with trees.

Most importantly, I know no really good UI to show and manipulate DAGs. For example, the opposite to a tree structure in forums is not a DAG, it’s a simple list ordered by time, and the relationships between posts have to be shown manually in the text.

(Of course, this does not affect the use of trees in cases where end users don’t see them, for example search trees, heaps and so on.)

What do you think? Due to too much spam, I’ve disabled the comments here for now, but you can reach me on Twitter via @zcochrane.