Lego Mindstorms Simulator

A while ago, [I wrote about my bachelor thesis here][1stpost], which was about creating a simulator for Lego Mindstorms robots, to use for teaching programming to school students. When I updated [my page][wiki] in the institute’s Wiki, I realized that I had never written anything here about the conclusion of the project. That was unacceptable.

The thesis is done now, of course, but I did some more work on the simulator. Here is a current screenshot:

![The virtual robot in a virtual environment.](/bachelorthesis/finished_screenshot_small.jpg)

The basic features haven’t changed much since the first blog post: The simulator executes Lego NXT Bytecode. This results in commands for a very, very simple physics layer, which moves the robot according to them and handles collision detection. It also calculates the values for the various sensors, and feeds those back to the bytecode emulator. At the upper end, there is a sound output and input (for the microphone sensor) layer, and of course a 3D interface, which is not the most graphically advanced, but does its job.

From a technological point of view, there is not a lot truly exciting in there, just a lot of writing code. The entire thing is written in C++, using OpenGL for graphics and OpenAL for sound. For interfacing to the operating system, it uses SDL. The arguably most interesting part is the network support: In this mode, a network layer sits between the emulator and the physics, and transmits all output commands (such as “turn on motor A”) to the server. This then moves the robots and sends the new positions and sensor values back to the clients. Again, not the most advanced implementation of its kind, but for LAN, it works well enough. Finally, there are convenience features, such as the ability to lift the robot, turn it and put it down elsewhere.

The actual thesis is mostly a technical documentation of the code. It turned out to be way too long, but nobody thought this was a problem. In the end, for every corner of the simulator, I can point out how this or that could have been done much better, much more detailed and so on. In the end, though, it is just a tool for school children, and it really does not need to be much more sophisticated. My advisors and the examiners were very pleased with the result, and gave me a grade of 1,0 (the best possible). I’d be lying if I said that I didn’t feel quite proud of that.

But there’s more

In fact, they liked it so much that they offered me a student job. So since some time, I’m working as programmer for the research group. My main task: Improve the simulator. Apart from some general improvements, there are two major projects.

The simple one is about developing new forms to control and program the real robot, e.g. from mobile phones and so on. Since the real robot hasn’t really gotten any cheaper, it would be great if the simulator could be used for those projects as well, so I altered it enough that it can be controlled remotely, instead of executing its own code. Most of the necessary work was already done, thanks to the network mode. The main changes were related to the UI. The projects using this haven’t integrated this yet, so all I can show right now is a small sample Java application to control the simulator.

![It has fields to choose the server, fields that show the current values of the sensors, and fields to control the motors.](/bachelorthesis/robosim_remote_small.jpg)

It’s not pretty, but it’s really just an example, so it will have to do.

An HTC Desire phone, with the simulator running on it.

The other big project: The simulator has to work on tablets as well, and since the Computer-Supported Learning Research Group (where I work) is the more-or-less official Android institute, this meant porting the simulator to Android. This is still a work in progress, but the basics are all there. As you may know, I’m more familiar with iPhone myself, so it is an interesting experience for sure. I’ll write more about this later. What I can tell you is that this won’t be a glowing recommendation; the Native Development Kit, which you need to write C++ apps in the Java-centric Android world, is still rather raw.

An Apple iPhone with the simulator on it.

In the interest of fairness, and mainly to test multitouch even without the hardware (the Android phone above belongs to the research group, not me), I also have a fully functional iPhone port. If you paid close attention, you may have noticed that neither the iPhone 3GS nor the HTC Desire are tablets, so to speak. Sadly, neither the research group’s Android tablets nor my own iPad 2 are here yet. The phones are quite suitable to test how well the program works on the particular operating system, though. Larger screens are not much of an issue after that.

Finally: Yes, there are plans to release the simulator to the public, as Open Source. When and how exactly is not yet fixed, but I’ll post here on this blog and [on Twitter]( once it’s up and running.

Written on April 14th, 2011 at 04:23 pm


  1. Posted 2 July 2011


    thank you

  2. Posted 10 October 2011

    Thomas Daniels

    Hallo Herr Kammer,

    habe mit großem Interesse Ihren NXT-Simulator gesehen und frage mich, ob man auch aus der graphischen Umgebung heraus einen bytecode erzeugen kann, um diesen dann mit Ihrem Simulator auszuführen.
    Geht das?
    Über zweckdienliche Hinweise würde ich mich sehr freuen.

    Freundliche Grüße,
    Thomas Daniels
    Carl-Fuhlrott-Gymnasium Wuppertal.

  3. Posted 14 December 2011

    Stefan Steinborn

    Bin sehr begeistert - funktioniert bestens!

    Ich wäre an einer Weiterentwicklung des Programms sehr interessiert, vor allem das Speichern der Einstellungen (z.B. Sensor-Konfiguration) fehlt mir (oder hab ichs nur nicht gefunden?).

    Open Source wär auch gut, dann könnte ich es selber machen :-)


    Viele Grüße von Stefan Steinborn.

  4. Posted 19 December 2012

    Piet Seibt

    Finde ich ein klasse programm.
    Wo kann man es Dowanloaden?

New comments can no longer be posted because it got to annoying to fight all the spam.