Enda O'Donohoe fixed two bugs regarding the UTF-8 decoder. Thanks for that!

Support for Jackson 2.0 was greatly supported by James Roper. Thanks again for your contributions, James!

Support for older Jackson versions will be dropped with bson4jackson 2.0. If you're looking for a version supporting the Jackson 1.x branch, then please download bson4jackson 1.3.0.

More information

For a complete description of bson4jackson (including how to download it) have a look at my tutorial.

Gergő Ertli has fixed the support for the ObjectId type. Object IDs are used as the primary key for MongoDB documents.

Support for the UUID type has been added by Ed Anuff. He added a new module which can be registered to Jackson's ObjectMapper:

ObjectMapper om = new ObjectMapper(new BsonFactory());
om.registerModule(new BsonUuidModule());

Thanks to the contribution by James Roper the BsonParser class now supports the new HONOR_DOCUMENT_LENGTH feature which makes the parser honor the first 4 bytes of a document which usually contain the document's size. Of course, this only works if BsonGenerator.Feature.ENABLE_STREAMING has not been enabled during document generation.

This feature can be useful for reading consecutive documents from an input stream produced by MongoDB. You can enable it as follows:

BsonFactory fac = new BsonFactory();
fac.enable(BsonParser.Feature.HONOR_DOCUMENT_LENGTH);
BsonParser parser = (BsonParser)fac.createJsonParser(...);

Apart from that, a lot of other minor bugs have been fixed. The library has been tested with Jackson 1.7 up to 1.9.

More information

For a complete description of bson4jackson (including how to download it) have a look at my tutorial.

Short disclaimer: in this article I'm not going to describe how to actually build projects with Buckminster. This article is rather about how to install Buckminster, so it is able to handle Scala projects like any other project. If you need a more general description, please refer to the Buckminster website.

Why Buckminster?

Let's first talk about why I prefer Buckminster over other solutions. Compared to other build tools, Buckminster has one great advantage: it provides an environment very similar to the one you're developing in—the Eclipse SDK. You can see Buckminster as a lightweight Eclipse SDK without all this distracting UI stuff, but with a command line interface instead. However, Buckminster still has a workspace where you can import your projects into, and of course a target platform which defines the environment for your RCP application. Running a Buckminster-driven build with continuous integration systems like Hudson/Jenkins is also very easy, since all you have to do is to create some configuration files—a resource map and a CQuery—and then run a simple command on the shell like buckminster build. However, there there is also a plug-in available for Hudson/Jenkins, which automatically downloads and installs Buckminster if necessary. For more information on the usage please refer to the great book Buckminster, The definitive guide by Henrik Lindberg and Thomas Hallgren.

Buckminster is also one of the few tools that are able to build a full-featured Eclipse RCP application. There are extensions for other build tools like Ant or—since we're talking about building Scala projects here—sbt. For example, Bnd is one of them. It allows building OSGi bundles without the need for creating a MANIFEST.MF file manually. Besides, you can always package your OSGi bundles yourself using the jar tool, but that is really not enough for serious RCP development.

If you want to build your RCP application headlessly, you have to use a tool that is really aware of what it is building. It has to understand the complex dependencies between your projects and the OSGi bundles from the target platform you use. Apart from that, it has to be able to parse files like MANIFEST.MF, build.properties, plugin.xml as well as configurations stored in .product or feature.xml files.

As far as I know, there are only three products available which support all these features—or at least most of them: PDE Build, Maven Tycho and of course Buckminster. I mention PDE Build here since it was formerly the recommended way to build RCP applications, but has now been superseded by Buckminster. PDE Build is also only a set of Ant scripts and does not support all of the aforementioned features. Tycho, on the other hand, has recently become a mature solution. It integrates very tightly into the well-known Maven build tool. This makes it very easy for people who are using Tycho for the first time, but already know Maven. Apart from that, Tycho contains some cool features like archetypes which automatically create pom.xml files for the bundles to build.

Nevertheless, I found Tycho always being slower than Buckminster—at least for very large projects with a lot of bundles. I don't know if this has something to do with the support for incremental builds in multi-module projects (Is this still a problem with Maven 3 anyhow?) or with the way Tycho is resolving the target platform and how it calculates bundle dependencies. Apart from that, Tycho builds projects in a slightly different environment than they are developed in. This eliminates all of the aforementioned advantages of having a build environment that is very similar to the Eclipse SDK—as I said, this is in fact what you have when you use Buckminster. These disadvantages make it of course not impossible to build RCP applications, but they are the reason why I would prefer Buckminster over Tycho. Of course, this may change eventually, since Tycho is continuously developed.

How to install Buckminster and the Scala IDE normally

There is a Buckminster plug-in for the Eclipse SDK which adds a context menu with build tasks to all workspace projects. You simply install it using the p2 update manager. The current update site can be found at http://www.eclipse.org/buckminster/downloads.html.

Installing this plug-in and then installing the Scala IDE is very easy and lets you build Scala-featured OSGi bundles in a few minutes. Nevertheless, setting up Buckminster with the Scala IDE for headless operation is a little bit harder. If you follow the documentation very strictly, you will do the following things:

First you have to download the latest version of the p2 director application: director_latest.zip. You can then install the Buckminster RCP application with the following command:

$ director -r <url to headless repo> -d <buckminster install folder>
      -p Buckminster -i org.eclipse.buckminster.cmdline.product

This will download Buckminster from the supplied update site—which can be found on this site—and install it into the given folder. After that you will most likely want to install some additional features in order to enable headless builds. Change into the directory where you installed Buckminster and run the following commands, for example:

$ buckminster install <headless repo> org.eclipse.buckminster.core.headless.feature
$ buckminster install <headless repo> org.eclipse.buckminster.emma.headless.feature
$ buckminster install <headless repo> org.eclipse.buckminster.git.headless.feature
$ buckminster install <headless repo> org.eclipse.buckminster.pde.headless.feature

Additionally, you have to install the Scala IDE into Buckminster. Go to http://download.scala-ide.org and select the update site that fits your needs. Then run the following command:

$ buckminster install <scala ide repo> org.scala-ide.sdt.feature

Cannot complete the install because one or more required items could not be found.
[0]Software being installed: Scala IDE for Eclipse 2.0.0
     (org.scala-ide.sdt.feature.feature.group 2.0.0)
[0]Missing requirement: Scala IDE for Eclipse 2.0.0
     (org.scala-ide.sdt.feature.feature.group 2.0.0)
   requires 'org.eclipse.ui.navigator 0.0.0' but it could not be found

Oops! What happened? It seems like the Scala IDE requires user interface bundles, but my Buckminster runs on the command line!

Workaround

In order to work around this problem, you have to install Buckminster a little bit differently. First, you need the p2 director application again, but this time you will install the Eclipse SDK:

$ director -r <eclipse sdk repo url> -d <eclipse install folder>
      -profileProperties org.eclipse.update.install.features=true
      -p SDKProfile -i org.eclipse.sdk.ide

After that, change into the directory where you just installed Eclipse and run the following commands to install Buckminster:

$ eclipse -nosplash -application org.eclipse.equinox.p2.director
      -r <url to buckminster headless repo>
      -i org.eclipse.buckminster.core.headless.feature.feature.group
      -i org.eclipse.buckminster.emma.headless.feature.feature.group
      -i org.eclipse.buckminster.git.headless.feature.feature.group
      -i org.eclipse.buckminster.pde.headless.feature.feature.group

Please note the additional .feature.group at the end of each feature to install.

Finally, you can install the Scala IDE:

$ eclipse -nosplash -application org.eclipse.equinox.p2.director
      -r <scala ide repo>
      -i org.scala-ide.sdt.feature.feature.group
      -i org.scala-ide.sdt.weaving.feature.feature.group

This should succeed without errors.

From now on you can launch Buckminster with the following command:

$ eclipse -nosplash -application org.eclipse.buckminster.cmdline.headless

You may get the following exception when the Scala IDE bundles are started:

java.lang.IllegalStateException: Workbench has not been created yet

This is due to the fact that you don't have a UI and thus no workbench. Fortunately, there's an (undocumented?) system property you can set to avoid this exception. Simply add the following line at the end of your eclipse.ini:

-Dsdtcore.headless

This tells the Scala IDE that you're running it in headless mode and that it should avoid accessing the workbench.

Summary

In this article I described how to headlessly build Scala projects with Buckminster. I talked about why I prefer Buckminster over other build tools and presented a workaround for installing Buckminster and the Scala IDE with the command line.

After installing Buckminster this way, you will have a full fledged tool to build Eclipse RCP applications including Scala plug-ins. However, installing a complete SDK would not be necessary if the Scala IDE bundles did not depend on Eclipse's UI. It's probably possible to create a feature containing only the core plugins like it has been done for the Groovy plug-in, but that's something the Scala IDE team has to figure out in the future.

Nevertheless, a complete Eclipse SDK has also an advantage worth to mention: everything that works on your local machine will also work on your headless build server. Actually, you may even install your local Eclipse the same way as described above. Then you can absolutely be sure that your application is built in the same environment as it is developed in. This can make debugging a lot easier.

BSON is binary encoded JSON. The format has been designed with fast machine readability in mind. BSON has gained prominence as the main data exchange format for the document-oriented database management system MongoDB. According to the JVM serializers benchmark Jackson is one of the fastest JSON processors available. Apart from that, Jackson allows writing custom extensions. This feature can be used to add further data exchange formats.

bson4jackson

This is the moment where bson4jackson steps in. The library extends Jackson by the capability of reading and writing BSON documents. Since bson4jackson is fully integrated, you can use the very nice API of Jackson to serialize simple POJOs. Think of the following class:

public class Person {
  private String _name;
 
  public void setName(String name) {
    _name = name;
  }
 
  public String getName() {
    return _name;
  }
}

You may use the so-called ObjectMapper to quickly serialize objects:

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import com.fasterxml.jackson.databind.ObjectMapper;
import de.undercouch.bson4jackson.BsonFactory;
 
public class ObjectMapperSample {
  public static void main(String[] args) throws Exception {
    //create dummy POJO
    Person bob = new Person();
    bob.setName("Bob");
 
    //serialize data
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    ObjectMapper mapper = new ObjectMapper(new BsonFactory());
    mapper.writeValue(baos, bob);
 
    //deserialize data
    ByteArrayInputStream bais = new ByteArrayInputStream(baos.toByteArray());
    Person clone_of_bob = mapper.readValue(bais, Person.class);
 
    assert bob.getName().equals(clone_of_bob.getName());
  }
}

Or you may use Jackson's streaming API and serialize the object manually:

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.JsonFactory;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonToken;
import de.undercouch.bson4jackson.BsonFactory;
 
public class ManualSample {
  public static void main(String[] args) throws Exception {
    //create dummy POJO
    Person bob = new Person();
    bob.setName("Bob");
 
    //create factory
    BsonFactory factory = new BsonFactory();
 
    //serialize data
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    JsonGenerator gen = factory.createJsonGenerator(baos);
    gen.writeStartObject();
    gen.writeFieldName("name");
    gen.writeString(bob.getName());
    gen.close();
 
    //deserialize data
    ByteArrayInputStream bais = new ByteArrayInputStream(baos.toByteArray());
    JsonParser parser = factory.createJsonParser(bais);
    Person clone_of_bob = new Person();
    parser.nextToken();
    while (parser.nextToken() != JsonToken.END_OBJECT) {
      String fieldname = parser.getCurrentName();
      parser.nextToken();
      if ("name".equals(fieldname)) {
        clone_of_bob.setName(parser.getText());
      }
    }
 
    assert bob.getName().equals(clone_of_bob.getName());
  }
}

Optimized streaming

One disadvantage of BSON is the fact that each document begins with a number denoting the document's length. When creating an object this length has to be known in advance and bson4jackson is forced to buffer the whole document before it can be written to the OutputStream. bson4jackson's parser ignores this length field and so you may also leave it empty. Therefore, you have to create the BsonFactory as follows:

BsonFactory fac = new BsonFactory();
fac.enable(BsonGenerator.Feature.ENABLE_STREAMING);

This trick can increase the serialization performance for large documents and reduce the memory footprint a lot. The official MongoDB Java driver also ignores the length field. So, you may also use this optimization if your bson4jackson-created documents shall be read by the MongoDB driver.

Performance

Version 1.1.0 of bson4jackson introduced support for Jackson 1.7 as well as a lot of performance improvements. At the moment, bson4jackson is much faster than the official MongoDB driver for Java (as of January 2011). For serialization, this is only true using the streaming API, since Jackson's ObjectMapper adds a little bit of overhead (actually the MongoDB driver also uses some kind of a streaming API). Deserialization is always faster. The latest benchmark results can be reviewed on the following website:

https://github.com/eishay/jvm-serializers/wiki

Compatibility with MongoDB

In version 1.2.0 bson4jackson's compatibility with MongoDB has been improved a lot. Thanks to the contribution by James Roper the BsonParser class now supports the new HONOR_DOCUMENT_LENGTH feature which makes the parser honor the first 4 bytes of a document which usually contain the document's size. Of course, this only works if BsonGenerator.Feature.ENABLE_STREAMING has not been enabled during document generation.

This feature can be useful for reading consecutive documents from an input stream produced by MongoDB. You can enable it as follows:

BsonFactory fac = new BsonFactory();
fac.enable(BsonParser.Feature.HONOR_DOCUMENT_LENGTH);
BsonParser parser = (BsonParser)fac.createJsonParser(...);

Support for Jackson 2.0

Support for Jackson 2.0 has been introduced in bson4jackson 2.0. Support for older Jackson versions has been dropped ever since. If you're looking for a version supporting the Jackson 1.x branch, then please download bson4jackson 1.3.0.

Download

Pre-compiled binaries

Pre-compiled binary files of bson4jackson can be downloaded from my GitHub-Repository. Additionally, you will need a copy of Jackson to start right away.

Maven/sbt/buildr

Alternatively, you may also use Maven to download bson4jackson:

<dependencies>
  <dependency>
    <groupId>de.undercouch</groupId>
    <artifactId>bson4jackson</artifactId>
    <version>2.0.0</version>
  </dependency>
</dependencies>

If you're using sbt, you may add the following line to your project:

val bson4jackson = "de.undercouch" % "bson4jackson" % "2.0.0"

For buildr use the following snippet:

compile.with 'de.undercouch:bson4jackson:jar:2.0.0'

This method still works. The OSGi bundles containing the Scala library and compiler are delivered together with the Scala IDE for Eclipse. This project is now not affiliated with the EPFL anymore and is developed mostly independently from the Scala tool chain. Hence, the OSGi bundle names have changed and so the Ant script from my previous post does not work anymore.

The following things have to be changed:

• The symbolic name of the Scala library OSGi bundle now starts with org.scala-ide.scala.library_
• The compiler is now provided by a bundle whose name starts with org.scala-ide.scala.compiler_
• The Scala Ant task now resides in scala-compiler.jar which is provided by the compiler bundle

I created a new script that can be downloaded here:

customBuildCallbacks.xml (4.0 KiB)

The simplified commands can be downloaded from my Github repository. Instructions for installing and usage can be found in the README.

This is a short summary of the most used commands:

add

Verbose add (executes git add -v and then git status). All parameters will be forwarded to git add.
Example:
add -p foobar.txt

commit

Alias for git commit

gadd

Pipes the output of git ls-files to grep to add only those files that match a given pattern. All parameters (including the pattern) will be forwarded to grep. Like add this command is verbose and calls git status at the end.
Example:
gadd -i '\.java$'

log

Alias for git log

pull

Executes git pull --rebase and then displays all commits since the last pull in a short format.

push

Alias for git push

st

Alias for git status

The commands have been tested unter msysgit, but they should also work on other platforms. A complete list can be found in the README.

In the future I may add further commands. I would appreciate any feedback.

Finally, I want to thank Simon Templer and Simon Thum for their ideas!

Internationalization with Lift

First of all, let me summarize the possibilities of internationalization with Lift: String resources have to be placed in so-called "property bundles". These are normal Java property files like the following:

title=My Website
yes=Yes
no=No
deletequestion=Do you want to delete everything?

The name of a property bundle always starts with lift, but different suffixes in the ISO format will be appended describing the file's language and country. For example, the file lift_en.properties contains English string resources, lift_de.properties contains German ones and lift_en_US.properties US American English. The prefix lift can be specified in the web application's boot loader:

LiftRules.resourceNames = "anotherPrefix" :: Nil

The variable resourceNames is a list of strings. Therefore, resource bundles can be divided into multiple files with different prefixes. This is useful for larger projects with a lot of string resources.

After the resource bundles have been created, strings in HTML templates can be translated using the <lift:loc locid="..." /> tag. For example, following template:

<html>
<head>
<title><lift:loc locid="title" /></title>
</head>
<body>
<lift:loc locid="deletequestion" />
<button><lift:loc locid="yes" /></button>
<button><lift:loc locid="no" /></button>
</body>
</html>

will be translated to the following HTML code (using the properties file from above):

<html>
<head>
<title>My Website</title>
</head>
<body>
Do you want to delete everything?
<button>Yes</button>
<button>No</button>
</body>
</html>

In the Scala source code the stateful object net.liftweb.http.S can be used instead. It provides the method ? which takes a string and looks for the corresponding translation in the resource bundle:

def snippet(xhtml: NodeSeq): NodeSeq =
  <p>{ S ? "deletequestion" }</p>

Tip: If you want to translate a tag's attribute in a HTML template (e.g. the attribute value of the inputtag), you should copy the respective code into a snippet. Therefore you can use the S object to translate the attribute:

def inputButton(xhtml: NodeSeq): NodeSeq =
  <input type="button" value={ S ? "yes" } />

Resource Bundles in the OSGi environment

Lift tries to find resource bundles in the classpath using the method getClass.getClassLoader.getResourceAsStream(). In an OSGi environment this is not going to work, because the classloader of the net.liftweb.lift-webkit bundle is used.

As mentioned above, this can be solved with a little workaround. A resource bundle factory has to be added in the application's boot loader:

LiftRules.resourceBundleFactories prepend {
  case (basename, locale) => ResourceBundle.getBundle(basename, locale)
}

Therefore, the resource bundles will be loaded by the classloader of the bundle that contains the boot loader.

Conclusion

The means for internationalization provided by Lift are comprehensive enough for most websites. If something is not translatable using the <lift:loc> tag (e.g. attributes) or if more control over the process is needed, snippets can be used instead. In OSGi environments a resource bundle factory has to be created, so property files will be loaded by the correct classloader.

This article relates to an old version of the Scala IDE for Eclipse. Information about the latest version can be found in a newer post!

First of all, you need an Ant script that can be hooked into the build process. The script's name should be customBuildCallbacks.xml and it should be located at the project’s root:

de.michel-kraemer.myplugin/
  |- bin
  |- META-INF
  |- src
  |- build.properties
  |- customBuildCallbacks.xml

The same directory contains the plugin's build.properties file. (Please do not mix this one up with the more general build.properties file used for headless builds!) The following lines have to be added to this file to hook the new Ant script into the build process:

customBuildCallbacks=customBuildCallbacks.xml
customBuildCallbacks.inheritall=true

The second line enables access to global properties like ${build.result.folder}.

The Ant script can contain targets with predefined names. They will be called by PDE Build eventually during the build process. You can copy a template, that already contains all predefined targets (but empty) from the following directory: ${eclipse.home}/plugins/org.eclipse.pde.build_*/templates/plugins/

The post.compile.@dot target will be called when the source files are about to be compiled, so that's the best point to execute the Scala compiler manually. Therefore, some additional Ant tasks have to be defined. These can be found in the Scala Library which is delivered with the Scala IDE. You can automatically search for the required jar files using the following snippet:

<!-- find eclipse.home -->
<pathconvert property="eclipse.home">
    <path location="${eclipse.launcher}" />
    <mapper>
        <!-- map "${eclipse.home}/eclipse.exe" to "${eclipse.home}" -->
        <globmapper from="*/eclipse.exe" to="*" handledirsep="true" />
    </mapper>
</pathconvert>
 
<!-- find scala bundle -->
<pathconvert property="scala_bundle">
    <path>
        <fileset dir="${eclipse.home}/plugins">
            <include name="scala.library_*" />
        </fileset>
    </path>
</pathconvert>
 
<!-- find scala tools -->
<pathconvert property="scala_tools_jar">
    <path>
        <fileset dir="${eclipse.home}/plugins">
            <include name="scala.tools.nsc_*" />
        </fileset>
    </path>
</pathconvert>

This places the path to the Scala OSGi bundle in the variable ${scala_bundle}. The variable ${scala_tools_jar} will point to the jar file that contains the Scala Tools and, thus, the Ant tasks.

The library bundle has to be extracted first, so the actual Scala library scala-library.jar can be used in the classpath during the build. Therefore, you can use the temporary build directory ${build.result.folder}:

<unjar dest="${build.result.folder}/scala-library" src="${scala_bundle}" />
<property name="scala_library_jar"
    location="${build.result.folder}/scala-library/lib/scala-library.jar" />

After this, the Ant targets can be defined:

<!-- define scalac task -->
<taskdef resource="scala/tools/ant/antlib.xml">
    <classpath>
        <pathelement location="${scala_tools_jar}" />
        <pathelement location="${scala_library_jar}" />
    </classpath>
</taskdef>

Before the source files can be compiled, a valid classpath has to be created. It consists of the classpath defined by PDE Build @dot.classpath and the Scala library:

<pathconvert property="my.classpath">
    <restrict>
        <path>
            <path refid="@dot.classpath" />
            <pathelement location="${scala_library_jar}" />
        </path>
        <!-- remove libraries from classpath that don't exist (optional) -->
        <rsel:exists />
    </restrict>
</pathconvert>

Removing non-existing libraries using <rsel:exists /> is optional. To make it work, you have to add the following namespace to the Ant script's root node:

<project name="Build specific targets and properties"
    xmlns:rsel="antlib:org.apache.tools.ant.types.resources.selectors">
    ...

Finally, the source code can be compiled:

<!-- compile scala source files -->
<mkdir dir="${target.folder}" />
<scalac srcdir="${source.folder1}"
    destdir="${target.folder}"
    classpath="${my.classpath}">
    <include name="**/*.scala" />
</scalac>

For production use, it is also a good idea to delete the source files from the target folder, so they are not deployed together with the binaries:

<!-- delete scala source files in output folder -->
<delete>
    <fileset dir="${target.folder}" includes="**/*.scala" />
</delete>

Conclusion

The method presented here uses the possibility to hook custom Ant targets into the PDE build process. The scripts try to find the Scala library and tools delivered with the IDE. If you don't like that, you may also copy the files scala-library.jar and scala-compiler.jar (contains the Ant task) from the Scala distribution into some directory of your project and change the classpath respectively.

The complete source of the generic customBuildCallbacks.xml file can be downloaded with the following link:

customBuildCallbacks-old.xml (3.8 KiB)

This article relates to an old version of the Scala IDE for Eclipse. An updated Ant script can be found in a newer post!

This file can be copied without changes into every OSGi bundle that should be compiled with PDE and that contains Scala code. You only have to change the build.properties file as described above.

The PDF version can be downloaded from the following website:
http://www.codecommit.com/scala-style-guide.pdf

An online version is available under the following address:
http://davetron5000.github.com/scala-style/

The style guide is well structured. Each definition also contains the reason why you should format your code this way. Counter-examples show how unreadable code becomes, if you don't.

Since I'm a proponent of structured coding I can recommend this document to everyone working with Scala. If you're new to the language, you should read the style guide very soon to avoid getting into "bad habits". By following the rules from the style guide you can increase readability of your Scala program which also improves extensibility and maintainability.

The free theme phpbb-iphone-style especially developed for the iPhone adds a lot of comfort. It removes unnecessary display elements, increases font size and improves controlling the forum with the touch screen.

The theme can be installed very easily. You simply have to download the project's source code and copy it onto your web server under the styles directory. If you like, you can also follow the instructions found in the read-me file, so the forum automatically uses the new theme if someone visits the site with a mobile device. Finally, you may disable the theme in the administration panel, so it cannot be activated in normal browsers.

I installed phpbb-iphone-style into the Spamihilator support forum. Here are some screenshots:

During the next few days I will add some enhancements, so the theme will fit the overall style of the Spamihilator website.