Simulating the High Replication Datastore Locally

Recently I was trying to add transactional support to certain batch processes in a Google App Engine app and it was coming up with strange errors. I was using Objectify. In particular it would tell me that it

can't operate on multiple entity groups in a single transaction

To my knowledge, I wasn't trying to operate on multiple entity groups. The problem turned out to be that the local dev datastore doesn't simulate the eventual consistency of the live datastore properly.

However, it's possible to turn on a simulation of this behaviour in your local app by following these instructions, ie by passing this command option:

-Ddatastore.default_high_rep_job_policy_unapplied_job_pct=1

The 1 on the end is the percentage of eventual consistency you want to see in your datastore. In practice any number bigger than zero is enough to get Objectify transactions working properly locally in these situations.

If you are using Maven to run your apps using the maven-gae-plugin, then you can configure this option in your pom.xml as follows:

        <plugin>
            <groupId>net.kindleit</groupId>
            <artifactId>maven-gae-plugin</artifactId>
            <version>0.9.6</version>
            <configuration>
                <jvmFlags>
                    <jvmFlag>-Ddatastore.default_high_rep_job_policy_unapplied_job_pct=1</jvmFlag>
                </jvmFlags>
            </configuration>
            <dependencies>
                <dependency>
                    <groupId>net.kindleit</groupId>
                    <artifactId>gae-runtime</artifactId>
                    <version>${gae-runtime.version}</version>
                    <type>pom</type>
                </dependency>
            </dependencies>
        </plugin>
Posted on May 8, 2013 and filed under dev.