vmx

the blllog.

GeoCouch: The future is now

2010-05-03 22:27

Update: This blog entry is outdated and kepts for historical reasons. Please do always check for newer blog posts. The up to date information on how to install and use GeoCouch can be found in its README.

An idea has become reality. Exactly two years after the blog post with the initial vision, a new version of GeoCouch is finished. It's a huge step forward. The first time the dependencies were narrowed down to CouchDB itself. No Python, no SpatiaLite any longer, it's pure Erlang. GeoCouch is tightly integrated with CouchDB, so you'll get all the nice features you love about CouchDB.

Current implementation

Thanks to the feedback after the FOSS4G 2009 and "GeoCouch: The future" blog entry" it was clear that people prefer a simple, yet powerful and tightly integrated approach, rather than having to many external dependencies (which was a showstopper for quite a few people).

I implemented an R-tree (I call it vtree as the implementation is subject to change a lot) from scratch. The reason why I haven't used the already existing R-Tree implementation available at Github is that I needed something to learn Erlang, it doesn't contain test or examples and that it is always a good idea to implement a data structure yourself to understand the details/problems. My implementation is far from being perfect but works good enough for now. The vtree is implemented as an append-only data structure just as CouchDB's B-trees are. Currently it doesn't support bulk insertion.

If you want to know details on how to create your own indexer, have a look at my Indexer tutorial.

Feature set

Following the "Release early, release often" philosophy currently only points can be inserted, the only supported query is a bounding box search. Though other geometries should follow soon.

Using GeoCouch

GeoCouch is now hosted at Github. Giving GeoCouch a go is easy:

git clone http://github.com/vmx/couchdb.git
cd couchdb
./bootstrap
./configure
make dev
./utils/run

To try the spatial features when it's up and running is easy as well. Just add a spatial property and a named function to your Design Document as you would to for show or list functions:

function(doc) {
    if (doc.loc) {
        emit(doc._id, {
            type: "Point",
            coordinates: [doc.loc[0], doc.loc[1]]
        });
    }
};

All you need to do is emitting GeoJSON as the value (Remember that point is the only supported geometry at the moment), the key is currently ignored.

curl -X PUT http://127.0.0.1:5984/places
curl -X PUT -d '{"spatial":{"points":"function(doc) {\n    if (doc.loc) {\n        emit(doc._id, {\n            type: \"Point\",\n            coordinates: [doc.loc[0], doc.loc[1]]\n        });\n    }};"}}' http://127.0.0.1:5984/places/_design/main

Before a bounding box query can return anything, you need to insert Documents that contain a location.

curl -X PUT -d '{"loc": [-122.270833, 37.804444]}' http://127.0.0.1:5984/places/oakland
curl -X PUT -d '{"loc": [10.898333, 48.371667]}' http://127.0.0.1:5984/places/augsburg

And finally you can make a bounding box request:

curl -X GET 'http://localhost:5984/places/_design/main/_spatial/points/%5B0,0,180,90%5D'

This one should return only augsburg:

{"query1":[{"id":"augsburg","loc":[10.898333,48.371667]}]}

Next steps

The development of GeoCouch was quite slow in the past, but it gets up to speed as my diploma thesis (comparable to a master's thesis) will be about GeoCouch. Additionally Couchio kindly supports the development.

The next steps are (in no particular order):

  • Better R-tree (better splitting algorithm, bulk operations)
  • Supporting more geometries
  • Polygon search
  • Improving CouchDB's plugin capabilities

Thanks

I'd like to thank all the people that kept me motivated over the past two years with their tremendous feedback. Special thanks go to Jan Lehnardt for getting me onto the Couch, Cameron Shorter for introducing me into the geospatial open source business and all people from Couchio for the great two weeks in Oakland.

Categories: en, CouchDB, Python, Erlang, geo

Comments

  1. Jón Grétar:
    2010-05-03 20:47:01

    The git path you post is actually the private URI to the repo.

    The right command would be:
    git clone http://github.com/vmx/couchdb.git couchdb-geocouch

  2. Volker Mische:
    2010-05-03 21:27:53

    Fixed. Thanks Jón.

  3. David Sonnen:
    2010-05-03 21:35:38

    Nice work. It will be interesting to see how geospatial capabilities are adopted in the CouchDB community. You've made an important contribution.

  4. Sameer Babu K K:
    2010-05-04 03:33:30

    Do we need couchdb 0.11 to be installed before installing geocouch?

    I got the following errors

    $ git checkout -t origin/geocouch
    fatal: git checkout: branch geocouch already exists

    $ ./bootstrap
    ./bootstrap: 55: libtoolize: not found

  5. Cameron Shorter:
    2010-05-04 05:34:08

    Volker, great to see this project grow through to maturity.

    I see that you hope to present again at FOSS4G 2010.

    It would be great if GeoCouch were incorporated in OSGeo marketing material.

    In particular, I'd love to see GeoCouch bundled on the OSGeo Live DVD. Also, I'm hoping one of the database comparison presentations will be selected at FOSS4G, which will include "Why you would, and why you would not select GeoCouch".

  6. Jón Grétar:
    2010-05-04 10:01:08

    @Sameer: CouchDB does not need to be installed as this is a modified version of the CouchDB source code. However you do need the dependencies required to build CouchDB. Namely icu4c and Erlang. Just check the CouchDB instructions on how to achieve these.

    Ignore the first error as it only means that this branch is already checked out.

    The second error however points to you not having the basic build tools installed on your computer. In MacOS it means you need to install Xcode. In Ubuntu/Debian you need to install the 'build-essential' meta package.

  7. Volker Mische:
    2010-05-04 10:40:07

    @Jón: Thanks for helping out.

    @Cameron: Indeed I hope to present at FOSS4G 2010. Though I think GeoCouch is not ready (yet) for OSGeo marketing purpose (or the DVD). It's still a long (though shorter) way to go :)

  8. Eike Send:
    2010-05-05 00:29:27

    Super awesome! Can't wait to try it out.

  9. Eike:
    2010-05-07 00:30:25

    Hi!

    I was able to compile geocouch, also to start the server and add the 2 documents and the view, but when I query with the bounding box as you have written, the request simply does not return.

    Where can I look for errors? The couch identifies itself as 0.12.0ade717b5-git

  10. Eike:
    2010-05-07 00:33:46

    I might add:
    - compiled on ubuntu lucid lynx
    - found the log file which is filled with thounds of these:

    [Thu, 06 May 2010 22:31:14 GMT] [error] [[0.389.0]] OS Process Error [0.4982.4] :: {os_process_error,{exit_status,127}}

    :(

  11. Volker Mische:
    2010-05-07 10:45:10

    @Eike: GeoCouch is moving fast atm, so please refer to the bundled README.geo. But that's not the problem. For Ubuntu issues please have a look at http://wiki.apache.org/couchdb/Installing_on_Ubuntu

  12. James Marca:
    2010-05-15 01:23:13

    Excellent work. I remember your initial postings on the subject, and it is awesome that you stuck with it and pushed out this milestone. I look forward to giving it a spin soon.

Comments are closed after 14 days.

By Volker Mische

Powered by Kukkaisvoima version 7