Already on the first hacking day we had a result, a Couchbase backend for the deegree blob storage. This means that you can now store your rich features (e.g. from INSPIRE) in Couchbase and serve them up as a WFS with deegree. In case you wonder what rich features are, it's the new, shiny and more descriptive name for complex features.

In the following days I worked together with Oliver Tonnhofer on a CouchDB/GeoCouch backend for Imposm. You are now able to store your OpenStreetMap data in CouchDB and make queries on it through GeoCouch. I've created a small demo that displays the some data import from Andorra with directly with MapQuery, without the need of any additional server/service. The CouchDB backend should be easily adaptable to Couchbase, if you want to help, let me know.

I've then spent some time on the GeoNetwork project and helped translating the language file to German. I cleaned it up a bit and fixed major wrong translations. It's not perfect yet, as I've only spent little time on it, but at least it should be way more understandable (and sadly less funny) than the original version which was generated by Google Translate.

When it was time for presentations, I give a quick overview over the Couch* ecosystem. From CouchDB to GeoCouch, BigCouch, PouchDB, TouchDB (TouchDB-iOS, TouchDB-Android), Couchbase Syncpoint and Couchbase. You can find the slides as PDF here.

On the last day I've spent my time on polishing GeoCouch a bit and getting it ready for the Couchhack in Vienna. I've backported all changes from Couchbase to the CouchDB 1.2.x branch and also ported the geometry search into an experimental branch. You can now search your GeoCouch with any Geometry GEOS supports.

The event was fun as always and I also get to know some new people (hello B3Partners guys). Thanks Jeroen from GeoCat for organizing it, and thanks all other hackers that made it such a awesome event. Hope to see you all next year!

The future of Apache CouchDB

The reactions were quite different. People who are not deeply involved with the CouchDB community think that this means the end of Apache CouchDB. My reaction was positive, I tweeted:

“It’s good to see the Damien is so open to [the] world”

The reason was, that for me it was pretty clear that it would happen, and I was just happy that Damien officially made the cut.

The reactions from CouchDB community members where pretty much what Till Klampäckel describes in his blog post. You could see it comming after Couchbase announced that they are not the CouchDB company and that their product won’t be Apache CouchDB compatible.

I agree with Till here, the way Damien wrote his blog post, isn’t the best imaginable. For outsiders, it really seems to be the end of Apache CouchDB, but it is not. For me it just shows, why foundations like the Apache Foundation are such a great idea. Even if the original creator leaves the project, it still lives on.

Apache CouchDB has a lot of contributers and the mailing lists and IRC channel is busy as always. That CouchDB has a future is also shown by the blog post from Cloudant. They will keep supporting Apache CouchDB.

The future of GeoCouch

After this quick recap what happened so far, it’s time to talk about the future of GeoCouch. As you may know, I work for Couchbase on the integration of spatial functionality into their product.

Currently the overlap between Apache CouchDB and the version Couchbase uses internally is still quite huge, but it will diverge more and more in the future. Thus it will get harder and harder to maintain a single version that supports Apache CouchDB and Couchbase.

The good news is, that GeoCouch is pretty much a data structure only. It's an R-tree that stores JSON documents. This can easily be used by CouchDB and Couchbase. Perhaps small wrappers will be needed, but those should be minimal.

The easiest way to understand how the future looks like is in a small illustration:

GeoCouch's core is the R-tree, it's the same code for CouchDB and Couchbase. On top of it there will be code that is specific to either CouchDB or Couchbase.

This means that the majority of the devlopment I do for Couchbase will also improve the GeoCouch you can use for CouchDB.

Conclusion

The future of all three, Apache CouchDB, Couchbase and GeoCouch looks bright.

My talk: The State of GeoCouch

I'm really happy how my talk went, I really enjoyed it. The were lots of people (although there was a talk from Frank Warmerdam at the same time) asking interesting questions at the end.

The talk is not only about GeoCouch but also gives you an overview of some of the features it leverages from Apache CouchDB. In the end you should have an overview why you might want to use GeoCouch for your next project.

You can get the slides right here.

• The slides (download optimized) as PDF (licensed under CC-BY-3.0-de).
• The slides with comments as HTML (licensed under CC-BY-3.0-de).
• The slides with audio. It’s the recording of the actual talk at the conference.

Other talks

I was happy to see that there was another talk about GeoCouch. Other talks I really enjoyed were:

• CartoSet, a new FOSS to create highly visual geo portals: So many true things were said, e.g. get the data from the customer first, before you start building anything.
• Introduction to OpenSource WebMapping: Beyond Google Maps: A very nice introduction off the full stack for web mapping. From the database, over the server to the client. It's for people that have no clue that they could have maps on their own infrastructure.
• Comparing GeoExt, MapQuery and Legato from a technical and collaborative point of view: A good comparison between GeoExt, MapQuery and Legato. I especially liked the “Bodo”-test.
• Beyond vectors: Adapting remote sensing research for environmental monitoring with open source hardware and software: Citizen mapping the BP oil spill with balloons and kites: Kite and balloon mapping is just awesome. It's kind of the OpenStreetMap for raster data.
• GeoGlobalDomination: The Musical: One of the highlights was the GeoGlobalDomination musical. Grab the video while it's hot.

And of course there were also great talks from in the plenary sessions from Paul Ramsey about Why do you do that? An exploration of open source business models and Schuyler Erle's so funny lightning talk about Pivoting to Monetize Mobile Hyperlocal Social Gamification by Going Viral

Code Sprint

At the code sprint I was working on MapQuery together with Steven Ottens and Justin Penka. Steven was working on TMS support, Justin on a 6 minutes tutorial and I on making manual adding of features possible.

The OpenLayers developers did the migration from Subversion to Git for their development. OpenLayers is now available on Github.

And luckily there was a fire alarm in between to take a group photograph.

Future of the FOSS4G

I really hope there won't be a yearly FOSS4G conference for the whole of the US. There should be regional events, as I think one big one would draw the attention away from the international conference. Why should you fly to Beijing for the FOSS4G 2012 if you can meet the majority of the developers in the US as well?

Final words

The FOSS4G was great. It was organized well and people were always out in the evenings. The only minor nitpick is that many people working remote had the city of their company in the name badge and not the one they live in. It seems that the original for you had to fill was confusing. So for next year it should perhaps say “Location where you live”. Hence I still don't believe that there were more Dutch than German people at the conference (Tik hem aan, ouwe! ;)

GeoCouch

GeoCouch, a spatial index for CouchDB gains, more and more attention. One of the reason is that the installation recently got way easier for developers as well as for normal users. You now can install GeoCouch as an extension right next to your already existing CouchDB instance. You may also download a binary of Couchbase-Server, which already includes GeoCouch. And finally there's the brand new Iris Couch hosting as well (previously known as the CouchOne hosting). So getting started with GeoCouch is easier than ever before.

Some people might have wondered about the state/future of GeoCouch, especially after the merger of CouchOne with Membase to Couchbase. I will keep on developing GeoCouch at Couchbase and it is (as it always was) fully open source licensed under the Apache 2.0 License.

The new home for the latest source is the Couchbase Github repository.

OpenStreetMap

The FOSSGIS was also about OpenStreetMap. The idea to put OpenStreetMap data into GeoCouch is very sensible, but wasn't really done (AFAIK) in a big fashion. Luckily Jochen Topf from Geofabrik told me about his Projekt Osmium, which makes it possible to process OSM data with JavaScript. There is already a script to output a Shapefile, so it should be really easy to output GeoJSON, which could be consumed by GeoCouch. So if you (who are currently reading this) have some spare time, please give it a go :)

MapQuery

MapQuery is a web mapping framework that builds on OpenLayers and jQuery. The goal is a framework that is just as easy to use as jQuery combined with the power of OpenLayers. It's meant for people that just want to get started with web mapping, but also for those who have already knowledge about OpenLayers and want to have easy integration into their jQuery application.

I was able to show a quick demo of the MapQuery API at the FOSSGIS. I won't publish it here, as things are about to move fast. After over one year of discussions about MapQuery and only little code contributions, it seems that we are finally getting somewhere. That feels so good :)

The wonderful EduGIS is build on an early version of MapQuery (source code), but will be merged with the most recent version of my fork.

Other big news is that the WhereGroup hired Christian Wygoda, who is a committer of the MapQuery project. This also means that Mapbender 3 will use MapQuery.

And finally I've also met a developer of a another company that was building a big mapping application based on OpenLayers and jQuery. I don't want disclose it here, as the code isn't open source yet, but the developer told me that it should be easily possible. I will keep in touch with them and hope they will contribute their code to MapQuery.

To get to a conclusion about MapQuery. If you want to stay in touch with the project, please subscribe to the official mailing list, this is where things are happening (there's also the little attended IRC channel #mapquery on freenode). If you want to be a user of MapQuery, you should be patient and wait a bit. If you plan to contribute, you can start now. The currently biggest item is moving the EduGIS MapQuery code base over to the MapQuery version of my fork. The "documententation" are the demos.

FOSSGIS

As people started to asked about the slides from my presentaion at FOSSGIS, here they are.

FOSSGIS was a really awesome event, where I met a lot of new people, but also a lot of friends I haven't seen in a while. I'm really looking forward to next year's conference, but also hope that I might see many of the people at this year's FOSS4g in Denver.

After several long discussions with Jan I finally wrapped my head around the document oriented concept. I was blown away, it was exactly what I would have liked to use on so many occasions at my one year internship at a geospatial company. Though CouchDB wasn't ready, I needed spatial indexing. One week later I had a first idea of how such an extension might look like.

And only 2 years later I'm really involved in CouchDB and people actually start using GeoCouch :) I'd like to use this blog post to thank the developers and the whole community, it's been a great time and the IRC channel just kicks ass. You all helped to make CouchDB 1.0 possible!

GeoCouch: Eine Erweiterung für CouchDB zur Abfrage räumlicher Daten

Er richtet sich an Geographen, wird also nicht zu sehr ins Detail der Implementierung gehen. Es sind auch keine Vorkenntnisse zum Thema CouchDB nötig. Wer also mehr über CouchDB und GeoCouch wissen will, ist herzlich dazu eingeladen. Danach stehe ich natürlich zu Fragen zur Verfügung.

Ich habe keine Ahnung wie groß die CouchDB Community im Raum Augsburg ist, aber sollte jemand dieser Einladung folgen, spricht auch nichts gegen ein anschließendes kleines CouchDB/GeoCouch/NoSQL Meetup. Am besten meldet ihr euch bei mir per Mail, denn wenn ein paar Leute sicher kommen, werden es sich andere bestimmt auch überlegen.

Sorry Planet CouchDB for writing in German, but this is about a talk in German.

I had the chance to tell people a bit more about CouchDB and Couchapps,

One project I haven't heard that much before of was Degree. They build the whole stack of OGC services you could imagine. For me it was of interest that they have a blob storage in their upcoming 3.0 release. The data isn't flattened into SQL tables but stored as blobs. This sounds like good use for a CouchDB backend in the future.

I was working with Simon Pigot on a GeoNetwork re-implementation based on CouchDB using Couchapp. We got the basic stuff like putting an XML document into the database, editing it and returning the new document, as well as fulltext indexing with couchdb-lucene work. Next steps are improving the JSON to XML mapping and integrating spatial search based on GeoCouch.

The event was really enjoyable, thanks Couchio for sponsoring the trip, thanks Jeroen for organizing it, and thanks all other hackers that made it such a awesome event. Hope to see you next year!

I did it! I'll speak at the FOSS4G Conference 2010 (Free and Open Source Software for Geospatial Conference), 6th–9th September in Barcelona about “GeoCouch: A spatial index for CouchDB”. As soon as the abstract is available online I'll link to it. Hope to see you there!

An idea has become reality. Exactly two years after the blog post with the initial vision, a new version of GeoCouch is finished. It's a huge step forward. The first time the dependencies were narrowed down to CouchDB itself. No Python, no SpatiaLite any longer, it's pure Erlang. GeoCouch is tightly integrated with CouchDB, so you'll get all the nice features you love about CouchDB.

Current implementation

Thanks to the feedback after the FOSS4G 2009 and "GeoCouch: The future" blog entry" it was clear that people prefer a simple, yet powerful and tightly integrated approach, rather than having to many external dependencies (which was a showstopper for quite a few people).

I implemented an R-tree (I call it vtree as the implementation is subject to change a lot) from scratch. The reason why I haven't used the already existing R-Tree implementation available at Github is that I needed something to learn Erlang, it doesn't contain test or examples and that it is always a good idea to implement a data structure yourself to understand the details/problems. My implementation is far from being perfect but works good enough for now. The vtree is implemented as an append-only data structure just as CouchDB's B-trees are. Currently it doesn't support bulk insertion.

If you want to know details on how to create your own indexer, have a look at my Indexer tutorial.

Feature set

Following the "Release early, release often" philosophy currently only points can be inserted, the only supported query is a bounding box search. Though other geometries should follow soon.

Using GeoCouch

GeoCouch is now hosted at Github. Giving GeoCouch a go is easy:

git clone http://github.com/vmx/couchdb.git
cd couchdb
./bootstrap
./configure
make dev
./utils/run

To try the spatial features when it's up and running is easy as well. Just add a spatial property and a named function to your Design Document as you would to for show or list functions:

function(doc) {
    if (doc.loc) {
        emit(doc._id, {
            type: "Point",
            coordinates: [doc.loc[0], doc.loc[1]]
        });
    }
};

All you need to do is emitting GeoJSON as the value (Remember that point is the only supported geometry at the moment), the key is currently ignored.

curl -X PUT http://127.0.0.1:5984/places
curl -X PUT -d '{"spatial":{"points":"function(doc) {\n    if (doc.loc) {\n        emit(doc._id, {\n            type: \"Point\",\n            coordinates: [doc.loc[0], doc.loc[1]]\n        });\n    }};"}}' http://127.0.0.1:5984/places/_design/main

Before a bounding box query can return anything, you need to insert Documents that contain a location.

curl -X PUT -d '{"loc": [-122.270833, 37.804444]}' http://127.0.0.1:5984/places/oakland
curl -X PUT -d '{"loc": [10.898333, 48.371667]}' http://127.0.0.1:5984/places/augsburg

And finally you can make a bounding box request:

curl -X GET 'http://localhost:5984/places/_design/main/_spatial/points/%5B0,0,180,90%5D'

This one should return only augsburg:

{"query1":[{"id":"augsburg","loc":[10.898333,48.371667]}]}

Next steps

The development of GeoCouch was quite slow in the past, but it gets up to speed as my diploma thesis (comparable to a master's thesis) will be about GeoCouch. Additionally Couchio kindly supports the development.

The next steps are (in no particular order):

• Better R-tree (better splitting algorithm, bulk operations)
• Supporting more geometries
• Polygon search
• Improving CouchDB's plugin capabilities

Thanks

I'd like to thank all the people that kept me motivated over the past two years with their tremendous feedback. Special thanks go to Jan Lehnardt for getting me onto the Couch, Cameron Shorter for introducing me into the geospatial open source business and all people from Couchio for the great two weeks in Oakland.

Stripping it down

GeoCouch needs an external spatial index, at the moment I use SpatiaLite for it, but a PostGIS backend would be easily possible. My inital idea was that it is better to use the existing power of spatial databases, rather than reinventing the wheel. I though I could use all the power they have, that I can even use them for complex analytics, but I can't. As I only store the geometries, I need to “ask” CouchDB for the attributes (no, I don't want to store attributes in my spatial index).

If I don't use the full power of the spatial databases, but only a small fraction, there might be better solution. Therefore I propose that GeoCouch will use a simple spatial index for storing the geometries, not a full blown spatial database. I haven't decided yet which one it'll be, but I really think about moving this part to Erlang (I know that quite a few people would love that move).

You will loose functionality like reprojection. The spatial index won't know anything about projections. So GeoCouch won't be projection aware anymore, but you application still can be. For example if you want to return your data in a different projection than it was stored, you do the transformation after you've queried GeoCouch.

You would also loose fancy things for geometries, like boolean operations on them. But this is something I'd call complex analytics, and not simple querying.

GeoCouch would only support three simple queries: bounding search, polygon search and radius/distance search. If the search would be within a union of polygons, let's say all countries of the European Union, you would simply make the union operation before you query GeoCouch.

Complex analytics

What I call “complex analytics” is things like: “return all apple trees that are located with a 10km range around buildings that have are over 100m high, but only in countries with a population over 50 million people” is not possible with GeoCouch as you would need the attribute values as well. Those are stored in CouchDB, so you would need to request them. What GeoCouch only supports is a simple: give me all IDs within a bounding box/polygon/radius.

Conclusion

Simple requests are needed for everyday use, thus they should be incredibly fast. Complex analytics don't necessarily need to handle thousands of requests per second, in most cases they don't even need to be processed in real-time. I'd like to see some layer build above GeoCouch, so CouchDB can even be used for analytics (which is a thing I wanted to have right from the start).

This means that GeoCouch will be mainly for high performance and massive sized projects that need some simple spatial bits, what I think the majority of users need.

If you either think you really need only those simple queries, but you want them to be fast, or you think this is wrong, that you need dynamic reprojection I can only invite you to leave a comment below or drop a mail to volker.mische@gmail.com. Thanks.