Archivo para 28 mayo 2009

GSoC Weekly report 3-4: The Akonadi-Nepomuk connection

First of all I apologize because last week I was busy doing school stuff and I didn’t get to do much GSoC related stuff. I’m glad to be able to say that today I will assist to my last university class in this academic year and that my holidays (of code :P) period has thus virtual started, because I have also already done my last exam last week, yay.

BTW,  I am going to Akademy too! I already can’t wait to get there 😀

I’ve been investigating how well can I make Akonadi and Nepomuk play together. It seems that my decision of only using Nepomuk through Akonadi is possible. I will thus have an usual Akonadi Resource which provides a set of collections with their items (bookmarks) and subcollections (bookmark folders), and then also I will have some virtual collections.

A virtual collection is a new concept in Akonadi, which still is not enterely developed and Konqueror will probably the second (after Akonadiconsole) application taking advantage of it. A virtual collection has all the properties of a collection, but the items it contains are peculiar. They are not items as in a normal collection, but virtual items, which link to real items inside real collections.

Using virtual collections, besides having your usual bookmarks menu, you will also get in konqueror some virtual collections like: Recently added bookmarks, Most visited bookmarks, Recent tags, or <please suggest your virtual collection in a comment please!>.

You might then wonder… okey so Recently Added Bookmarks would be a virtual collection which contains urls which have been bookmarked recently. But how can an Akonadi virtual collection know which items comply with that? Only Nepomuk knows the relevant information. Here comes the Akonadi-Nepomuk connection:  you can create an Akonadi Virtual Collection with SearchCreatJob class, which takes two arguments: the title of the collection (Unclassified Bookmarks for example) and the SPARQL query string for Nepomuk.

Using SearchCreateJob, I just need to write a SPARQL query that returns only the recently added bookmarks, which is quite trivial. Then Akonadi will handle that query to a Nepomuk persistant query, and it will tell akonadi whenever resources are added or removed to that query. In turn, Akonadi will get the Uri of those added/removed resources, and use that as the remoteId of the Items in the virtual collection, and add/remove those items accordingly. Autotically if there the item with that very same remoteId in a real collection, Akonadi will know, and the item in the virtual collection will have the same properties: same payload, same name, etc.

Usecase to understand and recapitulate:

  1. You click on the star appearing in the location box to bookmark current url.
  2. The url gets added to some Konqueror bookmarks collection (if it’s an unclassified bookmark, it will added to a special-purpose Akonadi bookmark collection called “Unclassified bookmarks”).So now your bookmark has its corresponding real Akonadi item.
  3. Akonadi bookmarks use Nepomuk as storage, so the bookmark is also added in Nepomuk database. In fact it’s only stored persistently there, Akonadi only caches it.
  4. When the new bookmark is added to nepomuk, automatically nepomuk detects that the bookmark enters in the persistant query for “Recently added bookmarks” and adds the new bookmark resource to it, notifying to Akonadi server which was looking for changes in that persistant query.
  5. Akonadi server gets the notification of a new resource added to the “Recently added bookmarks” Nepomuk query,  and adds a new item to its “Recently Added Bookmarks” virtual collection. That new item will actually refer to the item added in step 1, with the same name, payload (a KonqBookmark instance), etc.
  6. Konqueror automagically gets the new bookmark in “Recently added bookmarks” subcollection!

To some people all this might seems like a lot of steps, but to me it sounds like angels. Because all this will make konqueror feel faster and it will make its bookmarking system responsive (non-blocking!) and improve it’s architecture. I am already thinking about how easy will it be to create kross plugins for bookmarks, to be able to support natively new types of bookmarks, to be able to sync with del.ici.us-style services, and even to easily make all this a bit more flexible and let other apps (Konsole, Kate, etc) use this new Bookmarks System. But, one step at a time :D.

Anuncios

GSoC weekly report: week 2

This week I’ll present you part of the technologies used in the brand new Konqueror bookmarks system. I’ll explaining how they work and how I’m going to use them, thereby showing part of the design of the new bookmarks model at the same time. Finally, I’ll also show you what’s the current development state and what I plan to do for next week.

Nepomuk

Nepomuk is where I will store Konqueror bookmarks. It’s very easy to do so. In my code, I only need the following code to create a bookmark in Nepomuk and put it in a bookmark folder:


Nepomuk::Bookmark kdeBookmark("http://www.kde.org");
kdeBookmark.addTitle("KDE main site");
Nepomuk::BookmarkFolder rootFolder("konqbookmarks:/");
rootFolder.addContainsBookmark(kdeBookmark);

That’s it! This line of code will create the bookmark in the Nepomuk database if it wasn’t already there, set the title of the bookmark, and it will add the bookmark to the Konqueror’s root bookmark folder. After executing that code, if I can search in Krunner for kde.org, and the Nepomuk runner will find the bookmark. And any application who wishes to list the bookmarks in the root bookmark folder can do so with the following bit of code:


Nepomuk::BookmarkFolder rootFolder("konqbookmarks:/");
QList bookmarks = folder.containsBookmarks();

UHm..what if I wanted to remove a bookmark from Nepomuk? I can do so every easily too by just calling to bookmark.remove();. It also allows me to attach a lot more information related to a bookmark of course. All kinds of resources and properties can be attached to a Nepomuk bookmark: tag, creation time, whatever. All in all, Nepomuk stand for an easy to use semantic database.

Akonadi

Akonadi is a convenient way to access data stored elsewhere. It isn’t designed to store the data itself, because it’s a middle layer between the application and the data. It gives power to the develop by providing an abstraction layer in which all the data is represented by collections and items in a tree similar to the concept of dirs and files in a filesystem.

A Collection is made up of other subcollections and items. And of course, there is a root collection to rule them all, master of the puppets. To clasify the data, both collections and items have one or more mimetypes attached to them. For example, a collection has the mimetype Collection::mimetype(), but it can also have the mimetype “application/x-vnd.kde.konqbookmark” which is the one I’m using right now for Konqueror bookmarks.

Now a bit more talk about the items. You guessed it, the items can have some data attached to them, but how? Using payloads. A payload is a supported Akonadi class that stores some data. It could be an email, or in my case, a bookmark. I don’t use directly Nepomuk::Bookmark (but I could if I wanted), but a wrapper class that I called KonqBookmark.

When one application request the KonqBookmark payload of a bookmark item, that data needs to go from the akonadi server process to the application process. This is done serializing the data in the server and deserializing it in the other side. As KonqBookmark is not a natively supported class by akonadi, I have to give Akonadi a class that serializes it, and then it works.

An application connected to Akonadi can add new items to a Collection, modify them, delete them. Same with collections: they can be addd/modified/removed. Imagine I’ve got two instances running of my bookmarks editor. If I remove in instance 1 a boomark item via Akonadi, Akonadi itself updates instance 2 of the bookmark editor and shows there that item is gone. Without me needing to do anything! No need to sync via d-bus, etc. It’s like magic, but one that really works.

So far, we’ve got Collections and items in the Akonadi server.. but how? where do they come from? They come from Akonadi agents, concretely from an Akonadi resource. An Akonadi resource provides a set of collections and their items. I’ve created the Konqueror bookmarks resource that adds some collections of bookmarks to Akonadi. But you could add for example a new bookmarks resource different from mine using the same mimetype as me, and provide access to a del.ici.us collections. You just need to code for the resource the part in which you access to the del.ici.us bookmarks and create the correspondent collections and items in Akonadi.

Akonadi is a middle layer, it doesn’t really stores the data. Thus, the bookmarks resource I’m developing stores the data in Nepomuk, as I explained earlier. If you create a del.ici.us resource as I said in the previous paragraph, you could access your del.ici.us bookmarks but they wouldn’t be the same as those stored in nepomuk. A new kind of akonadi agent comes to the rescue: Akonadi feeders. You could create a del.ici.us feeder instead for konqueror bookmarks, and then when a new Konqueror bookmark is added, it would also be feeded to del.ici.us, etc.

And what if I’ve added a bookmark when I was offline? what happens there? Akonadi then caches the bookmark, and it would be added to del.ici.us when you connect to Internet. This is transparent to the del.ici.us feeder: it only needs to have a function to add bookmarks to del.ici.us, and Akonadi will simply call it when it’s appropiate. Again, it’s like magic.

Current state

So far, I’ve got an Akonadi resource which can list bookmarks in the Nepomuk database, add new bookmarks, modify them, remove them. The bookmark items use a KonqBookmark class payload, which is supported by Akonadi as payload because I’ve also implemented a KonqBookmark serializer/deserializer.

I’ve created a sample application using this resource, a bookmarks editor, which you can see in the screenshot below (I promised you would get one this time ;-). This bookmarks organizer shows you the bookmark folders and bookmarks as usual, using two views: a tree of bookmark folders and a listing of bookmarks.

First screenshot of the very simple Bookmarks Editor

The application itself is very simple, because it uses model-view & delegate classes for displaying and editing bookmarks and bookmark folders. Some of these classes are provided by Akonadi (the model for Bookmarks collection for example), and I developed the other classes, like the model and delegate for Konqueror bookmarks.

I’m developing all this in the most reusable way so that other web browser that wish to share with Konqueror the bookmarks system can do it very easily. I remind all those curious people that the code is available in github.

Next steps

Next week, I plan to improve both the editor and the resource. I also want to start playing with Akonadi virtual collections for doing searches in the bookmarks too, and maybe I start doing some tests with Kcompletion stuff for the new location bar. Have a nice day!

GSoC weekly report

I have decided to do the GSoC weekly report thing. Sometime after deciding that I noticed that I need to actually write and post in the blog the report because the report doesn’t write itself – go figure! Technology doesn’t seem to be so advanced to do it automatically just yet. So here I go.

In this project I will be reimplementing the Konqueror bookmarks system using Akonadi and Nepomuk. I already have KDE trunk compiled but I needed to be sure both Akonadi and Nepomuk are up and running correctly. Both worked fine in my Samsung laptop setup, but in my Netbook I needed to install libmysqlclient15-dev deb package, rebuild qt-copy with -plugin-sql-mysql config option, and of course rebuild Akonadi, because even if I had other DBs working, the only one currectly supported in Akonadi is Mysql.

Oh yeah you read it right, I do compile and develop in an Acer Aspire One netbook, with an Intel Atom and 1Gb of RAM, and it works just fine. It’s a bit slower when rebuilding a whole package, but other than that it’s ok. And when doing such a thing (rebuilding qt-copy for example) I can always use icecream and let the other machines at home handle the tough compiling stuff.

Alright, now I have a working environment. Next step? Get used to Akonadi and Nepomuk, learn how to use their APIs to make myself feel at home with them and thus be able to develop the concrete details about how to design the new Konqi bookmarks system. Thanks to the Akonadi and Nepomuk tutorials found in Techbase, the nice API documentation found in api.kde.org, the code itself of both libraries and the pieces of sofware which use those utilities (like krunner or dolphin), together with the invaluable help provided by friendly people in #akonadi and #nepomuk-kde channels, learning is a joy and it’s very easy to do so. That’s something to be proud of in the KDE community!

I’ve setup a public Github repository in which I will dump everything I’m doing related to my GSoC project. This way I will use it to synchronize easily my laptop and my netbook. That means this repo will really contain everything I’m working on. At the moments there’s nothing close to being useful for anyone else than me, it’s all tests and the tutorials of Techbase a bit modified for my needs. So children be aware this is not for public consumption, so you’ve been warned. On the other hand and once said that, it has the advantage that you can see what I’m doing right now if you are brave enough to do so.

This blog post is becoming larger than I pretend it to be, so I will stop here today. Next time I will talk about what’s Akonadi and what’s Nepomuk, how I’m going to design the architecture of the brand new bookmarks system and what is so cool about it. And I promise I will insert at least a picture in next gsoc weekly report even if that means choosing one randomly from 4chan.org/b or /dev/null (is there a difference anyway?). Have a nice day!