TimeVault has been busy taking snapshots of my data. But what good is that when I have no way of restoring files and folders from those backups? With that in mind I hacked together a quick way of restoring specific files and folders. Now provided with TimeVault are dolphin/konqueror service menus that allow for the following:

Remember that service menus are very simple. At the moment this means that you can attempt to restore files that weren’t included in the backup. This will fail (obviously) but it will tell you that it failed. You can also attempt to restore files at times older than the earliest backup. While this won’t fail, it’ll give you the earliest backup rather than what you might want. But it’s a start. And there are nice notifications when the restoration fails or is going to take a long time. The file/folder gets restored to filename.restored so you don’t have to worry about overwriting your current version.

At some time it’s going to be the time to start testing TimeVault on more systems. Backups are not things that should have uncertainty regarding their reliability so I will spend as much time testing as I feel is necessary. But for this to happen on a wider scale packages for common distros would be really useful. If anyone has the know how to make packages for kubuntu/openSUSE/Arch Linux/others and is interested please contact me. I would love the help. TimeVault itself shouldn’t be hard to package, getting the dependencies right might be a bit of a challenge, but that’s a one time deal that will certainly get easier as time goes on.

I’ve updated the README file which should give a much better idea of how to get things running. If you’re curious or want to experiment you can find TimeVault at http://gitorious.org/timevault

Lastly, I’ve taken to hanging out in #timevault on irc. Feel free to drop by and say hi. I’d love to answer any questions you have.

Edit: looks like krytzz has added timevault to the gentoo kde overlay.

The disk attached to the virtual server that powers this blog was becoming full so I spent a few minutes resizing it and allocating more space. Everything should be up and running again, please leave a comment if you find something isn’t working.

Edit: After speaking with Nuno and Mofux I’ve made some modifications to the System Settings page. This is how it looks now:

TimeVault is alive! Ok, so maybe it was never really dead. But I’ve found some time to work it and after reworking most of the internals I’ve finally gotten it to back up again.

For those who don’t know, TimeVault was originally a gnome backup system that I used way back when I was on Ubuntu. I liked the concept but disliked the implementation and so sometime after moving to KDE I started working on my own TimeVault. The goal is to have a user friendly, dead simple backup system that runs in the background, knows about (and exclusively uses) external media and isn’t afraid to integrate into KDE frameworks wherever it can.

Now the project has deviated quite a bit from the Gnome one with the same name. With that in mind, I’m contemplating a possible name change to reduce confusion. If people have suggestions feel free to leave a comment or send me an email.

The following is an image of the KDE System Settings page for TimeVault. It shows the current state and allows the user to select the backup drive and the folders that should be backed up.

Some new things of interest are:

• TimeVault no longer relies on a daemon running as root
• New KAuth integration (a part of KDE SC 4.4) which allows the system settings module to have it’s authorization bit
• Allows the user to select which paths are to be backed up
• Provides the user with notifications regarding the backup status (KNotification)
• Runs the backup with a KAuth helper running as root, much more secure than before

Some things that I still need to work on:

• The KIO slave has been neglected and no longer works. This is high on my todo list
• A dedicated restore framework is not yet in place. If the KIO slave isn’t good enough this will be needed
• The plasmoid also was neglected. I would like for it to contain backup statistics as well as information on the currently running file. Possibly also adding files/folders to the backup by dragging and dropping them on the plasmoid.

If you have ideas for me feel free to post them in the comments. I know I can’t think of everything that’s useful to a wide swath of people.

Update: Whoops, the wordpress cache was disabled. Not such a good idea as I’ve now found out all should be working again.

Nepomuk is a name that’s thrown around a lot. It’s claimed to be the answer to just about anything and at the same time is bashed for ‘not having a real effect’ on anything. With all of this buzz (for lack of a better term) surrounding the framework, I felt it might be time for me to figure out what’s going on myself. The following is a recollection of my learning process. It might not all work for you, heck it might not even be the correct way to do things. But it’s how I pieced the documentation together to comprehend what’s happening behind the scenes and why it matters.

First off, some explanations. From what I’ve gathered, Nepomuk is a database of relations and a framework for accessing that database. On a conceptual level, the triples that are contained within the database are simple, they’re “something — verb — something else” for example “My Task — is due — tomorrow”. If I understand it correctly, these relations (the verb parts) create a graph of all of the things contained in the database because they link all of the subjects together. Nepomuk/Soprano then allows you to search that graph and glean useful information from it.

Ok, so what does that mean to the application developer? To the end user? Let me try and explain with an example. One thing that I’ve always disliked about tags is their simplicity. A tag is a piece of text, no more. The only thing a computer knows about it is that it is in this huge general concept of ‘tag’ and that it contains some characters that may or not mean something to humans. So lets assume for example that I have a task (todo) to go and finish my engineering lab report with my lab group comprised of Bob and Sydney and myself. If I only have tags at my disposal, the best I could do would be to create a task with a subject such as “Finish Engin Lab” and then tag it with “Engineering”, “Lab”, “Sydney”, “Bob”, “Room 312″. If I’m looking at the task it’s reasonably clear what I meant, I need to do it with those two people and it’s located in room 312. However, what does the computer know about the lab? Almost nothing. I might have other labs that I’ve tagged with Sydney and Bob (seeing as they are my lab partners for the entire semester) but I might also have photos of a recent trip to Sydney, Australia also tagged with “Sydney”. Uh oh, now we have a problem. How can the computer figure out that my photos are not related to my lab? Nepomuk provides an answer to this.

Nepomuk contains a number of types that let you describe things more accurately. These types are included in things called Ontologies, but don’t worry about that just yet, just get familiar with the word. Anyways, instead of tagging my todo with just the text “Sydney” I could instead tag it with the PIMO::Person “Sydney” (when I use PIMO I mean Personal Information Model Ontology, one of a handful of namespaces for nepomuk). Now the computer knows that I’ve associated my lab with a person rather than just some text. Additionally, I could give it a location and so on. The real power in this is twofold. It opens up a whole new world for data visualization and it provides a good system for locating data-two things that we’re doing more and more of. Lets start with the data visualization. Instead of just seeing a list of tasks and hovering over/selecting them to see a list of tags, I could design an application that shows the task and then shows the people assigned to that task (and their faces or email addresses/phone numbers, site), the location it will take place in on the map (marble!) and the due date on a calendar instead of just a series of numbers/letters.

This leads into finding my data later on. Search is becoming a critically important part of the desktop and web experience. However, in a lot of ways our search is still quite primitive. It’s a keyword search that looks through the text of documents and filenames. Imagine a search client that let you specify that you wanted to find PIMO::Task objects that are assigned to a certain PIMO::Person and are due before a certain time. This is already way more useful than just searching for “Sydney” because I won’t get any of my photo results, and I won’t get any results that are of the wrong type (i.e. not tasks).

So, without touching any code I’ve tried to abstract away one use-case showing why Nepomuk might be useful. Hopefully it is a decent introduction, but if you’ve noticed errors or flaws in my understanding of Nepomuk feel free to point them out in the comments. I’m still learning about it myself. In my next post I’ll go into some details on my first experiences with actually working with Nepomuk and Soprano APIs and how I created a (very) mini example program.

You may remember me blogging about the remember the milk plasmoid before. I’ve finally had more time to work on it, and yesterday (wow, it’s that late) I re-enabled the rest of the features that had gone missing in the last few weeks. This marks another milestone, the applet is now in kdereview.

Around the start of April I felt that it was in that ‘almost ready’ stage, so I went and took a second look at the choices I had made in the library and dataengine. Wow, it’s amazing how some decisions that seemed like fine solutions at the time turn out to be horrible hacks. I ended up rewriting a huge portion of the library to be more consistent and qt-like. This provoked a cascading effort to rewrite the dataengine to actually be useful and then the plasmoid to use the dataengine instead of bypassing it.

The rememberthemilk plasmoid has certainly changed since I started working on it in early January. It has transitioned through a number of stages, from concept to library to plasmoid to dataengine and then some. The data engine is a heavy user of complex sources, services and jobs. The plasmoid relies entirely on the dataengine, it doesn’t even link to the library. The library makes use of KDE technologies such as KIO and has a very kde like api.

Hopefully the plasmoid is just the start. I’ve already started on an experimental Akonadi resource, modeled after the wonderful Google Data resource. I’m also contemplating a KRunner plugin. Wouldn’t it be great to add tasks with “rtm newtask My New Task due Thursday” or similar? It could also support search queries.

Below is the first iteration of the plasmoid, and then its current state.

Next up… TimeVault. I’ve been letting it (and the ideas behind it) simmer for a while, but after my finals are finished in early May I want to get back into the thick of development. It’s about time, KDE needs an integrated backup solution

In the comments for my previous post, Simon Edwards suggested that I write up some of my experiences with Python and DBus in a PyKDE4 tutorial. Creating (good) tutorials isn’t always easy, especially for the developer who is more focused on his code and only spends his free time helping out. However, when I saw that comment I remembered how crucial the tutorials on techbase were for me when I was just starting out with KDE development not so long ago. I decided to give it a go, and I now have my first tutorial up, the PyKDE DBus Tutorial. For now it’s a general introduction to DBus and qdbusviewer (which is useful to non-python programmers as well) along with some basic information on using already present DBus interfaces/methods from Python. I hope it helps someone out there, and don’t hesitate to add to it ( yay wikis! ) or solicit improvements.

Secondly, I’ve been mulling some more over my pet KDE backup project, TimeVault. I’ve got some pretty high standards for it if I ever get around to making it something releasable, so don’t expect it anytime soon. However, that doesn’t mean I haven’t been working on it. At the moment it’s using a rdiff-backup backend to do the heavy lifting. The problem with this is that ideally I would like continuous snapshots. This is theoretically possible with inotify, except that it doesn’t scale well to the entire filesystem. Today I did some research and stumbled upon Dazuko, which sounds like it might fit the bill. It exists as a FS overlay to any path in your filesystem. From there, applications can monitor the file requests and respond to them. It was initially created to allow antivirus applications to do on the fly scanning/cleaning, but it should work well for monitoring changes to an entire filesystem.

Thirdly, I’ve been hearing murmurs of a KDE blogging application called Bilbo Blogger. I had been wishing for something to replace the not-so-great web interface to WordPress, and a KDE4 client seemed like a dream come true. Granted, wordpress’s online editor really isn’t that bad, I just think I’m biased against web interfaces. I use Kontact for my gmail/uni email, Akregator for my feeds, Choqok for my identi.ca needs and so on. While others seem to be moving ‘to the cloud’ I seem to be syncing and interfacing with the cloud instead, which fits my ideas about networking more. Anyways, back to blogging. I downloaded the Arch Linux package for Bilbo Blogger using the excellent yaourt and AUR, and it seems like a great editor on first glance. More on that later.

Yesterday Sebas blogged about his desire for a small plasmoid that would allow him to suspend his computer when the screen is locked.

Since I already wanted an excuse to play around with a Python plasmoid, I decided this would be a cool little applet to code. Turns out that there is a (pending) freedesktop.org interface to power management and PowerDevil implements the important parts of this.

I started out with the great Python+Plasma tutorial and the second one on using the Plasma::Widgets that we know and love from C++. Couple that with the python bindings for dbus, and you’ve got yourself a little applet that suspends your computer at the press of a button.

You can find the applet on kde-look, and it should be present in GHNS as well. (working on this point… for some reason it’s not showing up for me).
Edit: Ok, now it’s showing up. I guess it just took some time.

It’s time for another update on my work on the remember the milk plasmoid. It’s shaping up nicely, and I’m often amazed at how nicely the different Qt and KDE libraries fit together. However, there are those times when I bang my head against a wall for hours trying to fix some bug or implement some feature only to discover that it wasn’t my problem in the first place.

This happened to be the case with drag and drop support for the plasmoid. It started off with an itch to have basic drag and drop support to change a task’s priority. I wanted to be able to move around my tasks visually instead of going into an editor and changing the priority. It turns out that QGraphicsScene/QGraphicsView had some bugs when dealing with widgets on canvas. For those who haven’t worked with woc, it’s a great idea. Putting any graphical widget onto a canvas that already has support for transformations and scaling and the works is a wonderful thing to be able to do, and it allows for things like Plasma to happen without lots of work reinventing (or reimplementing) the wheel. However, Qt 4.4 was the first release with WoC support and suffered from some bugs. I was quite frustrated, not understanding why my dragged task was offset, or why I was never getting any drop events, when I finally decided to see if the Qt 4.5 RC fixed it. After a rebuild I restarted plasma, and much to my surprise, it all worked! That’s another +1 to 4.5 from me, as drag and drop for rtm tasks is impossible without it.

In other news, I’ve gotten some great feedback from some brave souls willing to test drive the plasmoid. Thanks guys, and sorry about the mismatched commits that was leaving you all stuck at “refreshing” after authenticating. But anyways, your suggestions have lead to a few fixes already and have also got the ball rolling regarding displaying more complex data (and searches). As you can see in the screenshot, the tags and the due date text items (on the right side of the tasks that have them) are now italicized and smaller than the task name. Hopefully this should provide some visual hints to what a user needs to read to get the overview of his tasks (hint… that would be the task titles). I’ve also ellipsized (not a word, but you get the idea) the ends of tasks that are too long to show in the width of the applet. This works correctly whether or not the task has a due date and in my opinion vastly improves the visual look of the tasks.

But hiding part of the task leads to the question of how does the user see the full task name? At the moment, clicking on the task will bring up the edit window, which then shows the task name at the very top of the applet (where “Remember The Milk Tasks” is currently), but this isn’t good enough for me. I’ve got a few options for this. I could provide a tooltip upon mouse hover, but that is ugly for tablet pc users (hovering the pen in one place is not easy), and it also breaks the visual identity of the applet. I could also show the task on two lines (eliding it if it’s longer than that) which would work as well, but would add a lot of visual information and clutter to the applet.

I’m not happy with either of the above options, but I think I can tackle multiple problems with the addition of a larger task view. Ideally, for each task I would show a little “show more” arrow upon mouseover, which, when clicked, would smoothly expand the task into a more detailed view. This view would show the full task name and other data such as the notes attached to the task, whether it’s been postponed, the location and other people set, etc… This might even become the editor interface, replacing the faded in overlay that I have currently. The problem is that this might be difficult with my current implementation of the tasks view widget. I’m currently using a QStyledItemDelegate and a standard Plasma::TreeView to show the tasks, but I’m wondering if it would be better to move to an all QGraphicsScene based approach. I’ve heard that QGraphicsView layouts had some serious issues in 4.4, but hopefully these are also fixed with 4.5. Moving to a QGraphicsView would also allow some nicer animations as I wouldn’t be held back by the limitations of widgets on canvas. But anyways, I’m happy that I’ve even gotten this far, this is one of my first larger projects and there is certainly time for enhancements.

One last little thing. I made a blog post earlier on some ideas for a linux backup system made for end users. It’s slightly off topic from KDE and nowhere near a reality, and so I didn’t post it on the planet. However, if you’re interested in chiming in with your thoughts and ideas about backups, please feel free to read it and add your comments. http://blog.chatonka.com/2009/02/timevaultng-structure-document-draft/

Backups are crucial. If we all know that, then why do few people have a complete and tested backup system? Some people even fail to have any backups at all. A huge part of this has to do with simplicity. Backups need to be easy to set up, painless to maintain, and to work when you need them. This isn’t an easy task, and so far I haven’t seen a good solution for end users on Linux. For the moment, I’m backing up my drives using some scripting foo and the great (command line) utility rdiff-backup. Late last year I played around with a KDE frontend to rdiff-backup, but it was overly complex and ended up not going anywhere. With the results of that in mind, and some helpful discussion with others via email, I’ve come up with a list of qualities and abilities that I think are needed in a good desktop and end user backup system. This can also be considered a draft of a design document for TimeVaultNG, currently hosted at http://launchpad.net/timevaultng

Timevault should provide the following:

1. A single (rdiff-)backup for the entire filesystem. The partition state is saved so that it can be restored if the (advanced) user desires to, but it also supports (and possibly defaults to) just restoring to a single partition.

2. A backup daemon that runs as root/trusted user that does the backups and communicates (via DBus) with…

3. A KDE frontend (and possibly others) to manage where to backup, exclude lists, etc… This frontend has a very simple initial backup setup which consists only of selecting the backup drive and clicking enable. From there the daemon backups up on (for now hourly) interval whenever the backup drive is plugged in.

4. A set of “exclude plugins” which exclude specific parts of the system. For example, there could be an exclude plugin that does not save any of the Firefox cache. Application developers/system integrators could provide other exclude plugins depending on their setup.

5. A restoration utility that is provided on a livecd (and can also be included on a distribution’s livecd) that can restore in the following manner:
a. It wipes the target disk/partition
b. It (optionally) recreates the partition scheme
c. It (optionally) installs the base operating system
d. It installs the backup overlay on top of the disk
e. It ensures that the system is bootable (i.e. fstab matches partition layout, that initrd is correct, that grub is installed)

6. A utility (possibly built on top of dolphin) that lets users browse snapshots. They can see the current version of a directory and then “travel back” to see earlier version. This could be implemented in a kioslave which would allow any KDE application to directly access the (read only) backups.

I’ve received some great feedback on the Remember The Milk plasmoid and it’s time to share some of more recent changes.

My university courses started last Monday, and with them the inevitable shrinking of free time to hack. However, the workload hasn’t quite piled on yet and I’ve spent some time in the last few days on the core of librtm. Its handling of tasks is vastly improved, and it is now possible to work directly with RTM::Task objects, as they will send their updates (i.e. new names, due dates, tags, etc…) to the server. When the server responds to these requests, the RTM::Session receives the replies and keeps the internal state of tasks consistent. It helpfully emits taskChanged(RTM::Task*) so that any frontend application can stay up-to-date in whatever way it handles tasks.

I also made some enhancements to the plasmoid, the most visible of which is the new task configuration widget, shown below. When you click on a task, a widget slides into view that lets you edit its attributes. Upon clicking save, this dialog slides away and the changes are sent (via RTM::Task, enabled by the changes I mentioned above) online. The view updates itself when it receives the updates from the RTM::Session by way of the dataengine. Additionally, there is now a visible indication (the colored header items) of priority.

The plasmoid can be found in playground. To build it you need at least KDE 4.2rc, librtm (playground/libs/rtm) and the rtm dataengine (also in playground). Remember that it’s still in heavy development and I can’t guarantee that it won’t mess up your rememberthemilk account. Also, there are a number of visual bugs apparent even in the screenshots. I know about them, I’m working on them.