13 Dec 2009
TimeVault Progress Update
Edit: After speaking with Nuno and Mofux I’ve made some modifications to the System Settings page. This is how it looks now:
TimeVault is alive! Ok, so maybe it was never really dead. But I’ve found some time to work it and after reworking most of the internals I’ve finally gotten it to back up again.
For those who don’t know, TimeVault was originally a gnome backup system that I used way back when I was on Ubuntu. I liked the concept but disliked the implementation and so sometime after moving to KDE I started working on my own TimeVault. The goal is to have a user friendly, dead simple backup system that runs in the background, knows about (and exclusively uses) external media and isn’t afraid to integrate into KDE frameworks wherever it can.
Now the project has deviated quite a bit from the Gnome one with the same name. With that in mind, I’m contemplating a possible name change to reduce confusion. If people have suggestions feel free to leave a comment or send me an email.
The following is an image of the KDE System Settings page for TimeVault. It shows the current state and allows the user to select the backup drive and the folders that should be backed up.
Some new things of interest are:
- TimeVault no longer relies on a daemon running as root
- New KAuth integration (a part of KDE SC 4.4) which allows the system settings module to have it’s authorization bit
- Allows the user to select which paths are to be backed up
- Provides the user with notifications regarding the backup status (KNotification)
- Runs the backup with a KAuth helper running as root, much more secure than before
Some things that I still need to work on:
- The KIO slave has been neglected and no longer works. This is high on my todo list
- A dedicated restore framework is not yet in place. If the KIO slave isn’t good enough this will be needed
- The plasmoid also was neglected. I would like for it to contain backup statistics as well as information on the currently running file. Possibly also adding files/folders to the backup by dragging and dropping them on the plasmoid.
If you have ideas for me feel free to post them in the comments. I know I can’t think of everything that’s useful to a wide swath of people.
How about making it possible to work with gnome (not necessarily gnome’s TimeVault, just not too kde specific)?
Anyway, it looks great, thanks!
hkariti
December 14th, 2009 at 12:12 ampermalink
I would like to have it reasonably platform antagonistic, and I’m trying to make some decisions based on that such as using D-Bus and PolicyKit. But many things are tied tightly to Qt and kdelibs so I’m not sure how easy it would to be to separate things in the future.
astromme
December 14th, 2009 at 12:30 ampermalink
Hi, seems like great piece of software here, I’d really love to check it out, though I have some compilation problems (pulled git repo).
Problems are as following http://pastebin.com/d221a3208
I installed all policykit-related packages I could (except gnome ones) and still it won’t compile, could you help me with that somehow?
Thanks, Kuba
Kuba
December 14th, 2009 at 2:04 ampermalink
Could you maybe also update the README file? It’s a bit rough, and doesn’t help much.
# sorry for double posting
Kuba
December 14th, 2009 at 2:08 ampermalink
Looks very nice!
).
One question that comes to my mind is: do the external discs have to be running continuously in order for the backup to work? In my case I have them switched of most of the time. Maybe some Notification once a week or something like that?
Also in my case a possibility to backup specific directories to specific external harddrives would be useful (-you where asking for ideas
Thanks!
Jo
Jbraun
December 14th, 2009 at 2:38 ampermalink
I would recommend making it NOT DE agnostic. Please, integrate it as much as possible, the beauty of KDE is all the integration and how things talk to each other. It’s like the network effect.
The biggest reason I use KDE is all the integration, something which many DEs, Windows and GNOME especially lack.
saem
December 14th, 2009 at 2:39 ampermalink
I haven’t tried it out, so it’s possible that this suggestion is already implemented, but I didn’t see it in the screenshots: A way to configure for how long backups will be kept. This could be by default a simple “keep all backups for x days”, but I’d love to see more fine grained controls (ex. keep backups made on the first Sunday of the month for a year, keep everything else for 17 days) provided as well. A wish which is likely influenced by my (not so) subconscious desire to have TimeVault be Dirvish with all the flexibility of that system but better KDE integration.
elyk
December 14th, 2009 at 3:30 ampermalink
Hi there!
?
I’d also opt for a KDE solution. As such, I was just about to suggest to call it konserve, when I found out there’s already been a backup application of that name, http://konserve.sourceforge.net/history.html . However, its last release was more than five years ago, so maybe you could simply hijack the name
Rufus D
December 14th, 2009 at 3:54 ampermalink
Flavio Casteli is also working on a backup system (Kaveau) for KDE, why don’t you guys work together?
Arnaud Vallat
December 14th, 2009 at 4:32 ampermalink
This is looking great!
FYI, at Identi.ca I noticed some trolls were working on ‘coldstorage’, a backend for snapshot-based storage. Perhaps this can be helpful for you too?
Diederik van der Boor
December 14th, 2009 at 5:16 ampermalink
I’ve got a few names…
Karbonite
Kold Storage (but I just noticed that someone mentioned something else called “coldstorage”)
Kache
Keep Safe
Krystalise
Christopher Brian Pye
December 14th, 2009 at 5:28 ampermalink
If you are considering a new name, please consider one without a capitalized K. A lot of programs are moving away from that convention, perhaps just Vault.
Saleel Velankar
December 14th, 2009 at 5:45 ampermalink
I would REALLY like to see a tool like this that can backup the .kde and the user’s mail directory.
I’d further like that facility to be able to selectively turn off certain things whose storage is bulky and can be recreated from elsewhere, such as the storage for disconnected IMAP and the album covers/database used by Amarok.
Architecturally, in the long term, this might be something that each KDE application can configure “areas” that can be included/excluded (e.g. via .desktop settings).
For me, this would really set a backup tool apart from the crowd, and really add KDE (or even, fdo-wide) value for the user.
ShaheedHaque
December 14th, 2009 at 7:13 ampermalink
Seems very promising. I would opt for the KDE integration as well. As for me the most important features are:
- option full/incremental backups with scheduling
- option for compression algorithm and level of compression
- prompt for mounting the external backup drive, otherwise backup locally and as soon as the external drive is available move archives there
The last one might be useful when you travel with your laptop and don’t have your external hard drive with you. Maybe there is a better solution.
Keep up good work!
Grzegorz Kowal
December 14th, 2009 at 7:23 ampermalink
I’d vote for Carbonize without the K. But yes, I want to see in be with KDE specific. Also is there a chance for SystemSettings module backend for it also? I really love to use this application.
Keith Rusler
December 14th, 2009 at 7:43 ampermalink
My suggestion: keep it simple!
Why do you bother the user with a green LED-like indication called “deamon status”? Why does the average user even need to know about this, especially since there seems to be no problem anyway…
It is also unclear what I am supposed to do with the media display. Why do I need to know the free space of other places than the place where I am going to make my backup? And why do I need to know the free space of the place I *am* going to make it, provided that space is big enough?
Then: would it be possible to add directories to back up from any file browser? I mean: why do I need to go to TimeVault to add directories to a backup? Why not have it in the right-click menu of any directory not already in the backup, and have a “backup this directory now” option in that menu if it is included already?
Andre
December 14th, 2009 at 8:02 ampermalink
As far as the name goes, how about Chronos, or Kronos or a of the sort?
Aristotelis
December 14th, 2009 at 9:31 ampermalink
I vote for it’s current name: TimeVault. Simple and catchy, without an overload of K’s.
cumulus007
December 14th, 2009 at 9:33 ampermalink
Please, dont listen to the previous poster. Knowing the free space of the back up media is crucial as were the other options that he wanted to remove.
Gonzalo porcel
December 14th, 2009 at 9:37 ampermalink
Time Kapsule
Never forget the K!
Jim
December 14th, 2009 at 9:40 ampermalink
Time Kapsule ftw
keep the K, though without much noise.
Please keep the application simple yet functional, I know it’s hard to do, take a look at this post by Mark if you need some help in this regard: http://amarok.kde.org/blog/archives/1132-Micro-Options,-Many-Options,-No-Options-A-practical-guide-to-help-you-decide..html
# compilation/cmake help still needed
Kuba
December 14th, 2009 at 9:45 ampermalink
Wow, lots of feedback, thanks all!
Note to anyone who is compiling:
This requires KDE SC 4.4 and PolicyKit. The latter is often configured wrong on distros, I had a lot of frustration with both openSUSE and ArchLinux.
@Kuba
Yes, the stuff on gitorious is very outdated. I’ll get around to updating that soon, thanks for bringing it to my attention.
As far as your specific error goes, I had that same problem on ArchLinux and ended up ignoring that warning because I had everything installed. Do you get an actual error or does ‘make’ then work?
@Jbraun Nope that’s part of the framework
It will leave them unmounted and you can disconnect/connect them freely. I don’t have the gui for it yet but it should also notify you when it thinks you should do a backup if your drive is connected.
@saem don’t worry I’m using KDE frameworks and integration wherever I find it useful
. As of now I count a kded module, a system setting module, KNotification integration, dbus (which isn’t strictly kde but used throughout it), KAuth integration for nice password screen dialogs, use of solid for detecting backup drives and probably more that I’m forgetting about. Of course it would be nice to share the same backend tech, but it’s not a requirement.
@elyk That’s important for me as well. The ability to do that is very dependent on the backend. I’m currently using rdiff-backup to do the heavy lifting and I’m pretty sure it can forget about backups older than x days old. However I don’t think it supports merging snapshots (which is what is required for the more finely grained controls). I looked at making my own backup with xdelta or git or other tools but I wanted to get a solution in the next year rather than in the next decade so I opted for rdiff-backup.
@all of the name suggestions
)
Unless the K fits very well into the name (and is the normal, or an accepted way of spelling it in english) I don’t think that I’ll just find a cool name and put a K where a C should be. Sorry. However, I really like the following names (not saying I can use them or will use them, but still
Vault
Cold Storage < – That is such a good name, too bad they got to it first.
Carbonite <– already a windows tool with this name iirc
Carbonize
@Arnaud Vallat
I’ve looked at Flavio’s work. It’s good, but I think he is catering more to the tech crowd. I want this to be a dead simple fire and forget backup system.
@Diederik van der Boor
Thanks! Yes, I’ve looked at Cold Storage, but I really don’t feel that git is a good solution for long term backup. It’s binary deltas are very inefficient (non existent?) and it makes so many assumptions when storing text data that anything other than source files isn’t really fit for storage in git. I also am not sure how it would react to a full filesystem backup (which is my plan).
@Saleel Velankar
Yep, don’t worry I’m not at all interested in having the K ‘just because’
@ShaheedHaque
Yes, me too. I found myself going through my backup selections and removing every build directory to start, and already wishing for more finegrained control. I’ve come up with almost the exact same idea you have, where individual applications tell the backup system what it should back up. If you want to help me make this a reality I would love to work on it. It’ll be a lot of work, but I think you are right it’s where we need to go with backup systems. They need to know about their environment. Perhaps there is some room to connect with Nepomuk/Strigi for this.
@Grzegorz Kowal
Right now after the initial backup all backups are reverse incremental. This means that the most recent copy is kept in full and all snapshots are binary deltas to past versions of files. To me this is an acceptable solution. As far as compression goes, I’d rather not deal with it on the backup system level. If the drive/filesystem decides to do compression great, but otherwise I don’t feel that it’s all that useful. Backing up locally is an interesting idea but I’ve never found someone with enough space to store a copy of their data locally. At least I’m not like that and it would add a fair amount of complexity. But a good idea for sure.
@Keith Rusler
There already is! I’m showing you the System Settings module in that screenshot.
@Andre
You make a very good point. That status is from the time when the running-as-root daemon wasn’t very good and I was worried it could be enabled but not actually working. Consider it removed
I also took another look at the page this morning before I read these comments and decided the same thing you did… “woah that’s confusing”
A kind of dual free space thing where the amount of needed space is overlayed on top of the free space bar could work well… not sure if it’ll be hard/too much information. I’ll experiment with it.
Yes, I want to do this. It shouldn’t be hard and it is important. But remember also that this is supposed to be a fire and forget backup system. As far as a “backup this directory now” goes, I’m not sure if it is possible with the current framework. If rdiff-backup can do per-include snapshots I would love to know how.
astromme
December 14th, 2009 at 10:03 ampermalink
@Kuba
I’ve talked to drf and he says that the issue you’re encountering is actually a bug in kdelibs. They’ll have it fixed soon. Thanks!
astromme
December 14th, 2009 at 10:53 ampermalink
Having just tried to restore akregators feed list after a KDE screwup (the file was filled with data from my cookie file because the system was shut down due to battery running out), using kbackintime, I’ll happily try any other program that looks sane, so I will have a look.
Restore framework sounds very good.
Anders
December 14th, 2009 at 11:09 ampermalink
How about Tardis, built by the timelords and way more powerfull then time machine.
Jasmin
December 14th, 2009 at 1:28 pmpermalink
Will this handle simple backup to a network storage unit or drive?
Joshua L. Blocher
December 14th, 2009 at 5:33 pmpermalink
@Joshua
Sorry, that’s planned right now. If I find that there is a simple way to work with network (I assume you mean local network storage, not over the internet storage) storage then of course I will consider it but right now it doesn’t fit into the plan.
astromme
December 14th, 2009 at 6:17 pmpermalink
Please for the love of God, Time Capsule! WITHOUT THE K! *grumble grumble*
cabrey
December 14th, 2009 at 6:42 pmpermalink
@cabrey: “Time Capsule”
… has already been taken, and would certainly result in trademark infringement, since Time Capsule is an Apple disk drive + router made for, you guessed it, backups.
Personally, I would vote for Keep Safe except that Keep is already a KDE backup program, abandoned just like Konserve.
Michael "names" Howell
December 14th, 2009 at 9:15 pmpermalink
Is this able to continuously backup a truecrypt container file?
If it can that would be awesome.
DImble
December 15th, 2009 at 4:45 ampermalink
This looks promising. Does it then have somekind API to allow other applications to have their files included for it from settings?
What comes to the name, I would prefer that there is the K. It makes it clear it belongs to the KDE SC and can not be used on GNOME, we should not be afraid about K. It just makes it right away clear it is for KDE SC, and it is always easier to talk about application when it has a unique name and not typical one. Example, how easy you believe it will be to find with Google with “Time Capsule” or with “Time Kapsule” names?.
And when it comes to KDE SC + GNOME situation, I would take this for KDE SC only and then make a own for GNOME. We should follow as much as possible the standards but we can not start developing software for GNOME as well.
The GUI does have nice ideas. I just hope it will be possible to add/remove folders/files to be backupped from Dolphin etc. Mayby to get a own sidepanel if not possible to integrate to the Information (F11) panel. It would be same way great to see something similar on Dolphin view what is now in Back in Time KDE4 version, easy way to browse just date and you can browse archives. I would like to see being used a calender view (like plasma widget when clicking a clock) on sidepanel as well and seeing a dates when backups are available (files have changed) and clicking date it would show the file.
I really believe that if we get the backup/restore function to the sidepanel of Dolphin, we get normal users to use it more often and easier way. I do not want to search from backups from some external application. I just want to see the last version if needed. That was something what made Apples timemachine so great. It just was so intuitive, even we have had a rsync scripts and other apps for much longer.
I was thinking one feature for that external drive thing as well. It is not always enough we do have one backup somewhere. we should have possiblity to have a backup for two different external drives. So it would be like adding a multiple drives as clones. So you would have a system where you have once a month full backup and then you move it away and you backup daily to first backup drive. Then again when you attach the second one you get it synced. So just having added a (1) and (2) icons with numbers somewhere the UI would be nice.
And what is very important! Is that when you do not have a drive mounted, it should not create the directories and start backupping there! There are some backup softwares what does not check is the drive mounted or not to the /media/. When the drive is unmounted, the software creates (runs as systemwide as root) /media/backup_drive/wanted_path and fills the /media directory and usually the whole / partition full and normal users starts having lots of problems then.
Fri13
December 15th, 2009 at 6:37 ampermalink
I’d vote for WITH a K in the name, so that we know its a KDE app. Whats with the backlash to branding as a KDE app these days? When I get a KDE app, I know its going to be nicely integrated into my desktop and will have useful features.
Name suggestions: Kronos, DataKeeper, or even just KBackup. Oh how about KPasa, sounds like the Spanish ‘¿qué pasa?’ meaning, ‘whats going on?’ (or KPaso = qué pasó = what happened)
This is expected to use symbolic links much like RSnapshot does, correct? So that we can go back in time and have various weekly or monthly backups that don’t use extra space unless files change?
lefty.crupps
December 15th, 2009 at 8:29 ampermalink
[...] TimeVault Progress Update TimeVault is alive! Ok, so maybe it was never really dead. But I’ve found some time to work it and after reworking most of the internals I’ve finally gotten it to back up again. [...]
Links 15/12/2009: Elive 1.9.52 Released, Nook Liberated | Boycott Novell
December 15th, 2009 at 8:57 ampermalink
Just pulled from GIT and compiled fine but I found few problems.
1. Readme said about building daemon first. By first mkdir build, then cmake .. and then cd daemon. But the daemon directory is not created. But I tried make -j 2 and everything went fine and installed.
2. The Timevault does not show up on system settings. Not even I relogin. There is a command timevault and timevault-backup but last one gives something and stays there, while first one shows a timeview on small window.
And how about using name “Kopio” what is Finnish and means a “copy”. You could as well use word “Klooni” what means “a Clone”. You get K for the KDE SC and even meaning for copy/clone function and respect even for country where Linux OS was born (even that KDE SC runs on other OS’s well than just on Linux) =)
Fri13
December 15th, 2009 at 9:25 ampermalink
@Fri13
I updated the README yesterday, are you sure that you got the most recent version?
Also I want to stress that this is not going to be ready for use yet. I wish it were, I need the app as much as anyone else. I’m glad people are excited to try/test it out but I guarantee you that it’s not ready.
@lefty.crupps
No it uses rdiff-backup behind the scenes. You should google that, it uses reverse diffs rather than symlinks and new folder hierarchies.
@Dimble
Sorry it won’t be able to continuously backup an encrypted truecrypt container. Although I’m slightly confused by what you mean, so maybe I just misunderstand?
astromme
December 15th, 2009 at 11:08 ampermalink
@astromme
I just pulled the mainline first time. Plan was just test the UI to get more accurate understanding. Not to use it for real backup. I still use manually done work =)
(and for adding more name suggestion based Finnish, a slang verb “kopsata” for the action/verb “kopioida”.)
Fri13
December 15th, 2009 at 12:58 pmpermalink
@fri13
Hey sorry, apparently I forgot to push my changes from my local checkout on my laptop. Things should be updated now.
astromme
December 15th, 2009 at 3:32 pmpermalink
Re: Trucrypt Container Files.
sorry if i confused matters.
i would like to be able to use your back up utility to manually backup my TC container when it is dismounted.
at the moment i delete the file and re-copy it across to my USB drive, but it would be great if i could click “update SDA2″ and have it quickly adjust the backup for the diffs between the main container and the container on the USB drive.
is that possible?
DImble
December 21st, 2009 at 4:52 ampermalink
It’s looking good! Well done!
Please keep it integrated with KDE.
Sorry, maybe I missed that bit of info… does it support network and local storage for backup?
Say, you have a big NAS on your network, and want to save your backups there (I have this at work). Or you have a big spare disk on your desktop pc and want to save your backups there (I have this configuration at home). As you see, I’m quite interested
IMHO it also should do some “smart” backup removing, when the backup file system is full, leaving one copy of last week’s backup and (if available) one copy of last month’s backup, and so on, since you’re likely want to restore something from yesterday’s backup or last month’s backup, but most of the time you won’t care to restore something from *exactly* 42 days ago.
gerlos
December 13th, 2010 at 8:32 ampermalink