Mar 30, 2023, 7:10 AM
In my last post I talked about Obsidian. Here's a cool thing I figured out for syncing an Obsidian vault across multiple computers, even mobile devices.
Obsidian does offer a paid tier that includes a syncing service. I didn't want to pay monthly just to sync my notes, when I figured I could do something similar myself. So I dug in.
First try - Dropbox
First I tried just putting the whole vault in Dropbox. And this kind of worked. But I started getting conflicts in my .obsidian folder.
For a little technical background, an Obsidian vault is mostly just a folder full of markdown files, organized however you want. It can just be a giant dump of files, or they can be organized into whatever folder structure works for you. In addition, Obisidian creates an .obsidan folder that holds metadata about the vault that Obsidian needs to work.
The .obsidian folder contains a few folders and a bunch of .json files. These contain all the settings you set in the app, your theme, info about your plugins and their settings, etc. One of these files is workspace.json. This contains all the info about the current state of the Obsidian user interface - what files are open, tabs, splits, positions, recent files and all kinds of stuff like that. Whenever you open a file, close a file, change a tab, open or close a panel, etc. Obsidian updates the workspace.json file.
So, say you have your vault in Dropbox. And you have Obsidian on two computers using that synced vault. You create a new file. Dropbox starts syncing that file to your other computer. At the same time, Obsidian updates the workspace.json file. And a second later, Dropbox stars syncing that. On the other computer, Dropbox creates that new file for you. That causes Obsidan on that other computer to update its workspace.json file. And then Dropbox will want to sync that since it has a change. But just about then, the changed workspace.json file from the first computer is coming across. Dropbox now has a folder which has been changed on two different computers, so it marks it as being in conflict. At least that's my best understanding of what's going on there, and it seems to bear out in experiments.
Second try - Syncthing
My thought was to just not sync the .obsidian folder. Only sync the markdown content. There was not an easy way to do that in Dropbox, so I turned to Syncthing. I won't go too deep into how Syncthing works, but in short, you set up a folder you want to share and you can accept that share on other computers. Any changes in that folder on any of the shared computers will sync to the rest.
But most importantly, you can set ignore patterns. So I just ignore the .obsidian folder. The way I set it up is I have my home server that is the master copy of my vault. I don't actually even use Obsidian on that computer. Its just the center hub that my other computers and phone sync to. Of course, if that server goes down, nothing syncs, but that's ok with me for now.
So now, I create a new folder on one computer. Obsidian updates workspace.json and Dropbox syncs only the new file. The home server gets the new file and the other devices all get the new files shortly after that. When they get the new file, Obsidian updates the workspace.json file on each of them. But since Syncthing ignores that, there's no problem.
One downside is that I had to set up prefs and plugins, etc. across each device separately. And if I make a change to a setting on one computer that doesn't sync. I could try just ignoring the workspace.json file and syncing the rest of the .obsidian file, and I might try that at some point. Not sure if any of those other settings might cause sync issues. If not, it would be nice to have the settings sync across devices.
I will say that this is not a 100% failsafe against conflicts. I still occasionally get one. I think what happens is that I edit a file and it syncs. Then I open my laptop, open Obsidian and edit the same file before it has had a chance to sync. Now the sync comes in and a sync is going out, and that's a conflict. But it's pretty rare at this point, and mostly preventable by waiting a few seconds after opening my laptop before editing a file.
I did initially try syncing all devices to each other and I believe it made things more prone to individual file conflicts like this. So I stuck with everything syncing to the hub. But now I'm kind of doubting myself on this point, since it seems like that should not be a problem. Might be another thing to investigate.
Backup
One final things I wanted to do was keep a running backup of my vault. Since it's almost all just text files, with an occasional image, git seemed like a good solution. Oh my home server, I set up a script that looks roughly like this:
#!/bin/bash
commit=$(date --rfc-3339=seconds)
cd /location/of/Obsidian/
git add .
git commit -m "$commit"
git push origin master
This creates a commit message that's just a time stamp. Changes to my Obsidian vault, adds, commits and pushes. I have this set up to run a couple times a day with a cron job. There's a few extra things in there that I'm not showing, like using a specific ssh key and pinging Healthchecks so that if it ever stops backing up for too long, I get a notification.