One of the good things about encfs is that it allows you to keep your encrypted directory backed up on a removable disk or remote server, without having to decrypt and re-encrypt the data. And if you exclude the configuration file (that contains the key, which, in turn, is encrypted with a passphrase of your choice) from the backup, then you don’t have to worry about whether someone else could get a look at your backups — all they’d ever get is random gibberish.
The bad thing is that the config file normally sits within your encrypted directory. So if a Bad Guy manages to gain access to your primary disk (e.g. by stealing your laptop) and figure out your key-encryption passphrase, then you’re toast.
But encfs allows you to use an external program to get up to 2 KB worth of passphrase. So you could pick any nicely inconspicuous file (e.g. a photo of your cat) on your thumb drive, and mount your encrypted directory using a command line such as
encfs --extpass='cat /media/flashdisk/kitty.jpg | sha512sum' \
Now your cat photo would serve as a key to your encrypted data, so you better make a couple of backup copies thereof (e.g. on Flickr where you could always re-download it when you happen to lose your original). Of course there’s another obvious problem: if the Bad Guys get your laptop they’d also get your .bash_history and might want to come back to steal [a copy of] your cat photo.
So you might want to modify your encfs command line to ask you for the key file without leaving any traces into your .bash_history.
encfs --extpass='cat `ssh-askpass` | sha512sum' \