-
Notifications
You must be signed in to change notification settings - Fork 14
Allow separate mail and state paths #33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
1e18d7b
c50bc1e
d27a82c
2e793b4
b77f15e
bba235c
a559001
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -54,11 +54,43 @@ password_command = "pass [email protected]" | |
|
||
# convert_dos_to_unix = true | ||
|
||
################################################################################ | ||
## Path config | ||
## | ||
## mujmap needs places to store email and bits of working state. These paths | ||
## can all be specified in the config file, but most of the time you don't need to | ||
## do this and it will choose reasonable defaults. | ||
## | ||
## mujmap has two "configuration modes" that inform how path defaults are chosen. | ||
## | ||
## In "directory mode", the --path option points to the directory containing | ||
## mujmap.toml. In this mode, the default locations for mail and state files will | ||
## be in subdirectories under this directory. This is the right mode to keep the | ||
## entire mujmap data storage in a single place. | ||
## | ||
## In "file mode", the --path option points to a config file. In this mode, the | ||
## default location for mail will be determined from notmuch's `mail_root` config | ||
## variable, while the default location for state is operating-system specific. | ||
## This mode is intended for tighter integration with notmuch "profiles". | ||
|
||
## The cache directory in which to store mail files while they are being | ||
## downloaded. The default is operating-system specific. | ||
## downloaded. It must be an absolute path. The default is operating-system | ||
## specific. | ||
|
||
# cache_dir = | ||
|
||
## The location of the mail dir, where downloaded email is finally stored. It | ||
## must be an absolute path. If not given in the config file, mujmap will choose | ||
## an appropriate default for the configuration mode. You probably don't want to | ||
## set this. | ||
|
||
# mail_dir = | ||
|
||
## The directory to store state files in. It must be an absolute path. If not | ||
## given, mujmap will choose an appropriate default for the configuration mode. | ||
## You probably don't want to set this. | ||
|
||
# state_dir = | ||
|
||
################################################################################ | ||
## Tag config | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
use crate::jmap; | ||
use crate::sync::NewEmail; | ||
use crate::Config; | ||
use const_format::formatcp; | ||
use lazy_static::lazy_static; | ||
use log::debug; | ||
|
@@ -14,9 +15,7 @@ use std::collections::HashMap; | |
use std::collections::HashSet; | ||
use std::fs; | ||
use std::io; | ||
use std::path::Path; | ||
use std::path::PathBuf; | ||
use std::path::StripPrefixError; | ||
|
||
const ID_PATTERN: &'static str = r"[-A-Za-z0-9_]+"; | ||
const MAIL_PATTERN: &'static str = formatcp!(r"^({})\.({})(?:$|:)", ID_PATTERN, ID_PATTERN); | ||
|
@@ -35,17 +34,6 @@ pub enum Error { | |
#[snafu(display("Could not canonicalize given path: {}", source))] | ||
Canonicalize { source: io::Error }, | ||
|
||
#[snafu(display( | ||
"Given maildir path `{}' is not a subdirectory of the notmuch root `{}'", | ||
mail_dir.to_string_lossy(), | ||
notmuch_root.to_string_lossy(), | ||
))] | ||
MailDirNotASubdirOfNotmuchRoot { | ||
mail_dir: PathBuf, | ||
notmuch_root: PathBuf, | ||
source: StripPrefixError, | ||
}, | ||
|
||
#[snafu(display("Could not open notmuch database: {}", source))] | ||
OpenDatabase { source: notmuch::Error }, | ||
|
||
|
@@ -89,9 +77,7 @@ pub struct Local { | |
|
||
impl Local { | ||
/// Open the local store. | ||
/// | ||
/// `mail_dir` *must* be a subdirectory of the notmuch path. | ||
pub fn open(mail_dir: impl AsRef<Path>, dry_run: bool) -> Result<Self> { | ||
pub fn open(config: &Config, dry_run: bool) -> Result<Self> { | ||
// Open the notmuch database with default config options. | ||
let db = Database::open_with_config::<PathBuf, PathBuf>( | ||
None, | ||
|
@@ -105,30 +91,40 @@ impl Local { | |
) | ||
.context(OpenDatabaseSnafu {})?; | ||
|
||
// Get the relative directory of the maildir to the database path. | ||
let canonical_db_path = db.path().canonicalize().context(CanonicalizeSnafu {})?; | ||
let canonical_mail_dir_path = mail_dir | ||
.as_ref() | ||
// Get notmuch's idea of the mail root. If, for whatever reason, we get nothing back for | ||
// that key (ancient version of notmuch?), use the database path. | ||
let mail_root = db | ||
.config(ConfigKey::MailRoot) | ||
.map_or(db.path().into(), |root| PathBuf::from(root)) | ||
.canonicalize() | ||
.context(CanonicalizeSnafu {})?; | ||
let relative_mail_dir = canonical_mail_dir_path | ||
.strip_prefix(&canonical_db_path) | ||
.context(MailDirNotASubdirOfNotmuchRootSnafu { | ||
mail_dir: &canonical_mail_dir_path, | ||
notmuch_root: &canonical_db_path, | ||
})?; | ||
|
||
// Build the query to search for all mail in our maildir. | ||
let all_mail_query = format!("path:\"{}/**\"", relative_mail_dir.to_str().unwrap()); | ||
// Figure out our maildir. Either the configured thing, or notmuch's mail root. Which in | ||
// the worst case will be notmuch's database dir, but that's probably not the worst choice. | ||
let mail_dir = match &config.mail_dir { | ||
Some(ref dir) => dir.clone(), | ||
_ => mail_root.clone(), | ||
} | ||
.canonicalize() | ||
.context(CanonicalizeSnafu {})?; | ||
debug!("mail dir: {}", mail_dir.to_string_lossy()); | ||
|
||
// Build the query to search for all mail in our maildir. If the maildir is under the | ||
// notmuch mail root, then search under the relative maildir path (allowing multiple | ||
// maildirs per notmuch dir). If not, assume this is the only maildir for the notmuch dir, | ||
// and use a global query. | ||
let all_mail_query = mail_dir | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If the user types an absolute path for the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure there's a lot we can do really; how do we know if its a error if the path is valid? The best I can think to do is for a configured There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok, I've added a commit that does just that if The more I think about the more I think this will be rare, because most of the time I expect that people would mess with their config until its set up how they like, and would then walk away, so its unlikely that they're going to get a path wrong in a system they've already set up. Its still good to have a safety for an obviously-wrong situation though. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hmm, this isn't quite what I meant. I think I understand why you changed it like this now, to specially handle the case for if https://doc.rust-lang.org/std/path/struct.PathBuf.html#method.strip_prefix But the consequences of removing the
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not really following your scenario, and I'm not sure if that's because its imprecisely described, or if its because we're not quite in agreement about what we're doing here and so talking past a little bit. I will try and write down what I think is happening here, and hopefully I will understand by the end. I'm seeing So in your scenario, if they've set If on the other hand they haven't set Going back to the code, what I think I'm doing is that if the maildir is under the notmuch root, then the query becomes Ahh, maybe that's it. Are you talking about the case where the user has explicitly set I wonder if that's the real problem here, that the path prefix is kind of baking too much info about the existing paths into the notmuch database, and if they change we don't know how to hook it up properly? If so I'm not sure we can fix this here. Some possible solutions (off the top of my head):
Something else might be to do a sanity check. If there's a bunch of stuff in the index, but none of the paths exist on disk, abort and warn the user. Hmm, and thinking more on that, maybe the answer is that if a message is not on disk, but is in notmuch, and is on the server, re-download it. I'm not really arguing against the need for (I wrote this over the course of a busy and distracted day so I feel like everything has fallen out of my head even more than the original plan fell out of my head in the last couple of months; I'm really struggling. I'll try to think about this more over the weekend). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I think maybe this is where the confusion stems. notmuch doesn't actually support this configuration; everything has to be a subdirectory of My example was mostly just me trying to address what I thought your misconception was rather than your actual example, rather than me trying to necessarily specifically prevent these kinds of database migration issues, but I do agree that a sanity check for a really large unexpected change like this would be a great to have feature.
That's okay, me too 🙂 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I agree that's where the confusion comes from; if using My current notmuch config with profiles, fyi:
And then I run mujmap with this PR as: I did consider that maybe the intent of notmuch's profile support was still that the database was under the maildir, and maybe I was indeed getting accidental behaviour. The config doc is not clear on this, though perhaps that's just it not answering a question that no one asked. However, the tests for split-path configs seems to imagine that the database path and mail dir can be in very different locations, as perhaps does the commit that introduced it, and a code read seems to support that too. I think I don't have any more to add. I (of course) think I'm right about what notmuch is supposed to be able to do here; if you're not persuaded then I'm not sure what we do next. |
||
.strip_prefix(&mail_root) | ||
.ok() | ||
.filter(|rel| rel.components().count() > 0) | ||
.map_or("path:**".to_string(), |rel| { | ||
format!("path:\"{}/**\"", rel.to_str().unwrap()) | ||
}); | ||
|
||
// Ensure the maildir contains the standard cur, new, and tmp dirs. | ||
let mail_cur_dir = canonical_mail_dir_path.join("cur"); | ||
let mail_cur_dir = mail_dir.join("cur"); | ||
if !dry_run { | ||
for path in &[ | ||
&mail_cur_dir, | ||
&canonical_mail_dir_path.join("new"), | ||
&canonical_mail_dir_path.join("tmp"), | ||
] { | ||
for path in &[&mail_cur_dir, &mail_dir.join("new"), &mail_dir.join("tmp")] { | ||
fs::create_dir_all(path).context(CreateMaildirDirSnafu { path })?; | ||
} | ||
} | ||
|
Uh oh!
There was an error while loading. Please reload this page.