-
Notifications
You must be signed in to change notification settings - Fork 82
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cargo watch
triggering on file that hasn't changed (only read)
#193
Comments
I can't really tell. The macos infrastructure for for changes used in cargo watch so far (it's due to change next version) is a black box. However, from knowledge of how other platforms work, for example inotify on Linux, it's possible that a mere file read access is triggering the change. |
Is the lockfile changing? |
Thank you for replying :-) The workspace |
No, I mean in the output you gave, there's |
In those tests Here's another output with the same thing happening (repo not yet open sourced).
In this case it's saying:
We're not modifying the |
That will probably need a bit of tweaking, but could you use the latest watchexec and see if it's an issue?
Also any chance to run on e.g. Linux? Want to clarify if it's an issue with FSEvents (macos, cargo watch <=8 only) or with cargo watch itself. |
Ah so it doesn't reproduce if I use watchexec v1.18.5 (using the pre-built binaries via Thank you for the suggestion to try that :-) |
Right, so it's an FSEvents thing. Unfortunately I can't really mitigate that. However, it will be fixed next release, when upstream watchexec is used here: the backend changes away from FSEvent to the more precise/appropriate kqueue. |
That makes sense - thank you for your help! We will use watchexec directly in the meantime :-) |
Okay, so it's not actually an FSEvent thing, because Watchexec was using FSEvent all along too. It might be a Notify 4 vs 5 thing, or it might be something else that Watchexec 1.18 fixes. Hopefully it will still be fixed when I merge upstream Watchexec into cargo watch! |
Fingers crossed that will fix it! :-) |
Hullo, still running into this issue at the moment. I get the same Happy to help with a fix here or deeper in, though I'm unclear given the above: is the |
@tekacs Does this happen with watchexec? |
@passcod I just did a little investigation between I've got a static site generator. It has an example application and basic web sources. My hope was to set up a watcher to respond to changes in the source dir. Critically, Seems to me like there is probably an issue with OSX triggering these tools on file reads. My My In both cases I got an infinite loop. I was able to shut down the loop by ignoring the Thanks! |
@oclyke Sorry for the late answer, but if you could run with |
@passcod No worries! Okay, here's what I am seeing. Upon a first-run I am not getting a loop. In this case the
Now the command is waiting to observe changes in the filesystem (within
Relevant code which seemingly triggers the watcher is like this:
There's not really much here. Maybe I can make a gist about it - one sec. |
@passcod Here's a gist which reproduces the problem on my M1 Mac. (macOS Sonoma 14.2.1) https://gist.github.com/oclyke/bb9e59af565cafb06651330628c04c47 |
Right, I see what you mean. I think that's an inherent limitation of fsevents (something like "a modify event happened that concerned this file, but i'm not telling you if it was the copied or the copy"); you may have better luck with polling. Out of curiosity, what events do you get with:
I haven't been able to replicate using the ancient macbook air I have myself. |
Hey, I'm sorry I missed this earlier! I've since switched to I've split my build process from my run process and I'm doing this (simplified) now: backend-build: watchexec --on-busy-update=queue --print-events -w common -w backend -- cargo build -p backend
backend: watchexec --on-busy-update=restart --print-events -w ./target/debug/backend --no-vcs-ignore -- ./target/debug/backend With this setup I don't see any I'm happy to test things to evaluate if there's something I can do to help this effort. |
Thanks for all the attention you're giving this one @passcod! Okay, I dove into this a little bit with a bunch of scattered testing. I'll do a TLDR here and dump more notes below. TLDR Detail On a hunch I changed the method of copying files to use the io copy. I thought this might get rid of the // std::fs::copy(source, path)?;
let mut source = std::fs::File::open(source)?;
let mut dest = std::fs::File::create(path)?;
std::io::copy(&mut source, &mut dest)?; With this modification the generator is able to be driven by changes in the source directory as expected. oclyke@Owens-MacBook-Pro blanket-rs % watchexec --print-events -w examples/basic/site-content -r cargo run --example basic
[EVENT 0] Event
[Running: cargo run --example basic]
Finished dev [unoptimized + debuginfo] target(s) in 0.02s
Running `target/debug/examples/basic`
program start
removing output directory... done.
Root::generate
program end
[Command was successful]
[EVENT 0] Event source=Filesystem kind=Modify(Metadata(Any)) path=/Users/oclyke/Documents/oclyke/blanket-rs/examples/basic/site-content/index.html filetype=file
[EVENT 1] Event source=Filesystem kind=Modify(Data(Content)) path=/Users/oclyke/Documents/oclyke/blanket-rs/examples/basic/site-content/index.html filetype=file
[Running: cargo run --example basic]
Finished dev [unoptimized + debuginfo] target(s) in 0.00s
Running `target/debug/examples/basic`
program start
removing output directory... done.
Root::generate
program end
[Command was successful] Using a separate process I recorded the events in JSON, as @passcod suggested. (Formatted for easy reading) [p.s. I realized that events in oclyke@Owens-MacBook-Pro blanket-rs % watchexec --print-events --only-emit-events --emit-events-to=json-stdio -w examples/basic/site-content -w examples/basic/site-out
{
"tags": [
{
"kind": "source",
"source": "filesystem"
},
{
"kind": "fs",
"simple": "modify",
"full": "Modify(Metadata(Any))"
},
{
"kind": "path",
"absolute": "/Users/oclyke/Documents/oclyke/blanket-rs/examples/basic/site-content/index.html",
"filetype": "file"
}
]
}
{
"tags": [
{
"kind": "source",
"source": "filesystem"
},
{
"kind": "fs",
"simple": "modify",
"full": "Modify(Data(Content))"
},
{
"kind": "path",
"absolute": "/Users/oclyke/Documents/oclyke/blanket-rs/examples/basic/site-content/index.html",
"filetype": "file"
}
]
} As you can see the same events appear to be happening and I didn't really have the time to dive deep into how these system calls interact with FSEvents. It seems really odd to me that the received events (Content and Metadata modifications) did not change between the two cases. Eventually if I get some free time I think it would be neat to understand what is going on here. Thanks again for all the help everyone! |
Hi! Thank you for a great tool :-)
I'm currently experiencing
cargo watch -x test
looping indefinitely, with--why
reporting that the cause is a file that hasn't been modified (the file is read duringcargo test
and copied to a temp dir, but the original isn't modified itself).I was wondering if you might have some insight?
Steps to reproduce:
git clone https://github.com/Malax/libcnb.rs
(currently atfd1686f8d0fe07aca5e0b9592fbdee1919591fdb
)cd libcnb.rs/examples/example-02-ruby-sample/
cargo watch -x test --why
Expected:
Test command is run only once, and waits for next modification to the project source before being re-run.
Actual:
Test command is run repeatedly in a loop.
Output says:
Stating the files before/after shows nothing has changed:
During the test setup, the files in question are copied to a temporary directory here using
fs_extra::dir::copy()
:https://github.com/Malax/libcnb.rs/blob/fd1686f8d0fe07aca5e0b9592fbdee1919591fdb/libcnb-test/src/app.rs#L5-L21
...but the target directory is in TMP and so not watched, and the source file was never modified.
Additional information
Cargo watch was installed using
cargo install cargo-watch
(I reinstalled just now using--force
to ensure it was recently compiled, given not using--locked
).Versions:
(though this also repros on Rust stable)
This is on macOS 12.2.
The text was updated successfully, but these errors were encountered: