Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rustler recompiles although nothing changed #651

Open
VictorGaiva opened this issue Sep 4, 2024 · 8 comments
Open

Rustler recompiles although nothing changed #651

VictorGaiva opened this issue Sep 4, 2024 · 8 comments

Comments

@VictorGaiva
Copy link

I have an Umbrella App setup where rustler is being used in a ´Core.QrCode` module.

This module has no external compile time dependency except Rustler

defmodule Core.QrCode do
  use Rustler,
    otp_app: :core,
    crate: :qr_code

  defmodule QrCodeOptions do
    @type t :: %__MODULE__{
            content: String.t(),
            # ...args
          }

    defstruct [
      :content,
      # ...args
    ]

    def new(content, rest \\ %{}) do
      struct(__MODULE__, Map.put(rest, :content, content))
    end
  end

  def generate_svg(_arg1), do: :erlang.nif_error(:nif_not_loaded)
end

So when we change almost anything, modules that are not related Core.QrCode, it recompiles causing many other parts of the application to recompile. Since we are heavily using it in another umbrella app, it trigger the recompilation of 80+ files, even if nothing changed.

Is there any way to prevent it from being recompiled when nothing changed?

@VictorGaiva VictorGaiva changed the title Rustler recompiles althogh nothing changed Rustler recompiles although nothing changed Sep 4, 2024
@filmor
Copy link
Member

filmor commented Sep 4, 2024

  • Which version of rustler_mix are you using?
  • What is the output of MIX_DEBUG=1 mix compile when this happens?

@VictorGaiva
Copy link
Author

1 - {:hex, :rustler, "0.34.0", "e9a73ee419fc296a10e49b415a2eb87a88c9217aa0275ec9f383d37eed290c1c"
2 -

Running MIX_DEBUG=1 mix compile inside 'apps/gateway' umbrella app, to prevent 'reports' from recompiling:

-> Running mix loadconfig (inside Gateway.Mixfile)
<- Ran mix loadconfig in 16ms
-> Running mix compile (inside Gateway.Mixfile)
-> Running mix loadpaths --no-deps-loading (inside Gateway.Mixfile)
-> Running mix deps.loadpaths --no-deps-loading (inside Gateway.Mixfile)
-> Running mix archive.check --no-deps-loading (inside Gateway.Mixfile)
<- Ran mix archive.check in 0ms
-> Running mix deps.precompile (inside Gateway.Mixfile)
<- Ran mix deps.precompile in 0ms
==> caching
-> Running mix compile --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Caching.Mixfile)
-> Running mix loadpaths --no-deps-loading --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Caching.Mixfile)
<- Ran mix loadpaths in 0ms
-> Running mix compile.all --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Caching.Mixfile)
-> Running mix compile.yecc --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Caching.Mixfile)
<- Ran mix compile.yecc in 2ms
-> Running mix compile.leex --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Caching.Mixfile)
<- Ran mix compile.leex in 0ms
-> Running mix compile.erlang --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Caching.Mixfile)
<- Ran mix compile.erlang in 0ms
-> Running mix compile.elixir --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Caching.Mixfile)
<- Ran mix compile.elixir in 24ms
-> Running mix compile.app --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Caching.Mixfile)
<- Ran mix compile.app in 0ms
<- Ran mix compile.all in 40ms
<- Ran mix compile in 45ms
==> tasks
-> Running mix compile --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Tasks.Mixfile)
-> Running mix loadpaths --no-deps-loading --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Tasks.Mixfile)
<- Ran mix loadpaths in 0ms
-> Running mix compile.all --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Tasks.Mixfile)
-> Running mix compile.yecc --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Tasks.Mixfile)
<- Ran mix compile.yecc in 0ms
-> Running mix compile.leex --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Tasks.Mixfile)
<- Ran mix compile.leex in 0ms
-> Running mix compile.erlang --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Tasks.Mixfile)
<- Ran mix compile.erlang in 0ms
-> Running mix compile.elixir --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Tasks.Mixfile)
<- Ran mix compile.elixir in 1ms
-> Running mix compile.app --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Tasks.Mixfile)
<- Ran mix compile.app in 0ms
<- Ran mix compile.all in 5ms
<- Ran mix compile in 5ms
==> monitoring
-> Running mix compile --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Monitoring.Mixfile)
-> Running mix loadpaths --no-deps-loading --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Monitoring.Mixfile)
<- Ran mix loadpaths in 0ms
-> Running mix compile.all --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Monitoring.Mixfile)
-> Running mix compile.yecc --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Monitoring.Mixfile)
<- Ran mix compile.yecc in 0ms
-> Running mix compile.leex --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Monitoring.Mixfile)
<- Ran mix compile.leex in 0ms
-> Running mix compile.erlang --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Monitoring.Mixfile)
<- Ran mix compile.erlang in 0ms
-> Running mix compile.elixir --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Monitoring.Mixfile)
<- Ran mix compile.elixir in 1ms
-> Running mix compile.app --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Monitoring.Mixfile)
<- Ran mix compile.app in 0ms
<- Ran mix compile.all in 4ms
<- Ran mix compile in 4ms
==> analytics
-> Running mix compile --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Analytics.Mixfile)
-> Running mix loadpaths --no-deps-loading --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Analytics.Mixfile)
<- Ran mix loadpaths in 0ms
-> Running mix compile.all --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Analytics.Mixfile)
-> Running mix compile.yecc --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Analytics.Mixfile)
<- Ran mix compile.yecc in 0ms
-> Running mix compile.leex --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Analytics.Mixfile)
<- Ran mix compile.leex in 0ms
-> Running mix compile.erlang --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Analytics.Mixfile)
<- Ran mix compile.erlang in 0ms
-> Running mix compile.elixir --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Analytics.Mixfile)
<- Ran mix compile.elixir in 38ms
-> Running mix compile.app --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Analytics.Mixfile)
<- Ran mix compile.app in 0ms
<- Ran mix compile.all in 42ms
<- Ran mix compile in 42ms
==> core
-> Running mix compile --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Core.Mixfile)
-> Running mix loadpaths --no-deps-loading --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Core.Mixfile)
<- Ran mix loadpaths in 0ms
-> Running mix compile.all --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Core.Mixfile)
-> Running mix compile.yecc --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Core.Mixfile)
<- Ran mix compile.yecc in 0ms
-> Running mix compile.leex --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Core.Mixfile)
<- Ran mix compile.leex in 0ms
-> Running mix compile.erlang --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Core.Mixfile)
<- Ran mix compile.erlang in 0ms
-> Running mix compile.elixir --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Core.Mixfile)
Compiling 1 file (.ex)
Compiling crate qr_code in release mode (native/qr_code)
    Finished `release` profile [optimized] target(s) in 0.01s
<- Ran mix compile.elixir in 535ms
-> Running mix compile.app --from-mix-deps-compile --no-warnings-as-errors --no-code-path-pruning (inside Core.Mixfile)
<- Ran mix compile.app in 12ms
<- Ran mix compile.all in 561ms
<- Ran mix compile in 561ms
==> gateway
<- Ran mix deps.loadpaths in 1053ms
<- Ran mix loadpaths in 1055ms
-> Running mix compile.all (inside Gateway.Mixfile)
-> Running mix compile.yecc (inside Gateway.Mixfile)
<- Ran mix compile.yecc in 0ms
-> Running mix compile.leex (inside Gateway.Mixfile)
<- Ran mix compile.leex in 0ms
-> Running mix compile.erlang (inside Gateway.Mixfile)
<- Ran mix compile.erlang in 0ms
-> Running mix compile.elixir (inside Gateway.Mixfile)
<- Ran mix compile.elixir in 81ms
-> Running mix compile.app (inside Gateway.Mixfile)
<- Ran mix compile.app in 5ms
-> Running mix compile.surface (inside Gateway.Mixfile)
<- Ran mix compile.surface in 616ms
<- Ran mix compile.all in 708ms
-> Running mix compile.protocols (inside Gateway.Mixfile)
<- Ran mix compile.protocols in 16ms
<- Ran mix compile in 1813ms

@filmor
Copy link
Member

filmor commented Sep 5, 2024

Could you test it with rustler_mix 0.33? The last change that touched the recompilation was 7fef3c5.

@VictorGaiva
Copy link
Author

I'll be running some tests after downgrading. Will be posting again when possible, thanks for the attention

@VictorGaiva
Copy link
Author

VictorGaiva commented Sep 9, 2024

So far it seems to have worked. I've spent this morning changing code somewhere that would usually cause this issue. I'll be downgrading the version in my main branch, and keep an eye if the issue arises again.

Also an important note is that this issue happened to me and another collegue, while others were not facing it. In my case it happened on two different computers.

I can provide more information about each enviroment if necessary.

@filmor
Copy link
Member

filmor commented Oct 1, 2024

Hmm, I may have been using @external_resource wrong when trying to fix #616. @ggwpez, I think I have to look into this again. I have to check when exactly the file contents are checked.

@VictorGaiva Can you provide a minimal example that reproduces the issue on your machine?

@ggwpez
Copy link

ggwpez commented Oct 1, 2024

I also experienced the frequent recompilation issue, but did not though much of it.
I am using MacOs with a nix flake and a Rust workspace but no mix umbrella project. Maybe that overlaps with @VictorGaiva's setup?

Every now and then when i run mix test it would try to recompile the Rust crates, although nothing changed in the binding module or Rust code.

@VictorGaiva
Copy link
Author

VictorGaiva commented Oct 1, 2024

I'll be creating a minimal repo in a few days, and posting it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants