projectorganizer: ignored directory patterns are not applied to project root paths

[stefanct]

I often work with big projects mounted via sshfs or nfs. If the projectogranizer plugin is loaded then geany reads in all directories of the project recursively which (potentially and practically) takes a long time via a network - so much time actually that I cannot use the plugin.
To mitigate the problem I tried using the "Ignored directory patterns" (ignored_dirs_patterns) feature by adding *sshfs to the list as I strictly name the mount points like that. Unfortunately this does not help at all.

After looking at the code I am pretty sure this is because get_file_list (which is called recursively) does apply all filters to the children of the given utf8_path only but not utf8_path itself. This makes somewhat sense due to the recursive nature of the function. However, prjorg_project_rescan_root does not apply the filters to the root dir either and thus the filters do not work for my use case. I can of course work around this by nesting the mount point within the project directory but that's not exactly ideal. Since prjorg_project_rescan_root sets up the ignored_dirs_list anyway it would be quite easy to add a check for that before initially calling get_file_list AFAICT. What do you think?

[techee]

I'm not sure if I understand the problem correctly. "root" for the plugin is

• either the project directory itself, e.g. /home/techet/projects/geany
• or added external directories, e.g. /home/techet/projects/geany-plugins and possibly some more

Since /home/techet/projects/geany is the directory containing the project, it doesn't make sense to ignore it - you wouldn't see any files in the sidebar if you did it. Similarly, it doesn't make sense to do this for external directories because you can simply remove those you don't want to see.

So how is your project organized - the mounted directories are mounted somewhere inside the project directory itself?

Couldn't the problem rather be in the "Index all project files" settings under Project->Properties->Project Organizer which, when enabled, parses all files (and possibly loads their full contents over network)?

[elextr]

To be clear, the ignore patterns are only applied within the project directory tree, not to the path to the root, so as @techee said, unless the mount points are inside the tree they will not be filtered. Any filter applied to the path to root could only ignore the root and therefore the whole project, which does not make sense, the plugin would not do anything.

The point of the plugin is to scan the filesystem and optionally scan all files for symbols, so if the filesystem where the project is stored is slow the plugin will be slow. Possibly you can improve the SSHFS/NFS caching, but startup would probably still be slow as everything in the cache is validated.

[stefanct]

You both seem to ignore the fact that one can have projects where the plugin makes sense and others where it doesn't (work).

@techee In the instance I am describing here, the whole project directory is remote. The problem is not the indexing itself but the directory traversal which happens no matter what the index setting is AFAICT - and that it is done synchronously (apparently even in the UI thread). I don't use the plugin's sidebar BTW - I haven't even noticed that there is one till now. I am using the File Browser plugin for my file navigation needs - well that, search in files, and code navigation's and recently projectorganizer's lookup system but I'll take a look in the future.

@elextr Since there is no way to disable the plugin per project, ignoring the whole project tree is exactly what I am trying to achieve in the respective cases! Also, since neither documentation nor GUI indicates otherwise, I would argue that it is not at all obvious that the filter does not apply to the project's root directory. One could even argue that the filter should be applied to all the path components leading to the project's root IMHO. This would make effectively disabling the plugin based on the project's location even more generic (but since there is no customization for the plugin's default settings applying it to the rest of the path doesn't make too much sense in this regard).

I'd also be fine with an option to disable to plugin/directory traversal on a per project basis (although I actually think projectorganizer does lookups on just the open tabs even without the index from what I've seen in the code!). I am currently using the workaround of having the mountpoint as a subdirectory successfully and that's good enough for now.

[elextr]

You both seem to ignore the fact that one can have projects where the plugin makes sense and others where it doesn't (work).

Plugins can be disabled in the plugin manager.

The problem is not the indexing itself but the directory traversal which happens no matter what the index setting is AFAICT

What evidence do you have for that? Directory reading is likely to be much cheaper than reading all the files in that directory.

And AFAICT traversal does not enter a directory which matches the directory ignore patterns. So you can likely stop the tree traversal by setting the directory and file ignore patterns to * and only the root directory will be read. That is as close as there is to disabling the plugin except for actually disabling it in the plugin manager AFAIK.

and that it is done synchronously (apparently even in the UI thread).

Correct, Geany is non re-entrant single threaded code because the UI framework, GTK, is single threaded and most of Geany is interacting with the UI.

I would argue that it is not at all obvious that the filter does not apply to the project's root directory.

Documentation contributions welcome.

I'd also be fine with an option to disable to plugin/directory traversal on a per project basis

So thats a UI and storing and restoring the option and making functionality dependent on it and "somebody"s gotta do it. Each of the plugins in this collection are mostly the contributed work of one individual (with (un?)helpful comments from the rest of us ;-). Most plugins are therefore focused on that individuals use-case, in this case large projects on fast machines with local fast storage.

the workaround of having the mountpoint as a subdirectory successfully and that's good enough for now.

Great.

[techee]

I don't use the plugin's sidebar BTW - I haven't even noticed that there is one till now.

Just curious - what are you using the plugin for then?

I'd also be fine with an option to disable to plugin/directory traversal on a per project basis

I can do that - IMO that makes more sense than ignoring the whole project based on the ignored directory patterns. To be clear, this option would disable the whole plugin completely for certain projects because when ProjectOrganizer cannot see files in the project directory, it cannot do anything useful.

although I actually think projectorganizer does lookups on just the open tabs even without the index from what I've seen in the code

I'm not aware of that, could you point me to the code? In any case, the documents in open tabs are opened and parsed by Geany already so the files have to be transferred over network anyway.

[elextr]

I can do that - IMO that makes more sense than ignoring the whole project based on the ignored directory patterns.

But using patterns allows the plugin to autoselect which projects to disable since it can distinguish between project1_dir/remote (which is the slow remote mount) and project2_dir/local (which is fast local storage).

Also it is available now and in release 2.0.

To be clear, this option would disable the whole plugin completely

I think thats the idea, for plugins where the project directory is on a slow mount disable the whole PO functionality but have it work for projects that are local. As the OP said "You both seem to ignore the fact that one can have projects where the plugin makes sense and others where it doesn't"

[techee]

But using patterns allows the plugin to autoselect which projects to disable since it can distinguish between project1_dir/remote (which is the slow remote mount) and project2_dir/local (which is fast local storage).

But I think the problem (if I understand it correctly) is, that when you have a project foo which contains only remote directories and files (imagine something like e.g. Geany's root project directories and files), you don't want even to list the files inside foo and ignore it completely before anything gets loaded. But one can overcome it by creating the Geany project one directory above so foo becomes a single directory of the project and can get ignored as a whole.

[elextr]

creating the Geany project one directory above so foo becomes a single directory of the project and can get ignored as a whole.

Which is why I showed it as project_dir/remote where remote is the mount and the ignored pattern.