0

This problem has plagued me for years. And, I need to be able to exclude certain folders in order to get the Eclipse indexer unstuck.

I have a project folder tree. It has a build directory named "output", which stores build content, auto-generated source code, executables, etc. There are hundreds of thousands of folders and files in that output directory alone.

We want to EXCLUDE almost all of that folder to keep it from bogging down Eclipse's indexer and refresher, and since most of it is not source code, but we must INCLUDE a few sub-folders from it so we can index their auto-generated source code and view C++ class and function definitions and variables and things from that auto-generated source code.

How can we best accomplish this?

Here is the project folder tree:

my_project
    source
        ...
    dir1
    dir2
    dir3
    output
        board1
            build
                foo-lib1
                foo-lib2
                foo-lib3
                ...
                foo-lib99
                bar-lib1
                bar-lib2
                bar-lib3
                foo2-lib1
                foo2-lib2
            another-dir
                subdir1
                subdir2
            file1
            file2
            file3
        board2
            ...
        board99
    ...
    dir99999

Assume I want to exclude the entire output dir EXCEPT FOR the files in the directories my_project/output/board1/build/foo-lib1 through my_project/output/board1/build/foo-lib99, which I want to include. I also may want to include a few other files or folders on a case-by-case basis in the future as I work on new libraries, and I want to include my_project/output/board1/file1 through file3.

Again, here are the 100+ files and folders I'd like to NOT exclude from the output dir:

my_project/output/board1/build/foo-lib1
my_project/output/board1/build/foo-lib2
my_project/output/board1/build/foo-lib3
...
my_project/output/board1/build/foo-lib99

my_project/output/board1/another-dir/subdir1
my_project/output/board1/another-dir/subdir2

my_project/output/board1/file1
my_project/output/board1/file2
my_project/output/board1/file3

What's the best way to exclude the entire output dir except those files? (I'd like to include those files).

Similar questions

This is related to, but not a duplicate of:

  1. Completely exclude certain directories from Eclipse CDT project
  2. How to filter resource folders in a certain subpath of the project only?

Neither of those are the same question, and neither of them had enough information to solve my problem.

Gabriel Staples
  • 36,492
  • 15
  • 194
  • 265
  • 1
    I could make a guess about the downvotes. For the question, it is best to ask a straightforward question (as if you had not already solved it). This is often quite hard to do, as once a problem has been resolved, people rarely still have access to the code/logs/etc that they had at the start of the problem. I have tried to amend the question to that format. – halfer Dec 11 '21 at 12:11
  • For the answer, I think this is rather downvote-worthy, unfortunately. It is highly overformatted, including whole sentences in title formatting. All-caps titling was OK in the day of newsgroups and bulletin boards, but not now we have (heading) formatting. Images can be embedded in lists (using a double indent) but currently they look rather squashed in. The answer length means that although editors _could_ fix it, it probably isn't worth the time it would require. – halfer Dec 11 '21 at 12:14
  • The answer also contains an appeal to readers to fill in something you don't know (I think that should be removed) and a keyword list, presumably for search engines (also should be removed). And the bit about Eclipse's Resource Filters is rather ranty (I don't think it serves much of a purpose, other than cathartic release). – halfer Dec 11 '21 at 12:16

1 Answers1

0

Summary:

  1. Follow section "1. THE STEPS" below.
  2. Then, implement the Resource Filters as described in section "3. RESOURCE FILTER SETTINGS YOU SHOULD SET IN ORDER TO ACHIEVE THE DESIRED RESULT AS SPECIFIED IN THE QUESTION".

Done. The question is answered.

Details:

Tested on Linux Ubuntu 18.04 in "Eclipse IDE for Embedded C/C++ Developers (includes Incubating components)":

  • Version: 2021-06 (4.20.0)
  • Build id: 20210612-2011

Note that the below instructions should also apply to and work in other Eclipse-based IDEs, such as STM32CubeIDE. Since the Arduino PRO IDE is apparently also Eclipse-based, it may work in that IDE as well.

Sometimes I hate Eclipse due to nuances (and bugs/undesired behavior) like what I'll demonstrate below. But, it's important to document and share this knowledge, as it took me several years to figure it all out, so here you go:

Detailed example:

This example shows how to:

  1. exclude an entire folder
  2. re-include it as an empty virtual folder
  3. add Linked Folders as necessary, and then
  4. add specialized rules to selectively include just what you want from them

Here are some examples to point out what DOES and does NOT work, and to show some techniques I use.

The approach: since we want to exclude nearly all of the output dir in the project, we will first exclude ALL of it. Then, we will re-create that folder as a virtual output folder which is empty. Then, we will add board1 as a "Linked Folder", which is basically an Eclipse-based symbolic link, or symlink. Finally, we will add "Resource Filters" as necessary to include just what we want from the my_project/output/board1 dir.

You might ask: "Why exclude the output dir entirely first? Why not just do necessary include-style Resource Filters for that directory?" Here are some of my answers:

  1. You canNOT just add the necessary include-style Resource Filters for contents within that output directory alone. It is not possible. Rather, adding even a single "Include only" Resource Filter automatically excludes everything NOT explicitly included. That's not good. That seems like it would get really complicated quickly, trying to explicitly include the hundreds of thousands of files and folders we need included just to exclude one single directory (output) except for a few things within it.
    1. Remember that to be included (assuming any "Include only" Resource Filter is in use) a file or folder MUST be included by an "Include only" resource filter and NOT excluded by any "Exclude all" resource filter.
  2. A somewhat weak and not-very-great answer is: "You could totally do that. I just want to exclude it as soon as possible is all because it's so stinking huge it bogs down my system just having Eclipse try to refresh it to add its contents. And, the indexer immediately gets bogged down on trying to index it. So, let's just exclude it immediately."
  3. Another answer is that I suspect few people know about Eclipse "Virtual Folders", and sometimes they are really useful, so it's worth introducing them.
  4. Referring back to reason 1 above: if you make a mistake with your resource filter and it tries to include hundreds of thousands of files again, you just froze your computer for 15 minutes while you either try to force-kill Eclipse or potentially even do a hard reboot to make Eclipse quit hogging up 100% of your resource. This won't happen as easily or as badly if you made the output dir a virtual folder instead, because at a bare minimum it at least keeps all of the immediate subdirs in the output dir which have NOT been added as Linked Folders from being refreshed and/or indexed.

1. THE STEPS:

  1. First, Turn OFF the indexer, so it doesn't bog down your system: Window --> Preferences --> C/C++ --> Indexer --> uncheck the "Enable indexer" box at the very top --> click "Apply and Close" at the bottom:
    1. enter image description here
  2. Exclude the entire output dir: right-click your my_project project name in the Project Explorer --> Properties --> Resource --> Resource Filters --> click "Add Filter" --> choose "Exclude all", "Folders", check the box for "All children (recursive)", File and Folder Attributes, Project Relative Path, matches, "output", case sensitive, as shown here:
    1. enter image description here
    2. Do NOT type output/ instead of output. With the trailing slash it will NOT work.
    3. Click "OK" and "Apply and Close".
  3. Add output back as an empty Virtual Folder: right-click project name --> New --> Folder --> click "Advanced >>" --> choose "Folder is not located in the file system (Virtual Folder)" --> type in "output" as the "Folder name" --> click "Finish". Mine shows it in the Project Explorer with a little tiny "V" next to it now to indicate it is a "Virtual Folder":
    enter image description here
  4. Add board1 as a "Linked Folder": right-click the newly-created output dir --> New --> Folder --> click "Advanced >>" --> choose "Link to alternate location (Linked Folder)" --> click "Browse" and navigate to the my_project/output/board1 dir and select it --> click the "Open" button in the top-right to choose it --> click the "Resource Filters..." button to add resource filters NOW (see Important Note just below, and details on adding resource filters in the section below) --> click "Finish" back in the main "New Folder" window when done.
    1. IMPORTANT NOTE: if you click "Finish" just above withOUT adding Resource Filters now, the entire board1 dir will get refreshed and added. If it contains too many files and folders for your system to handle, it may severely bog your system down and/or take a long time (many minutes, perhaps dozens of minutes) to complete. If it bogs your system down, you should have instead clicked the "Resource Filters..." button right in the main "New Folder" window and added the necessary Resource Filters NOW before you click the "Finish" button to finish adding this new Linked Folder!
    2. Note that once created, "Linked Folders" will have a little chain-link icon instead of the "V" icon on them.
  5. If you didn't already do so above, add "Resource Filters" as necessary to include just what we want from the my_project/output/board1 dir. See below for details.
  6. Once done adding Resource Filters, turn the indexer back ON.

2. Notes about properly adding necessary "Resource Filters":

Eclipse resource filters appear to be BADLY BROKEN. They only kinda-sorta work. Essentially, when trying to include or exclude a file or directory by path, including by "Project Relative Path" matches, it only works one single level deep. If you are trying to specify exact paths of files or folders to include or exclude, the work-around to keep all filters only one level deep is shown below:

EXAMPLE OF RESOURCE FILTER SETTINGS WORKAROUND:

>> ALL filters below have "Include only" and "File and Folder Attributes" 
selected. <<

Applies to (Files, Folders, or Fi_&_Fol (Files and folders))
  |      All children (recursive) checked?
  |       |  Filter Details (PRPm = Project Relative Path matches, 
  |       |  Name = Name matches)
  |       |   |   Case sensitive checked?
  |       |   |    | Regular expression checked?
  |       |   |    |  |  Filter text
  |       |   |    |  |  |
------------------------------------------------------------------------------

(THIS DOES **NOT** WORK DUE TO BUGS IN ECLIPSE)
Instead of applying this 1 resource filter to the `output` dir:

Fi_&_Fol  N  PRPm  Y  N  output/board1/build/foo*


(THIS **DOES** WORK because it keeps the filter levels limited to 
**1 level deep**)

...you must apply these 2 filters to separate dirs:

1. Apply this filter to the `output/board1` dir:
Folders   N  Name  Y  N  build

2. Then apply this filter to the `output/board1/build` dir once the filter 
above has included that dir:
Fi_&_Fol  N  Name  Y  N  foo*

3. RESOURCE FILTER SETTINGS YOU SHOULD SET IN ORDER TO ACHIEVE THE DESIRED RESULT AS SPECIFIED IN THE QUESTION

To set a Resource Filter, right-click the folder of interest in Eclipse (note: you may have to Left-Click it first to bring it back into the proper context before Right-Click works again) --> Properties --> Resource --> Resource Filters --> Add Filter --> choose the options desired as shown below --> click "OK" to add the filter --> click "Apply" or "Apply and Close" to apply the filter. If you messed up the filter and accidentally typed in a buggy version of it, or if it includes too many files, your computer will freeze at this time. :)

Here is what the Resource Filter editor window looks like. This is a screenshot of the first filter below, listed as Folders N Name Y N build:

enter image description here

>> ALL filters below have "Include only" and "File and Folder Attributes" 
selected. <<

Applies to (Files, Folders, or Fi_&_Fol (Files and folders))
  |      All children (recursive) checked?
  |       |  Filter Details (PRPm = Project Relative Path matches, 
  |       |  Name = Name matches)
  |       |   |   Case sensitive checked?
  |       |   |    | Regular expression checked?
  |       |   |    |  |  Filter text
  |       |   |    |  |  |
------------------------------------------------------------------------------
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
1. Apply these filters to the `output/board1` dir:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
A) Include the `build` dir, which is 1 level deeper than the `board1` dir:
Folders   N  Name  Y  N  build

B) Include the `another-dir` dir, which is 1 level deeper than the `board1` dir:
Folders   N  Name  Y  N  another-dir

C) Include all files within the `board1` dir, which includes `file1` through 
`file3`. NB: do NOT use "Folders" or "Files and Folders" here! You MUST set it
to only "Files":
Files     N  Name  N  N  *

D) Alternatively, you could do this instead of the filter just above: include
files matching the "file*" pattern:
Files     N  Name  Y  N  file*

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
2. Apply these filters to the `output/board1/build` dir:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
A) Include all `foo-*` dirs, which are 1 level deeper than the `build` dir:
Folders   N  Name  Y  N  foo-*

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
3. Apply these filters to the `output/board1/another-dir` dir:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
A) NA: do nothing if you want EVERYTHING (all files and folders) to be included 
in this dir. 

B) Alternatively, you could match the "subdir*" folder pattern only:
Folders   N  Name  Y  N  subdir*

4. Other filter examples and results, demonstrating the extreme bugginess of Eclipse's Resource Filters feature:

This is to show some filters which DO and do NOT work.

Notes:

  1. You can individually apply "folder-level" filters to any folder in the project. Just beware that when using the "Project Relative Path", it is relative to the root of the project, not to the folder path of the folder to which it is applied. So, use output/board1/build/foo-lib1, for instance, NOT build/foo-lib1 to include the foo-lib1 dir via a Resource Filter applied to the build dir.
    1. Also keep in mind the 1-level-deep bug as described above. This filter will only work when applied to the build dir. Applying it to any dir above that should work, but does not, since Eclipse's Resource Filters are very very very very very very very irritating and buggy.
  2. Many of the filers below which do NOT work actually cause Eclipse to freeze, and I have to force kill Eclipse. I do this on Linux by running ps aux | grep -i eclipse to see all Eclipse processes, including the Java one, then I kill them with kill <pid1> <pid2>, where <pid1> and 2 are what I noted in the output of the ps command.
  3. I also tried using a Virtual Folder I created named output_ instead of output, in case there were conflicts with the top-project-level resource filter which exludes output, but it made no difference for any of the broken filters below.

Note about using regex filters: see https://regex101.com/ for help building regular expressions properly. Example: this regular expression:

my_project\/output\/board1\/build\/foo.*

OR this equivalent NON-regular expression wildcard match:

my_project/output/board1/build/foo*

...will match these paths:

my_project/output/board1/build/foo-module1
my_project/output/board1/build/foo-module2
my_project/output/board1/build/foo-module3
...
my_project/output/board1/build/foo-module99

See that example yourself here: https://regex101.com/r/pRee5O/2

>> ALL filters below have "Include only" and "File and Folder Attributes" 
selected. <<

Applies to (Files, Folders, or Fi_&_Fol (Files and folders))
  |      All children (recursive) checked?
  |       |  Filter Details (PRPm = Project Relative Path matches, 
  |       |  Name = Name matches)
  |       |   |   Case sensitive checked?
  |       |   |    | Regular expression checked?
  |       |   |    |  |  Filter text
  |       |   |    |  |  |
------------------------------------------------------------------------------
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
Resource Filters applied to the `my_project/output` folder directly:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 

Does NOT work!
Fi_&_Fol  Y  PRPm  N  N  output/board1/build/foo-lib1

Does NOT work! It will show the "build" folder, looking like it works, but 
nothing is in it.
Fi_&_Fol  Y  PRPm  N  N  output/board1/build

Does NOT work! I don't know why. ALL dirs in `output/board1` still show up.
Fi_&_Fol  N  PRPm  N  N  output/board1/build

Does NOT work! I don't know why. ALL dirs in `output/board1` still show up.
Eclipse totally freezes when setting this filter.
Fi_&_Fol  N  PRPm  N  N  output/board1/build/foo*

Does NOT work! Nothing shows up inside the `output/board1` dir now.
Fi_&_Fol  Y  PRPm  N  N  output/board1/build/foo*

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
Resource Filters applied to the `my_project/output/board1` folder directly:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
Does NOT work! 
Fi_&_Fol  Y  PRPm  N  N  output/board1/build/foo-lib1

WORKS! The `output/board1/build` dir and all subdirs and files in that dir 
show up.
Folders   N  PRPm  N  N  output/board1/build

WORKS! Same as above.
Folders   Y  PRPm  N  N  output/board1/build

WORKS! The `output/board1/build` dir and all subdirs and files in that dir
show up.
Folders   N  Name  N  N  build

Does NOT work! Trailing slash (/) not allowed.
Folders   N  PRPm  N  N  output/board1/build/

Does NOT work! I don't know why.
Folders   N  PRPm  N  N  output/board1/build/foo-lib1

Does NOT work! I don't know why.
Folders   N  PRPm  N  Y  output\/board1\/build\/foo.*

Does NOT work! I don't know why.
Folders   Y  PRPm  N  Y  output\/board1\/build\/foo.*

Does NOT work! I don't know why.
Folders   N  PRPm  N  N  output/board1/build/foo*

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
Resource Filters applied to the `my_project/output/board1/build` folder 
directly:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
WORKS!
Folders   N  Name  Y  N  foo*

Conclusions:

Overall, I have found Eclipse's Resource Filters to be HORRIBLY DIFFICULT TO MAKE WORK, HORRIBLY implemented, VERY BUGGY, HORRIBLY SLOW AND LIKELY TO FREEZE YOUR SYSTEM WHEN CONFIGURING, and HORRIBLY DOCUMENTED, but a little better than nothing.

[UPDATE: THIS IS MY PREFERRED CHOICE] Going Further: write your own manual script to make symlinks:

An alternative to trying to get complicated with Eclipse's badly-broken Resource Filters might be to write your own script (in Python or Bash, for example) which acts on filters you create in a .gitignore-file-like fashion, which script then creates symlinks according to the filters in that file. You'd then simply use Eclipse to create the virtual output dir, followed by a Linked Folder to a custom dir you made with a bunch of symlinks in it to whatever resources you'd like to include. Done! This way, Eclipse just brings in your custom dir, and you manually manage the symlinks in that custom dir with your custom script and tools OUTSIDE of Eclipse, so that Eclipse's broken Resource Filters don't waste your life for days every time they freeze your system and screw up (because they won't be doing that anymore) since your script will manually manage the filtering instead).

Pros of the above-described filter script:

  • you get full control
  • Eclipse no longer freezes on your and wastes hours of your time while trying to configure filters

Cons:

  • you have to periodically re-run your script to update the symlinks to include any new, matching files and folders which may have been created since you last ran the script

Here is a starting script to do the above:

First, do this one time:

cd path/to/repo

# Make your own `output` dir, which will contain symlinks. 
mkdir output_gs
# Add this to your .gitignore file
echo "/output_gs/" >> .gitignore
# OR, to not affect that file for others, add it to your
# private ".git/info/exclude" file instead
echo "/output_gs/" >> .git/info/exclude

Then, create this script here: path/to/repo/output_gs/update_symlinks.sh. Run it whenever you want to update the symlinks in your custom output_gs dir.

update_symlinks.sh:

#!/usr/bin/env bash

# See: https://stackoverflow.com/a/60157372/4561887
FULL_PATH_TO_SCRIPT="$(realpath "$0")"
SCRIPT_DIRECTORY="$(dirname "$FULL_PATH_TO_SCRIPT")"
SCRIPT_FILENAME="$(basename "$FULL_PATH_TO_SCRIPT")"

# 1. symlink all files 1 level inside board dir. 
# This symlinks over these files:
#   my_project/output/board1/file1
#   my_project/output/board1/file2
#   my_project/output/board1/file3
dir_source="$SCRIPT_DIRECTORY/../output/board1"
dir_target="$SCRIPT_DIRECTORY/board1"
mkdir -p "$dir_target"
cd "$dir_target"
echo "Updating symlinks in: \"$(pwd)\""
find "$dir_source" -maxdepth 1 -type f | xargs -I{} -- ln -srf "{}" .

# 2. symlink all dirs of interest inside the board build dir

# 2.A: first, all of these files (I've added more than what were originally in
# the question, for demo purposes):
#   my_project/output/board1/build/foo-lib1
#   my_project/output/board1/build/foo-lib2
#   my_project/output/board1/build/foo-lib3
#   ...
#   my_project/output/board1/build/foo-lib99
#
#   my_project/output/board1/build/bar-lib1
#   my_project/output/board1/build/bar-lib2
#   
#   my_project/output/board1/build/foo2-lib1
#   my_project/output/board1/build/foo2-lib2
#
#   my_project/output/board1/build/bar2-lib1
#   my_project/output/board1/build/bar2-lib2
dir_source="$SCRIPT_DIRECTORY/../output/board1/build"
dir_target="$SCRIPT_DIRECTORY/board1/build"
mkdir -p "$dir_target"
cd "$dir_target"
echo "Updating symlinks in: \"$(pwd)\""
find "$dir_source" -maxdepth 1 \
| xargs -I{} -- basename "{}" \
| grep -E '(^foo-.*|^bar-.*|^foo2-.*|^bar2-.*)' \
| sort -u \
| xargs -I{} -- ln -srf "$dir_source/{}" .

# 2.B: all files and folders in "my_project/output/board1/another-dir"
# which begin with `subdir`:
dir_source="$SCRIPT_DIRECTORY/../output/board1/another-dir"
dir_target="$SCRIPT_DIRECTORY/board1/another-dir"
mkdir -p "$dir_target"
cd "$dir_target"
echo "Updating symlinks in: \"$(pwd)\""
find "$dir_source" -maxdepth 1 \
| xargs -I{} -- basename "{}" \
| grep -E '(^subdir.*)' \
| sort -u \
| xargs -I{} -- ln -srf "$dir_source/{}" .

Related:

  1. The links in the question
  2. My Eclipse setup doc: https://github.com/ElectricRCAircraftGuy/eRCaGuy_dotfiles/tree/master/eclipse
    1. Direct link to it on Google Drive

Keywords: Eclipse Resource Filters, eclipse resource filter "one level deep" bug; Eclipse include some files and folders, eclipse exclude all but a few files and folders

Gabriel Staples
  • 36,492
  • 15
  • 194
  • 265