${workspaceFolder}/** is auto-added when default.includePath is set and no c_cpp_properties.json exists

[maxim-babenko]

Type: LanguageService

Describe the bug

• OS and Version:
  MacOS 10.15.4

• VS Code Version:
  1.45.1

• C/C++ Extension Version:
  0.28.2

• Other extensions you installed (and if the issue persists after disabling them):
  Remote -- SSH

• Does this issue involve using SSH remote to run the extension on a remote machine?:
  Yes

• A clear and concise description of what the bug is, including information about the workspace (i.e. is the workspace a single project or multiple projects, size of the project, etc).

After VSCode is being used a while (say, couple of hours) Goto Symbol in Editor could take tens of seconds to show the list of symbols in the current file. Closing VSCode and killing all vscode-related processes on the remote machine, and then restarting everything back solves the issue... for some time.

Is there anything I could do to help diagnosing this? E.g. should I collect the logs when this feature becomes laggy?

Thanks in advance.

Steps to reproduce

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior

Logs

Insert logs here.

Screenshots

Additional context

[sean-mcmanus]

You could set the C_Cpp.loggingLevel to "Debug", but my guess is that the logigng wouldn't show anything useful if the process is internal to the symbol processing thread.

Could you attach a debugger to cpptools when it's taking a long time and then break and look for a call stack that appears to have a lot of work/frames (i.e. not one of the many shallow thread call stacks that aren't really doing any work)?

[maxim-babenko]

Thanks for the suggestion, I'll try to do that (and perhaps try to collect some perf tops).

[maxim-babenko]

OK, here's what I've got after spending some time with top, perf and strace.

To give some initial context: our company stores most of its codebase in a huge monorepo. Its working copies are accessible via a fuse-powered filesystem.

It seems that my problems (or at least some of them) are not CPU-related. When goto symbol becomes laggy, I see (via strace) a huge number of FS-related syscalls, in particular stat, fstat, open, fcntl, read etc. Being backed by fuse, this is not likely to be fast.

Are there any caches in VS code that could help here? Currently it seems that each time I strike "goto symbol", VS code traverses the whole source tree starting from the current file.

Another strange thing I see is that "node" with "--type=watcherService" arguments keeps calling "inotify_add_watch" for files that are in explicitly blacklisted via files.watcherExclude directories. There are literally millions of them and this just can't scale properly.

[maxim-babenko]

Also this watcher node process constantly gets kills due to OOM:
https://gist.github.com/maxim-babenko/9b3e608ba7ff96b99aaf5b897c5f2c33

Could it be possible that without watcher being available we fallback to direct FS scan?

[maxim-babenko]

Changing watcherExclude to absolute paths seems to have helped me.
This oddity was also mentioned here:
microsoft/vscode#29594

[sean-mcmanus]

Go to symbol is supposed to use our database. Are you able to tell if the FS-related calls are to our database or other files? If it's to other files, then it seems likely the FS-related calls could be due to existing operations unrelated to the go to symbol that are just blocking the go to symbol, i.e. due to active parsing that is occurring. Our logging should probably have info on what existing operations are occurring.

It's possible the database is getting too large, although that implies that it's starting off smaller and growing (it should stay about the same size after the initial workspace parsing, unless something causes it to reset).

[maxim-babenko]

I have upgraded to 0.29.0 but still suffer from this problem.

To me it seems related to the fact that cpptools is traversing files it should definitely not be touching at all.

Here's my code-workspace file:
https://gist.github.com/maxim-babenko/28326adad81b70b1a199b49f6a438b59

I'm seeing tons of newfstatat calls to files outside of the supposed indexing range.

Also here are some results of ls /proc//fd I've obtained while cpptools was rebuilding its indexes.

$ ls -lht /proc/13378/fd
total 0
lrwx------ 1 babenko 64 Jul 29 13:18 0 -> socket:[814952859]
lrwx------ 1 babenko 64 Jul 29 13:18 1 -> socket:[814952861]
lr-x------ 1 babenko 64 Jul 29 13:18 10 -> /yt/nvme1/babenko/arc/store/.arc/objects/
lr-x------ 1 babenko 64 Jul 29 13:18 11 -> /yt/nvme1/babenko/arc/store/.arc/objects/37/

$ ls -lht /proc/13378/fd
total 0
lr-x------ 1 babenko 64 Jul 29 13:19 12 -> /yt/nvme1/babenko/arc/arcadia/alice/paskills/dialogovo/src/
lr-x------ 1 babenko 64 Jul 29 13:19 13 -> /yt/nvme1/babenko/arc/arcadia/alice/paskills/dialogovo/src/main/
lr-x------ 1 babenko 64 Jul 29 13:19 14 -> /yt/nvme1/babenko/arc/arcadia/alice/paskills/dialogovo/src/main/java/
lr-x------ 1 babenko 64 Jul 29 13:19 15 -> /yt/nvme1/babenko/arc/arcadia/alice/paskills/dialogovo/src/main/java/ru/
lr-x------ 1 babenko 64 Jul 29 13:19 16 -> /yt/nvme1/babenko/arc/arcadia/alice/paskills/dialogovo/src/main/java/ru/yandex/
lr-x------ 1 babenko 64 Jul 29 13:19 17 -> /yt/nvme1/babenko/arc/arcadia/alice/paskills/dialogovo/src/main/java/ru/yandex/alice/
lr-x------ 1 babenko 64 Jul 29 13:19 18 -> /yt/nvme1/babenko/arc/arcadia/alice/paskills/dialogovo/src/main/java/ru/yandex/alice/paskill/
lr-x------ 1 babenko 64 Jul 29 13:19 19 -> /yt/nvme1/babenko/arc/arcadia/alice/paskills/dialogovo/src/main/java/ru/yandex/alice/paskill/dialogovo/
lr-x------ 1 babenko 64 Jul 29 13:19 20 -> /yt/nvme1/babenko/arc/arcadia/alice/paskills/dialogovo/src/main/java/ru/yandex/alice/paskill/dialogovo/domain/

I see two big issues here:

1. /yt/nvme1/babenko/arc/store/.arc/objects/ -- why cpptools is ever reaching this directory? The workspace folder is /yt/nvme1/babenko/arc/arcadia; cpptools should never be leaving it
2. /yt/nvme1/babenko/arc/arcadia/alice is explicitly blacklisted in files.exclude but cpptool is traversing it

[bobbrow]

1. Do you have any symlinks that could be sending our indexer here?
2. We may traverse some paths but we won't parse the actual files in those paths if they are excluded.

If you enable "C_Cpp.loggingLevel": "Debug", our extension will print out the files that it is actually parsing in the Output Window (you have to select the "C/C++" log channel from the drop down box to see it).

[maxim-babenko]

1. Not likely.
2. Our monorepo is pretty big, some millions of files in total. Like I mentioned before, we use FUSE for accessing it. So even traversing the whole source tree (via FUSE, again) could take quite a while; and it is not needed, in fact I expect vscode not to touch the directories listed in the exclusion list.

I have "C_Cpp.loggingLevel": "Debug" in my settings.json

[maxim-babenko]

A complete log is here:
https://gist.github.com/maxim-babenko/13b66261e7e1309364432f16df3e9e1f

[maxim-babenko]

OK, 1) above is more interesting; indeed there's a link
$ stat /home/babenko/arc/arcadia/.arc
File: '/home/babenko/arc/arcadia/.arc' -> '/yt/nvme1/babenko/arc/store/.arc'

But .arc is in files.exclude, so why does vscode follow it?