I get no static analysis

[Onturenio]

Hi, thanks for this great plugin which I have been using for some time now. Yesterday I changed some things in my setup and tried to reinstall all my vim plugins. Unfortunatelly I cannot make coc-pylsp work right regarding the integration with coc-ruf.

I have followed these steps:

• My .vimrc has this line alone Plug 'neoclide/coc.nvim', {'branch': 'release'}.
• my .vim/coc-settings.json has this content:

  {
  "pylsp.plugins.pycodestyle.enabled": false,
  "pylsp.plugins.flake8.enabled": false,
  "pylsp.plugins.flake8.ignore": ["E265", "E266"],
  "pylsp.plugins.pydocstyle.enabled": false,
  "pylsp.plugins.mccabe.enabled": false,
  "pylsp.plugins.pyflakes.enabled": false,
  "pylsp.plugins.yapf.enabled": false,
  "pylsp.plugins.ruff.enabled": true,
  "pylsp.plugins.ruff.extendSelect": ["E", "D", "C90"],
  "pylsp.plugins.ruff.lineLength": 79,
  "pylsp.builtin.installExtrasArgs": [
  "flake8",
  "yapf",
  "mccabe",
  "pycodestyle",
  "pydocstyle",
  "pyflakes",
  "pylint",
  "rope"                                                                                                                                                               
  ],
  "pylsp.builtin.enableInstallPythonLspRuff": true,
  "pylsp.builtin.enableInstallPylsIsort": true, 
  "diagnostic.messageTarget": "float", 
  "diagnostic.checkCurrentLine": true,
  "suggest.enablePreselect": false,
  "suggest.noselect": true
  }

• I install coc-pylsp from within vim with :CocInstall u/yaegassy/coc-pylsp.

• I install all extras required by the latter by running also :CocCommand pylsp.builtin.install.

• My ~/.config/coc/extensions/@yaegassy/coc-pylsp-data/pylsp/venv/bin directory has python, ruff, pylint, and the rest of necessary binaries.

Still, when I open a python file, I get no static analysis whatsoever, nor any warning pointing to any error I can try to fix. Any idea how to figure out what link is missing in the chain?

Thanks a lot!

[yaegassy]

The following steps can be taken to check if something is wrong with the language server.

1. :CocCommand workspace.showOutput
2. Choose or enter pylsp

[Onturenio]

Thanks this indded points to some problems:

2023-11-23 15:00:36,543 CET - ERROR - pylsp_ruff.plugin - Can't execute ruff with given executable 'ruff'.
2023-11-23 15:00:36,543 CET - WARNING - pylsp.config.config - Failed to load hook pylsp_lint: local variable 'p' referenced before assignment
Traceback (most recent call last):
  File "/home/navarro/.config/coc/extensions/@yaegassy/coc-pylsp-data/pylsp/venv/lib/python3.10/site-packages/pylsp/config/config.py", line 40, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/navarro/.config/coc/extensions/@yaegassy/coc-pylsp-data/pylsp/venv/lib/python3.10/site-packages/pluggy/_manager.py", line 457, in traced_hookexec
    return outcome.get_result()
  File "/home/navarro/.config/coc/extensions/@yaegassy/coc-pylsp-data/pylsp/venv/lib/python3.10/site-packages/pluggy/_result.py", line 114, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/home/navarro/.config/coc/extensions/@yaegassy/coc-pylsp-data/pylsp/venv/lib/python3.10/site-packages/pluggy/_result.py", line 76, in from_call
    result = func()
  File "/home/navarro/.config/coc/extensions/@yaegassy/coc-pylsp-data/pylsp/venv/lib/python3.10/site-packages/pluggy/_manager.py", line 454, in <lambda>
    lambda: oldcall(hook_name, hook_impls, caller_kwargs, firstresult)
  File "/home/navarro/.config/coc/extensions/@yaegassy/coc-pylsp-data/pylsp/venv/lib/python3.10/site-packages/pluggy/_callers.py", line 113, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/navarro/.config/coc/extensions/@yaegassy/coc-pylsp-data/pylsp/venv/lib/python3.10/site-packages/pluggy/_callers.py", line 77, in _multicall
    res = hook_impl.function(*args)
  File "/home/navarro/.config/coc/extensions/@yaegassy/coc-pylsp-data/pylsp/venv/lib/python3.10/site-packages/pylsp_ruff/plugin.py", line 136, in pylsp_lint
    checks = run_ruff_check(document=document, settings=settings)
  File "/home/navarro/.config/coc/extensions/@yaegassy/coc-pylsp-data/pylsp/venv/lib/python3.10/site-packages/pylsp_ruff/plugin.py", line 394, in run_ruff_check
    result = run_ruff(
  File "/home/navarro/.config/coc/extensions/@yaegassy/coc-pylsp-data/pylsp/venv/lib/python3.10/site-packages/pylsp_ruff/plugin.py", line 480, in run_ruff
    (stdout, stderr) = p.communicate(document_source.encode())
UnboundLocalError: local variable 'p' referenced before assignment

No idea how to proceed from there. As I mentioned in my first comment I have followed all the steps in the documentation.

[yaegassy]

@Onturenio I don't know if this is due to an update or change in python-lsp-ruff itself, but it seems to not be detecting the ruff command correctly.

2023-11-23 15:00:36,543 CET - ERROR - pylsp_ruff.plugin - Can't execute ruff with given executable 'ruff'.

Also, all linter processes appear to be terminating abnormally due to python-lsp-ruff errors.

Regarding this issue, try adding the pylsp.plugins.ruff.executable setting to coc-settings.json.

---

coc-settings.json:

"/path/to/" should be adjusted to your environment

{
  // ...snip
  "pylsp.plugins.ruff.executable": "/path/to/.config/coc/extensions/@yaegassy/coc-pylsp-data/pylsp/venv/bin/ruff",
  // ...snip
}

[Onturenio]

I have edited coc-settings.json to put this line:

"pylsp.plugins.ruff.executable": "~/.config/coc/extensions/@yaegassy/coc-pylsp-data/pylsp/venv/bin/ruff"

And the output of :CocCommand workspace.showOutput is still:

2023-11-23 19:23:45,465 CET - ERROR - pylsp_ruff.plugin - Can't execute ruff with given executable '~/.config/coc/extensions/@yaegassy/coc-pylsp-data/pylsp/venv/bin/ruff'.

However that file does exist and it can be executed from the shell normally:

$ ~/.config/coc/extensions/@yaegassy/coc-pylsp-data/pylsp/venv/bin/ruff
Ruff: An extremely fast Python linter.

Usage: ruff [OPTIONS] <COMMAND>

Commands:
  check    Run Ruff on the given files or directories (default)
  rule     Explain a rule (or all rules)
  config   List or describe the available configuration options
  linter   List all supported upstream linters
  clean    Clear any caches in the current directory and any subdirectories
  format   Run the Ruff formatter on the given files or directories
  version  Display Ruff's version
  help     Print this message or the help of the given subcommand(s)

Options:
  -h, --help     Print help
  -V, --version  Print version

Log levels:
  -v, --verbose  Enable verbose logging
  -q, --quiet    Print diagnostics, but nothing else
  -s, --silent   Disable all logging (but still exit with status code "1" upon detecting diagnostics)

For help with a specific command, see: `ruff help <command>`.

Any further idea?

[yaegassy]

Try using the actual path (/home/navarro/), not the ~.

[Onturenio]

OK, that works!

Do you think then this is a bug in python-lsp-ruff, which is not auto detecting the executable?

[yaegassy]

In my opinion, tools installed in the same python environment (including virtual environments) as the language server should be automatically detected and used.

For example, python-lsp-black and pylsp-mypy work that way. It also works that way for yapf, etc., which installs as a language server dependency.

Therefore, I think it is safe to assume that this is a bug in python-lsp-ruff.

Also, I think that if there is some kind of error in python-lsp-ruff, all linter processes on the language servers are stopped. I think this should also be adjusted somehow on the python-lsp-ruff side.

[chris-reeves]

I've just encountered a similar issue (failing to get any diagnostics at all after installing python-lsp-ruff). I believe this might actually a bug with coc-pylsp (or coc.nvim itself) and the way configuration is handled.

System state

• python: 3.11.6
• coc.nvim: 0.0.82
• coc-pylsp: 0.10.4
• pylsp config settings: pylsp.builtin.enableInstallPythonLspRuff: true
• no system-wide installation of ruff

Investigation

When editing a python file outside of any Python virtual environments I see an error similar to the one above:

UnboundLocalError: cannot access local variable 'p' where it is not associated with a value

when the linter runs.

When editing a python file with a venv activated that has ruff installed the linter runs correctly, however other linters (e.g. pyflakes, pycodestyle) which are supposed to be automatically disabled by python-lsp-ruff are also run.

Looking more closely at the language server output:

2023-12-13 11:15:57,428 GMT - ERROR - pylsp_ruff.plugin - Can't execute ruff with given executable 'ruff'.

and comparing this to the source of python-lsp-ruff (the run_ruff() function in particular) shows that pylsp.plugins.ruff.executable must be set to ruff in this scenario:

    if executable is not None:
        log.debug(f"Calling {executable} with args: {arguments} on '{document_path}'")
        try:
            cmd = [executable]
            cmd.extend(arguments)
            p = Popen(cmd, stdin=PIPE, stdout=PIPE, stderr=PIPE)
        except Exception:
            log.error(f"Can't execute ruff with given executable '{executable}'.")
    else:
        cmd = [sys.executable, "-m", "ruff", str(subcommand)]
        cmd.extend(arguments)
        p = Popen(cmd, stdin=PIPE, stdout=PIPE, stderr=PIPE)
    (stdout, stderr) = p.communicate(document_source.encode())

despite the default in python-lsp-ruff being None:

@dataclass
class PluginSettings:
    enabled: bool = True
    executable: Optional[str] = None

Problem statement

The problem is that coc-pylsp is defaulting pylsp.plugins.ruff.executable to ruff in package.json. This is a different default value to that of python-lsp-ruff itself (which is None). With python-lsp-ruff's default value of None, run_ruff() will use the copy of ruff in the current python environment (e.g. pylsp's virtual environment) and things should just work. With the default set to ruff by coc-pylsp, calling run_ruff() will fail if the ruff executable is not in $PATH.

The fact that the linters which should have been automatically disabled by python-lsp-ruff continue to run alongside ruff is likely due to a similar issue. The python-lsp-ruff docs say:

This plugin will disable pycodestyle, pyflakes, mccabe and pyls_isort by default, unless they are explicitly enabled in the client configuration.

and so coc-pylsp setting "default" values for these overrides the python-lsp-ruff default behaviour of disabling these.

Proposed solution

Ideally coc-pylsp shouldn't pass configuration values to the language server that haven't been explicitly set by the user (or are required by the relevant plugin/extension). Unfortunately it seems that defining the configuration variables in package.json (which is necessary to give autocompletion/descriptions when editing CoC config) results in those variables being sent to the language server regardless of whether they've been explicitly set by the user. This is something that would likely need to be fixed in coc.nvim, as I don't think this issue is specific to coc-pylsp.

Something that is within coc-pylsp's power to fix is to ensure that the default values set in package.json match those of the underlying plugins. That would solve the immediate issue with pylsp.plugins.ruff.executable.

The issue of alternative linters not being automatically disabled is a much trickier problem to solve. Admittedly, it looks like python-lsp-ruff is doing something fairly unusual here in disabling other plugins. In the absence of a fix for coc.nvim's handling of defaults for config values, and short of some (probably custom) code for dynamically choosing the defaults for pylsp.plugins.pycodestyle.enabled, etc. depending upon whether python-lsp-ruff is enabled (replicating the code within that plugin itself), it seems that the simplest fix for this would be a note in the README to make users aware of this.

There are no doubt things that could be fixed in python-lsp-ruff to make things fail less catastrophically when the ruff executable isn't found, but coc-pylsp should respect (and not override) its defaults so that things continue to work out of the box.

Workarounds

Specifying the full executable path to the version of ruff installed via this plugin/python-lsp-ruff is one way of resolving this (as @Onturenio has done), but that is quite messy.

Instead, setting pylsp.plugins.ruff.executable: null will revert to the python-lsp-ruff default. Once this is done, editing a python file outside of any Python virtual environments will result in the same behaviour as in an activated venv (ruff runs, but so do supposedly disabled linters). Also setting pylsp.plugins.pycodestyle.enabled, etc. to false will disable those linters again.

[yaegassy]

Hi, @chris-reeves. I have checked that setting pylsp.plugins.ruff.executable to null solves this problem.

Thank you for the pull request. I will check and merge it later.

[yaegassy]

Merged chris-reeves's pull request and published new version of coc-pylsp to npm.

Update coc-pylsp with :CocUpdate command.