This forum has been archived. All content is frozen. Please use KDE Discuss instead.

problem with python code completion using anaconda

Page 1 of 1 (4 posts)

Tags: None
(comma "," separated)
adamc
KDE Developer
Posts
46
Karma
0

problem with python code completion using anaconda

Wed Oct 30, 2019 11:34 am
Hi,
my problem is that code completion is not working for the pytorch module.
Code: Select all
import torch

torch.zero*Ctrl+Space*

gives only ZeroDivisionError, but not the torch.zeros() function, which is in the torch module. Strangely, torch.tensor works (torch.tensor is a class, while torch.zeros is a function afaik). but after creating a torch.tensor object, i can't complete on its methods. I can't complete on method parameters either. I believe that it has worked on my machine before, but i might be mistaken.

my environment:
I'm using kdevelop 5.4.2 from the kubuntu repositories, python 3.7 and anaconda. anaconda is installed into ~/bin/anaconda.
$ conda --version
conda 4.7.12
$ which python
/home/me/bin/anaconda3/bin/python
$ python --version
Python 3.7.4


what I have tried so far:
* code completion works on standard modules like math
* starting via PYTHONPATH=/home/me/bin/anaconda3/lib/python3.7/site-packages/ kdevelop
* setting the python interpreter path in the project configuration to /home/me/bin/anaconda3/bin/python
* in kdevelop settings / python documentation generate docs for the torch module

generating the documentation module looks successful, but i'm not sure it runs correctly as the return type is 'None' for most (or all) functions:
$cat ..kdevpythonsupport/documentation_files/torch.py
...
def zeros():
"""
zeros(*size, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) -> Tensor

Returns a tensor filled with the scalar value `0`, with the shape defined
... more docs ...
"""
return None
...


some suspicious shell outputs while running Kdevelop are:
kdevplatform.serialization: repository "/home/madam/.cache/kdevduchain/kdevelop-{fdc4bc3a-3044-436f-aba1-f80bddb27fab}" was write-locked, it probably is inconsistent
kdevplatform.serialization: "The data-repository at /home/madam/.cache/kdevduchain/kdevelop-{fdc4bc3a-3044-436f-aba1-f80bddb27fab} has to be cleared."
qrc:/qml/storage.js:24: TypeError: Cannot call method 'openDatabaseSync' of undefined
kdevelop.plugins.python.duchain: [ !!! ] did not get a typetrack container object when expecting one! Fix code / setup.
kdevelop.plugins.python.duchain: tuple type object is not available
kdevelop.plugins.python.parser: Discarding parts of the code to be parsed because of previous errors ## i fixed those later on ##
SyntaxError: ('invalid syntax', ('<kdev-editor-contents>', 13609, 21, 'def tpass#rch.ops():\n'))
---- Parsing FAILED ----
OverflowError: Python int too large to convert to C long
qrc:/qml/storage.js:24: TypeError: Cannot call method 'openDatabaseSync' of undefined
OverflowError: Python int too large to convert to C long
kdevplatform.language: invalid item for index 57 0 0
TypeError: can't convert complex to int
OverflowError: Python int too large to convert to C long
kdevplatform.language: Could not open file "/usr/share/kdevpythonsupport/documentation_files/PyQt4/__init__.py" (path "/usr/share/kdevpythonsupport/documentation_files/PyQt4/__init__.py" )

(but those didn't appear every time i started kdevelop)
flherne
Registered Member
Posts
23
Karma
0

Re: problem with python code completion using anaconda

Wed Oct 30, 2019 4:02 pm
I could reproduce this locally.

Because pytorch is mostly not written in Python, kdev-python can't parse it directly to find functions or types and relies on the stubfile produced by the 'Generate Documentation' tool.

There appears to be a bug in the documentation generator, which causes a syntax error in the generated Python file.

If you edit `~/.local/share/kdevpythonsupport/documentation_files/torch.py`, and edit line 13609 to replace `def torch.ops()` with just `def ops()`it should work.

torch.tensor works anyway because that's one of the few parts of Torch that actually *does* exist in Python code - there's a `site-packages/torch/tensor.py` file, and kdev-python parses it fine.

EDIT: The return types of the generated stubs are indeed `None`, which is an expected limitation of the generator.
At some point, kdev-python might be able to use the `.pyi` stubfile shipped with pytorch instead of trying to generate one itself, which would solve both of these issues.
adamc
KDE Developer
Posts
46
Karma
0

Re: problem with python code completion using anaconda

Wed Oct 30, 2019 6:16 pm
Thanks a lot!

flherne posted a workaround in the irc chat, namely copying the .pyi files into documentation_files/torch.py
this gives not perfect, but okayish completion. Certainly much better than before :)

not sure this is overkill or not, but i wrote a short script to do that for all torch modules:
Code: Select all
cd $CONDA_PREFIX/lib/python3.7/site-packages
dest="$HOME/.local/share/kdevpythonsupport/documentation_files/"
for i in $(find ./torch -iname "*__init__.pyi"); do
    mkdir -p ${dest}$(dirname $i) &&
    cp $(pwd)/$i $dest${i::-1};
done;

for i in $(find ./torch -iname "*.py"); do
    mkdir -p ${dest}$(dirname $i) &&
    ln -s $(pwd)/$i $dest${i};
done;

use at your own risk!


not perfect: torch.tensor doesn't auto complete for instance
Code: Select all
import torch
M = torch.tensor([[0]])
# M. doesn't show anything useful
ronaldgevern
Registered Member
Posts
1
Karma
0

Re: problem with python code completion using anaconda

Tue Jun 28, 2022 7:25 am
The super class of ZeroDivisionError is ArithmeticError. This exception raised when the second argument of a division or modulo operation is zero. In Mathematics, when a number is divided by a zero, the result is an infinite number. It is impossible to write an Infinite number physically. Python interpreter throws “ZeroDivisionError: division by zero” error if the result is infinite number. While implementing any program logic and there is division operation make sure always handle ArithmeticError or ZeroDivisionError so that program will not terminate.

Code: Select all
try:
    z = x / y
except ZeroDivisionError:
    z = 0


Or check before you do the division:

Code: Select all
if y == 0:
    z = 0
else:
    z = x / y


The latter can be reduced to:

Code: Select all
z = 0 if y == 0 else (x / y)

Page 1 of 1 (4 posts)

Bookmarks


Who is online

Registered users: bartoloni, Bing [Bot], Google [Bot], Sogou [Bot], Yahoo [Bot]