MYPYPATH doesn't respect PEP420 namespace packages

[mr-rodgers]

Take this directory tree:

/path/to/src
└───namespace
    └───package
        │   __init__.py
        │
        └───__pycache__
                __init__.cpython-35.pyc

And this MYPYPATH:

> $env:MYPYPATH
/path/to/src

then running mypy -p namespace.package gives:

> mypy -p namespace.package
Can't find package 'namespace.package'

However, mypy works fine when you add an empty __init__.py into namepace (turning it into a regular package).

[gvanrossum]

I know... I'm not sure what to do about this. The algorithm from PEP 420 is quite complex (namespace packages are only considered if no "classic" package or module is found). The workaround of adding a dummy __init__.py is simple enough (you could also have a dummy __init__.pyi so that Python is not affected by this file).

I also expect that usually you'd want to create a separate set of stubs rather than type-check the original source code (though obviously this may vary per package).

[djetelina]

I'd like to bump this. In future, there will be python programmers who never heard of pre PEP420 era of __init__.py and that's good. What's not good is mypy forcing them to create dummy files.

[agronholm]

It might get the ball rolling if somebody explained why this is such a big problem for mypy.

[agronholm]

I just bumped into Jukka Lehtosalo and he said there shouldn't be any major issue blocking this. But given the easy workaround that exists for this, it'll likely require an outsider to write a PR.

[gvanrossum]

What's not good is mypy forcing them to create dummy files.

This issue is more complex than you think.

[djetelina]

I don't think it's trivial, I've read your previous comment - so I don't even think I'd be able to write a PR. I just think it's an issue worth looking into eventually

[richardmss]

I'm working on a project with a directory structure like the following:

./a
./a/c
./a/c/file1.py
./b
./b/c
./b/c/file2.py

Note that both, a and b, contain a subfolder called c. Adding an __init__ everywhere defines some modules multiple times (the actual project contains dozens folders) and things get hairy. Any progress on this issue would be much appreciated.

[gvanrossum]

Adding an __init__ everywhere defines some modules multiple times [...]

I presume you mean in each c folder but not in a or b right? So c is a namespace package and you have both a and b in PYTHONPATH or sys.path. (And when running mypy, you pass it both a and b, which will have the same effect -- or perhaps you pass it their parent directory or the current directory.)

With that setup, I agree you have a predicament. We will have to support namespace packages in mypy. At the very least it should be possible to do something like setting MYPYPATH=a:b and then run mypy -m c and have it find both parts of c.

[carljm]

Just a philosophical note here that might influence the way mypy handles this: in my experience in a Py3-only world, packages without __init__.py are not a special edge case for namespace packages, they are the default case. It's pretty unreasonable to expect a new Py3 developer to remember to drop empty __init__.py turds everywhere when their code demonstrably runs perfectly fine without them. It's hard to even make a sensible case in code review for why they should add __init__.py (other than "it's the way we've always done things in the old world, get off my lawn" or "the typechecker is buggy and won't find your code without it.") The presence of __init__.py becomes the special case, used only when you need some package-level code or imports.

This reversal may not have been an intended consequence of PEP 420, but AFAICS it's an inevitable one.