« first day (3510 days earlier)      last day (1422 days later) » 
01:00 - 17:0017:00 - 00:00

wim
5:00 PM
(and pipx doesn't magically solve that problem for you, it just automates the "one venv per install" tedium somewhat)
 
@JamesMcIntyre note that importx is the name of the Cython import hook. Might want to pick something else.
 
Ahh so you are saying that to fix that problem, you would have to rename sub dependencies Wim?

Good to know, thanks MisterMiyagi. I'll think of anouther name
 
wim
@JamesMcIntyre yes and vendoring is essentially renaming deps too. msgpack is renamed as pip._vendor.msgpack in sys.modules
it's a glorified copy-and-paste of source code.
 
Ahh I see. My main reason for doing this is it just seems much simplar that using a differt vertual enviroment for every project and more elegent.

vendoing seems like a good thing to look into. Thank you :)

P.s. I've just imporved the code so it's global. It was previously staded that it wouldn't work becuase they didn't have a folder called "JamesMcIntyre" on their computer and now it would.
import os

CL = os.getcwd()

os.chdir(os.path.expanduser("~") + '/.local/pipx/venvs/opencv-python/lib64/python3.8/site-packages')

import cv2

os.chdir(CL)
 
wim
IMO it's an ugly solution because import statements have to be rewritten (above you can see vendored cachecontrol importing from vendored msgpack, for example)
 
5:07 PM
but you still need to create a vitural enviroment for every project no? rather than just install what you need and... "use it"
 
wim
you still didn't handle the "cache hit" case where the module already exists in sys.modules. it means that the first time cv2 got imported, that version will win for later imports too.
 
@JamesMcIntyre Now what happens if projectA needs opencv version X and projectB needs opencv version Y? One of them has to import it from somewhere else
 
wim
some other users already tried to explain this point to you but I don't think you understood it..
really what we need here is a major import system feature to be added into the language directly, I don't believe there is any acceptable hack around with the way sys.modules and python packaging currently works.
 
hrm...I need more chalkboards (totally missing having an office with whiteboards stapled to every wall right now :\ )
 
wim
I would like to see something like this:
import cv2 @ opencv-python>=4.2
syntax for specifying a package name and a specifier constraint at the import time
 
5:18 PM
That seems inconvenient. What if you need to import that same version of cv2 in multiple places?
 
wim
the specifier would be optional
without the stuff after the @ would be equivalent to "satisfied by cv2 from any installed package at any installed version"
currently we have site-packages/cv2/*.py
 
Ok, but that doesn't really solve the problem. Say you need exactly version 1.2.3 and you need it in both foo.py as well as in bar.py. Will you hard-code the version number twice?
 
wim
with my idea, it would be more like site-packages/cv2/opencv-python/4.2.0.34/*.py
if both foo.py and bar.py specify to pin it, then yes, I would hard-code that twice
 
I guess a workaround would be to do the import in a dedicated module and then import the module from there. Like from .imports import cv2, where imports.py contains import cv2 @ v1.2.3
 
wim
if foo.py and bar.py were both contained in some package, then this part could be lifted up to an top-level __init__.py somewhere
 
5:23 PM
And then you'd do from . import cv2? That'd work
 
wim
yeah
Features we are currently missing that I want to see:
- multiple distributions can provide the same import name
- different versions of package can be simultaneously installed
site-packages/cv2/opencv-python/3.4.9.33/*.py
site-packages/cv2/opencv-python/4.2.0.34/*.py
site-packages/cv2/other-opencv-provider/1.2.3/*.py
then import system figures out which one to use. if there is a specifier which restricts to a version and/or a provider that is not currently installed, that can be an ImportError
 
@Aran-Fey I'm not following you sorry. Both A and B are in their own vertual envriometns and so have their own versions of their own dependencies
 
So /.local/pipx/venvs/opencv-python/lib64/python3.8/site-packages is the virtualenv for projectA, and projectB's virtualenv is somewhere else? It doesn't say projectA anywhere in the file path, so how does that work?
 
so in the above example opencv (cv2) would be one module and there would be a second module imported.

cv2 in this case is in it's own vertual envrioment and so has all of it's own dependencies which don't afftect other modules dependencies (unless there is a cashing issue)
 
I don't really follow, but as long as you're aware that you need to separate the dependencies of different projects, that's fine
 
5:32 PM
pipx sepeartes the dependenices for you. pipx is not the same as pip
 
wim
5:53 PM
> which don't afftect other modules dependencies
they do though!
just the fact that they were installed to a separate venv doesn't change that. sys.modules remains a plain old dict of {module_name: module}
pipx is great, but you seem to be overestimating what it actually provides here.
 
6:07 PM
All of this doesn't solve the problem that most modules are not written with multi-versioning in mind. Unless a module is always used only internally, V2 will have to deal with its bizarro twin V1. Or V1.9, or V2.1, or V2.0.0.dev4
 
wim
most modules are not written to be thread-safe either, that would be up to the author if they want to do that or not
 
Versioned imports themselves could at least be done with an effect-handler-like import hook. You set a desired version and all consequential imports use that – e.g. with vimport.pytest >= 4.3: import pytest.
 
wim
you still want something like {(module_name, provider, version): module} in sys.modules if you are going to have multiple versions usable
 
yes, the hook/handler would have to take care of the bookkeeping. sys.modules would be compatible with that, though.
 
If virtual environments do not affect each other then what's the point of them. You might as well just install every external module to global?
 
6:20 PM
@MisterMiyagi wim should implement it and call it wimport
@JamesMcIntyre FYI you can edit/delete messages for 2 minutes in chat
 
wim
@AndrasDeak hah. cute, but I'd want it to be an extension of the import statement grammar (and sys.modules datastructure) directly
 
@AndrasDeak thanks
 
@wim so your library wimport patches and recompiles cpython. Problem solved.
 
@wim I'm wondering whether having a working prototype would be sufficient to promote a PEP on syntax support.
 
wim
doing this kind of thing in third-party is how distutils/setuptools and pkg_resources got into such an unholy mess in the first place. it has to be owned by the language proper, otherwise it will be forever tangled up by the "needing to be installed/setup in the first place" conundrum.
 
6:23 PM
ugh, I used my previous laptop for 5 days, and already I'm missing keys being back on the new one
 
wim
@MisterMiyagi probably- could do it in a CPython fork, and show in python-ideas first
 
sounds a bit too revolutionary for a steering council (saying this only based on my prejudice of councils; I have no idea how python's has been working)
 
wim
restructuring the sys.modules in a way that's backwards compat would be the most tricky part, I think
(any solution that is not backwards compat would be DOA)
 
yeah
 
wim
maybe sys.modules could be a multidict (same key but many values, quacks like the old dict when necessary)
 
6:27 PM
although if you write a handy helper script that helps fix backward compat issues...
 
Some of you guys don't seem to think that creating this public repo is a bad idea. Considering "Importx" is already taken. Any ideas for a name? [hovers finger over create repo button]
 
wim
that's like asking someone else to name your kid
 
@wim Surely it would be easier and cleaner to introduce a new dict in addition to sys.modules?
 
wim
@Aran-Fey and maintain both sys.modules and sys.modules_v2? hmm.
 
yeah, that's the idea
 
6:33 PM
@wim lmao Musk should possibly have done that
 
wim
so what would actually go in sys.modules? first version loaded, or it gets dynamically updated?
 
I'm not sure the 2nd option is even possible tbh. Storing the first loaded version should be fine anyhow
 
with the way sys.modules currently work, versioned modules should either live there under a dummy name or not at all.
if you want to go all the way, sys.modules would have to be thread-local/contextvar, reflecting the current restriction on modules.
is there some canonical dupe for adding __repr__ to a custom type?
 
6:53 PM
Can anyone tell me why __import__('cv2') isn't working the same as import cv2?

Stack overflow answers seem to imply that it would?
 
The main difference between the two is that __import__ doesn't bind the module object to a name. You'd need cv2 = __import__('cv2') for that
 
so I'm trying to do the following, but it's not working:

def PipxImport(mod):
import os

CL = os.getcwd()

os.chdir(os.path.expanduser("~") + '/.local/pipx/venvs/opencv-python/lib64/python3.8/site-packages')

__import__(mod)

os.chdir(CL)
ooo so you need some kiknd of "as"
 
The easy way is to return the module object and require the user to use syntax like cv2 = PipxImport("cv2"). If you want just PipxImport("cv2") to magically create a variable named "cv2" in the calling scope and bind it to the module object automatically, that's going to be tricky
In some cases the language will flat-out refuse to acknowledge the existence of the variable even if you inject it with brute force. A lot of name resolution happens at compile-time, so if it doesn't see an assignment statement in the right place, it will assume nothing by that name exists
 
TLDR: It only works for globals.
 
Yes that works. Thank you!

def PipxImport(mod):
import os

CL = os.getcwd()

os.chdir(os.path.expanduser("~") + '/.local/pipx/venvs/opencv-python/lib64/python3.8/site-packages')

mod = __import__(mod)

os.chdir(CL)

return mod
(it works with cv2 = PipxImmport('cv2'))
 
7:06 PM
"It only works for globals" isn't too dire of a drawback, since import statements are supposed to all be in the global scope anyway
 
Erp, just had a function local import last week.
Might actually be useful with version support, so that specific functions could work with specific version.
 
wim
@MisterMiyagi if you find one, you can close stackoverflow.com/q/9799493/674039 to it as well
 
7:53 PM
Thanks again for all your help guys.

I've created the repo now and I'm goning to call it a day. Have a good nigth all (or day, depending where you are)

https://github.com/FunckyFizz/PipxImport
 
Good night and alarm bells
Imports are already somewhat wonky in Spyder. It's probably not the best IDE to be developing this project :P
 
To be honest, my eyes are going wonky :P

But before I go. how are imports wonky in Spyder?
 
Feb 21 '18 at 20:50, by roganjosh
I found something unusual with Spyder/iPython yesterday through using logging that I now need to look up. Within Spyder, it's perfectly ok to have logger = logging.handlers.RotatingFileHandler (it even auto-completes for me) but running python my_script.py with exactly the same version of Python will fail; it only accepts from logging.handlers import RotatingFileHandler. Any ideas what Spyder/iPython is doing here in the broader sense?
 
I'm afraid I don't know much about loggers. I've only just come accross them recently when a coleage at work sent me a scrtpt to review which included them.

( I work in the automations department and we use an SQL table for logging outcomes of automations)
I do appricate the warning though. If my imports are going a bit squif then I'll try running it outside of Spyder
 
The module is not important. What is important is how it resolves imports
Click the timestamp of the quote and you can jump to the discussion
 
8:03 PM
@MisterMiyagi Good God please no, not in the actual code. Better to have a version-requirements/dependencies file, in a separate file. (But then it comes down to what tool/build manager/package manager is supposed to enforce/update that)
 
@JamesMcIntyre there is (was?) an interesting behaviour where on spyder you didn't have to import scipy.optimize (e.g.), it was enough to import scipy and you already had access to its submodules. Now, this doesn't work outside spyder. Really good way to write broken code.
 
ahh, that makes sence. I'll watch out for that.

Thanks guys. I really need to go now as I eyes are turning to mush and I was supposed to call a friend hours ago
 
I'm really conflicted over my recent Spyder criticisms. It got me through so many years of learning and I still fire it up to answer SO questions because it is really simple to use. There's booby traps all over, but it is dear to me
 
8:56 PM
Hmm...numpy arrays are not Sequences, just Iterables
 
Hello all, I am attempting to learn programming using python. While WWW3 schools is a great source for learning the concepts, does anyone know of any free tutorials where I can learn to develop an entire application using python
 
I personally think that W3 Schools is a poor intro to python because it has some inaccuracies. What kind of thing do you have in mind for using Python for?
 
Is python your first language?
 
9:32 PM
I had a question about typing, can I ask it here?
 
Sure
 
I have a class like this
class F:

    def __init__(self, a=None):
        # type: (Optional[int]) -> None
        self.a = a # type: Optional[int]

    def check(self):
        # type: () -> None
        if self.a is None:
            raise

    def f(self):
        # type: () -> None
        self.check()
        self.a += 1
In f, I am calling a function which checks if self.a is None, so a type checker shouldn't complain when I run this piece of code through it right
but I get something like error: Unsupported operand types for + ("None" and "int") on self.a += 1 when using mypy
Although writing this in another way, where I take the definition of check, and put in in f, passes type checking
class F:

    def __init__(self, a=None):
        # type: (Optional[int]) -> None
        self.a = a # type: Optional[int]

    def f(self):
        # type: () -> None
        if self.a is None:
            raise
        self.a += 1
 
wim
@roganjosh can you try out this on spyder
OP claims it doesn't work, I was curious about why. I know nothing about spyder.
 
@wim ModuleNotFoundError: No module named 'readchar'
Do I need to go through and fix the imports etc?
 
wim
no
that's just a third party dep, pip install it as usual
 
9:40 PM
@wim we know that it can easily produce weird results
 
@AndrasDeak I'm happy to test it anyway
 
I'm either missing something or they misunderstand the distinction between Spyder/Anaconda. I was foot-out-the-door for a cig when the ping hit, so I'll go ponder it for a few mins :)
 
9:59 PM
wait_for(key=sys.argv[1], timeout=int(sys.argv[2])) is causing issues for me; I have never passed arguments to Spyder code, but it can be done (lol at the accepted answer here).
readchar.readkey() is not an issue, though
 
10:13 PM
@roganjosh "code executed inside Spyder", you mean
 
wim
Command line args are not really related, can just hardcode "x" and 3 for arv1 and argv2 if you want. I'm more interested in the timeout handling and how their env has interfered with it. That user sees some crash in matplotlib, which has nothing to do with the code..
 
matplotib also does some weird thread and interrupt things...so perhaps there are multiple shenanigans getting tangled
 
wim
importing matplotlib.pyplot in the module doesn't break anything for me
 
@wim Well, with that, I can confirm that it doesn't work in Spyder
 
wim
"running" it in an IPython session works too
I wonder what exactly Spyder is doing that messed things up
 
10:26 PM
It's crashed the kernel as they stated, too
 
nice
 
wim
does a threading Timer without the os.kill work?
e.g. one that just prints to stdout after 3 seconds
 
Can you give me the exact code you want me to run please? Clearly my interpretation was off on the first trial
The kernel didn't auto-restart from that last trial, either, btw. I've had to restart Spyder
 
wim
11:02 PM
oh you meant for just them timer?
threading.Timer(3, lambda: print("hello from 3 seconds later")).start()
it should be a non-blocking call and it should spam stdout with that string exactly 3 seconds later
 
Oh, hold on. I'm getting errors in red now
 
actually, the hello is from 3 seconds earlier...
 
This was from me making mistakes and bashing keys, but may be relevant later:
 
already closed
 
11:10 PM
@wim it crashes the kernel:
 
your threads are leaking out into chat :P
 
Well, I am using my phone for interwebs. Tiktok would be all over this, right? :P
I'm just thinking back to the happy googly-eyed robot dancing away in a garden and not about the realities of debugging. I feel happier :)
I have to go to bed. I'm only 50% sure that the SyntaxError shown in that SS was caused by something I'd already set running
 
11:35 PM
huh...co-worker just got mad at me because of the answer I gave when they asked how to use a CHECK constraint to see if a column references a unique value in a different table (my answer was "use a foreign key" for the record)
 
11:56 PM
Hello
I know theres this rule that question must not be posted here if its not old enough, but please can I? I have a whole public server that depends on me, and if I dont get help soon, it might be dead
-_-
 
01:00 - 17:0017:00 - 00:00

« first day (3510 days earlier)      last day (1422 days later) »