« first day (3271 days earlier)      last day (1671 days later) » 
01:00 - 18:0018:00 - 00:00

1:09 AM
Is there a way to make pdb start an interact session without having to enter it manually?
 
 
3 hours later…
3:55 AM
kamori@Kamori-PC:/mnt/c/Users/kamor$ python3.7 -m pdb -c interact /tmp/myscript.py                                      *interactive*
New in version 3.2: pdb.py now accepts a -c option that executes commands as if given in a .pdbrc file, see Debugger Commands.

Source: docs.python.org/3/library/pdb.html
 
4:36 AM
 
5:34 AM
@holdenweb Oh, right. I'm stupid.
@AndrasDeak I think some CPython versions between 3.4 and 3.6 handle _ differently than 3.7 and 2.7 do. I only have 3.7 and 2.7 installed though, so I can't test it
 
6:13 AM
Lots of drama happening on Meta again.
 
6:35 AM
*sigh* just noticed I didn't commit one of the files in my project before I left and now I can't do any work until I get back home. Why can't git just automatically commit all the dang changes
 
commit -a will probably do that
Unless you have a new file
 
it is a new file :/
 
That's why you don't just use commit -a ;)
I always add everything manually so I have to check the changes every time.
 
technically I have to blame IntelliJ, but git still annoys me
 
Ah, IDEs...can't comment on that
 
6:56 AM
the default of pycharm is to commit new files in commits when using the GUI. Or just always run git status before you push and only do so if you are cool with what you see.
 
Running git status and looking at it defeats Aran's purpose. He just wants it To Work. Even if he forgot that he had scribbled down a file named tmp that includes his credit card details (and no, "gitignore" is an unacceptable response :P)
 
yeah it usually adds new files, but this isn't the first time it hasn't done that. Happens every now and then
By the way, is it common practice to commit the .idea folder? Sometimes it tries to commit files from there
 
37
Q: Should I ignore the .idea folder when using PyCharm with Git?

RayI read about Git integration in PyCharm, and created a Git repository from PyCharm. I did this in PyCharm because I was hoping PyCharm would know whether the .idea folder should be ignored, and if that's the case, it would automatically create a .gitignore file with the line .idea/ in it. But i...

I always put the whole folder in .gitignore, I didn't realize there was nuance to that discussion
 
I wonder if IntelliJ respects the global .gitignore. I should try adding it there, I guess
Clicking through the question you linked, but haven't found a single reason to commit anything from the .idea directory yet
 
Same
 
7:38 AM
Guess the output:
class NonDataDescriptor:
    def __init__(self, name):
        self.name = name

    def __get__(self, instance, owner):
        return self.name

class DataDescriptor(NonDataDescriptor):
    def __set__(self, instance, value):
        pass


class NonDataMeta(type):
    desc = NonDataDescriptor('meta')

class NonDataClass(metaclass=NonDataMeta):
    desc = NonDataDescriptor('class')


class DataMeta(type):
    desc = DataDescriptor('meta')

class DataClass(metaclass=DataMeta):
    desc = NonDataDescriptor('class')
 
7:52 AM
descriptors and meta classes... you're expecting a lot for a monday morning
 
8:24 AM
re.findall(r'\d{8}', '/home/201909114/text_20190916.sometext')
i only want to extract 20190916 from the above string ( extract only 8 digit string prefix and suffix by some text)
i tried re.findall(r'.\d{8}.', '/home/201909114/text_20190916.sometext')
the stricg can be variable and i cannot apply split method to get the date
 
@pythonRcpp you need a literal \. after the digits
 
no not just a dot , it could be an underscore aswell, anything<8digitsonly>anything
i am trying to extract those squeezed 8 digits
 
but you'll match the directory name first...
you have to somehow explain that you're not looking for that
perhaps negative lookaheads for /
 
directory name is not 8 digits, its 9 digit
actually i should say anynondigit<8digitsonly>anynondigit
 
so say that
 
8:31 AM
is what needs to be extracted
 
you're looking for (negative lookbehind not digit)(8 digits)(negative lookahead not digit)
 
8:47 AM
Hi guys! Does anyone of you here do big projects in web development (like web analytics) using Python Django?
 
9:05 AM
Is it just me or is the spoiler utility not working any more? sopython.com/spoiler
 
the spoiler button userscript works still
 
@AndrasDeak woah, it's been years since my last slowpoke reference. took me a long while to remember that
 
9:19 AM
cbg
 
9:30 AM
@Aran-Fey my guess
 
9:54 AM
Python3: I have an XML file I have run open and then print on. Works fine. How can I show all newlines and such as \n etc? I currently just print(file.read()).
 
@DrOnline sounds llke you don't particularly care that it's XML and we shouldn't either
for line in file: print(repr(line))
will do something similar to what you want, though repr() does more than make the line endings visible
if that's really all you want, print(line.replace('\n', r'\n'))
 
Aha - thanks a lot.
 
if your requirements are somewhere in the middle, add more replace actions, or tell us in more detail what more you need
 
cabbage
too broad stackoverflow.com/questions/58134439/… - imprecisely described question
 
10:40 AM
Hi guys is there a way in cartesian product to set start and stop line with iterools.product?

I will try to exlplain. This code give me Adc, Ada, Ad3, Adq , A2c and so one.

Is there a way to start from Adq and stop on b2a. I tried with itertools.count(start=0, step=1) but no luck.

Adq is 4 on list and b2a 22. Can I add somehow itertools.count(start=4, stop=22)

import itertools

somelists = [

['A', 'b', 'C', '1'],
['d', '2', 'A', '4'],
['c','a', '3', 'g']


]
for element in itertools.product(*somelists):
 
10:55 AM
@Pijes Almost. Take a look at dropwhile & takewhile docs.python.org/3/library/itertools.html#itertools.dropwhile
 
@PM2Ring Thank you. I will try that
 
@Arne only half correct!
 
But if you want to stop on "b2a" you can shorten your 1st list to ['A', 'b']
 
@PM2Ring This is just example. I have many list to implement solutions
 
import itertools

def get_product_values(somelists, from_, to_):

    _from = tuple(from_)
    _to = tuple(to_)

    gen = itertools.product(*somelists)
    while True:
        elt = next(gen)
        if elt != _from:
            continue
        else:
            yield elt
            break
    while True:
        elt = next(gen)
        yield elt
        if elt == _to:
            break


somelists = [
    ['a', 'b'],
    ['c', 'd'],
    ['e', 'f'],
    ]

gen = get_product_values(somelists, 'ade', 'bde')
 
11:00 AM
I also have a Level 2:
class DataMetaMeta(type):
    desc = DataDescriptor('metameta')

class DataMeta(type, metaclass=DataMetaMeta):
    desc = DataDescriptor('meta')

class DataClass(metaclass=DataMeta):
    desc = DataDescriptor('class')

print(DataClass.desc)
 
@ReblochonMasque Thank you but not work with my list
I can't edit list all the time. I have many lists
 
q is not in your list!!!
Which is why I did not use your data!
 
That's just typo
 
from Adq and stop on b2a will never work if q does not appear.
No that's not 'just a typo'!
That is the data you provided
 
@Pijes "not work" is very vague. You need to be specific. Reblochon Masque's code looks fine to me.
 
11:07 AM
@PM2Ring I get error. That's why don't work
 
Thank you @PM2Ring
What error did you get?
 
RuntimeError: generator raised StopIteration
 
Restart your kernel if you are on ipython
 
The same error
 
Yes, it throws StopIteration when it exhausts itself after not finding q!
 
11:10 AM
@Pijes Thankyou. You will get StopIteration if the _from or _to args never appear in the product.
 
Place it inside a try/except block to protect against typos (non existent data in your arguments)
 
@ReblochonMasque Now it's work. Thank you
 
Cool, you are welcome. :)
This approach exhausts the generator prior to your start point; Maybe there is a better way, IDK.
 
Have a nice week. And thank you again. There is always better way but I think that will be good for now.
 
11:28 AM
cbg folks!
Just saw that a lot of mods were resigning from SO.
 
cbg
@Aran-Fey huh, don't the docs just say that metaclasses are not considered for attribute resolution?
@GamesBrainiac yeah, that looks mildly scary and concerning..
 
datetime.datetime.strptime("20190916_9:15",'%Y%M%d_%H:%m')
ValueError: unconverted data remains: 5
why am i getting this
 
@Arne Yes, I cannot imagine Rob leaving. He has been here forever. I wonder what Shogi is upto.
 
hey is there a way to ask pip freeze to create a requirements.txt with latest as version for all the things, instead of specific versions?
I don't really wanna get stuck on these arbitrary versions, but just update the code with the packages
 
@pythonRcpp %M is minute and %m is month, see docs.python.org/3/library/…
 
11:32 AM
@Arne Sounds like you trust the docs more than I do :D
 
datetime.datetime.strptime("20190916_9:15",'%Y%M%d_%H:%M')
error: redefinition of group name 'M' as group 5; was group 2 at position 103
 
you have M twice
 
Where did you get that info @GamesBrainiac? asking for a schadenfreunde
 
start here and links therein
 
Thanks @AndrasDeak
 
11:38 AM
@Aran-Fey not sure if they are wring at times, but the docs always get confusing to me when the question is whether types behave like proper objects. :/
 
@Hakaishin you should only list the dependencies you directly use anyway, and let your package manager figure out what secondary ones you will get. If you do that, you should only have a hand full remaining, in which case it shouldn't be too much work to turn a numpy==1.17.2 into numpy>=1.17
@Aran-Fey I wanted to try to answer that question without test code, seems that's going to be tough.
> The default behavior for attribute access is to get, set, or delete the attribute from an object’s dictionary. For instance, a.x has a lookup chain starting with a.__dict__['x'], then type(a).__dict__['x'], and continuing through the base classes of type(a) excluding metaclasses.
 
@AndrasDeak there's a hidden level of indirection, actually. ;)
 
was I too naive?
 
I have no idea what "continuing through the base classes of type(a) excluding metaclasses" even means
that's like saying "all cars excluding bicycles"
 
@Arne So how do I do that? Is there a way for pip freeze or other tool to generate a list of needed packages with biger than x version number?
I mean ofc I understand that I can do it by hand, I'm asking if somebody knows an automatic workflow
 
11:48 AM
@Arne - this is sad - that explains why the 'featured on meta' was pulled off 3 weeks ago. Nothing happens by accident!
 
@MisterMiyagi I was wondering about that wording too, uni taught me that baseclass and metaclass are completely distinct concepts.
@Hakaishin ehm.. history | grep "pip install" should be a good start
I don't think there is a tool. But if your project is complex writing a script that collects all third part imports from your source files shouldn't be too hard
 
@Arne No, you're on the right track. But it'll be difficult to find all the relevant pieces of information in the docs
In fact I'm not convinced that all the pieces even exist
 
12:05 PM
Let's say my unittests rely on a bunch of data to confirm the code is working correctly. But into git goes just code, how do you guys handle such additional data, just shared drives or cloud?
 
@Hakaishin that's a big topic you're just casually cutting into
@Aran-Fey I guess I'll end up just trying stuff, my attention span doesn't allow for in-depth docs hunting =/
 
Hmm, come to think of it, maybe I had a slight misunderstanding about how descriptors and attribute access interact. In which case the docs do contain all the necessary information
 
@AndrasDeak That's what I thought, too. Not sure if it "really" counts because it isn't in a do-while loop.
 
@Hakaishin Test data is usually pushed alongside the test code, you got to write your tests in a way that a small amount of data is enough to assert that the code is working as expected.
 
Hmm I see, that's kinda what I thought too.
 
12:11 PM
cbg guys o/
 
is 60Mb a lot of test data? I guess not, but not sure
Lol also 50 of that is 1 exe, where the only test is if I can read the version info which should be bundeled with the exe.
 
you usually shouldn't upload an exe to git
 
then how do I make this test pass? The test checks if I can get the version of the exe
 
hard to answer if i don't know how your environment works. but in general, once the code is pushed a, test suite will run. there, it will first try to build to project (inclduing the exe). then it will run the unit tests (using the exe). The exe doesn't need to get uploaded to git, since its building should be part of the tests that are run, and then the other tests have access to it.
 
@Hakaishin for text data, no. for binary data, yes.
 
12:18 PM
Ah, yeah I guess that could be done as well good point
 
diffs of binaries are usually very inefficient, and small changes can snowball to large diffs
@Aran-Fey can you link to what you have in mind? I found the docs on data descriptors to be very vague
most importantly whether descriptors are part of Python, of object or type
 
well, considering how tightly type and object are bound to the interpreter, it's often hard to say whether a mechanism is part of object or type or the interpreter
 
true. I've ran into some cornercases of overwriting __getattribute__ and friends where the distinction would have been relevant.
I only remember tears and frustration, though ^^
 
How do you guys test database stuff? Right now I just once installed the db and then the table creation, data filling etc was tested. Now I'm migrating to a new pc and all the test are failing because there ofc is no db available. Do you have the db installation also as a test? Sounds like a lot of effort for little gain or do you just manually install it on every new system?
 
12:27 PM
Can you mock it?
Then you provide the mocks with the tests
 
I think all the information you need to figure it out is contained in these two paragraphs:
> For instance, a.x has a lookup chain starting with a.__dict__['x'], then type(a).__dict__['x'], and continuing through the base classes of type(a) excluding metaclasses. If the looked-up value is an object defining one of the descriptor methods, then Python may override the default behavior and invoke the descriptor method instead. Where this occurs in the precedence chain depends on which descriptor methods were defined.
 
I'm in the test the real thing camp
 
> Data and non-data descriptors differ in how overrides are calculated with respect to entries in an instance’s dictionary. If an instance’s dictionary has an entry with the same name as a data descriptor, the data descriptor takes precedence. If an instance’s dictionary has an entry with the same name as a non-data descriptor, the dictionary entry takes precedence.
 
I asked a more senior dev, if they mock it or test the real thing and they told me they use the real thing, had bad experience with trying to mock the db
 
D=
 
12:30 PM
Really? O_o
 
are we still talking about unit tests?
 
Good question @Arne
 
type.__getattr__ has conditional blocks that refer to descriptors more than once so I'm inclined to say that the mechanism is part of type, as opposed to being part of object or the interpreter
 
Arguably typeobject.c itself is part of the interpreter :P
 
@Aran-Fey nice, that was informative. DIdn't know that HowTo existed.
 
12:33 PM
At this level the distinction does indeed get foggy
 
@Aran-Fey vaguely waves at gibberish in the PyPy source repository
 
In comparison, PyObject_GetAttr does not mention descriptors
 
But yeah, it's actually relevant that descriptors are invoked by object.__getattribute__/type.__getattribute__ because that means you can prevent that by overriding __getattribute__.
 
fun fact: I find the C code of CPython easier to understand than the RPython code of PyPy
 
And the bytecode interpreter's implementation of LOAD_ATTR doesn't either
 
12:35 PM
IIRC there are some hacks that circumvent descriptors -- basically the same that prevents special methods to be fetched from instances
 
@Arne Sorry, tests, using the badly named unit test package which to my knowledge can also be used for all kinds of tests
 
it might be worth it to separate unit tests and functional/smoke/integration/high-level tests
and the unit tests should then be free of any kind of external dependency like DB or web-APIs, and just mock that stuff
 
I was going to say "in principle it should be possible to create your own type class without inheriting from or otherwise depending on the built-in type type" but it might be tricky to completely purge built-in type from your hierarchy
If you define class KevinType:, then KevinType's metaclass is type. You can define class KevinMeta: and class KevinType(metaclass=KevinMeta):, but then KevinType's metaclass' metaclass is still type
If only you could somehow do class KevinType(metaclass=KevinType):
 
that'd require a special kind of magic
 
12:47 PM
Also, is it even possible to have an MRO that doesn't end with object? We don't want to use object either, since object's metaclass is type too.
 
Only if you use pythoff
 
I suspect it used to be possible with old-style classes
So yeah, Python 2 only
 
>>> class Foo:pass
...
>>> Foo.mro()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: class Foo has no attribute 'mro'
*confused screaming*
 
@Kevin unrolled loop of length 7 ;)
 
This reminds me of when I was trying to define the class hierarchy for KevinScript and I couldn't figure out how to say "type's metaclass is type". Or something like that, I don't think KS actually has metaclasses
@AndrasDeak The reader simply has to use their imagination and add a do{...}while(0); block around the real code. I'm sure it compiles to the same result anyway.
 
12:57 PM
Yup
 
Impress your managers by adding 10,000 lines of code to your repository with this one weird trick
do{
    do{
        do{
            do{
                ...
 
baby shark
 
            }while(0);
        }while(0);
    }while(0);
}while(0);
 
1:14 PM
I'm confused by this syntax. Those dictionaries don't look valid, the parentheses around the 0s aren't necessary and those weird characters that kind of look like colons are apparently greek question marks?
weirdest code I've seen in my whole life
 
It is the tongue of an ancient and advanced people, whose kingdom sank below the waves. Too many memory leaks.
 
I made a haiku with it:
the tongue of ancient people
sunk below the waves
too many memory leaks
 
1:31 PM
I award full points despite it not adhering to the typical 5-7-5 measure, since English syllables don't correspond directly to on anyway
 
1:45 PM
Ha, that's a 7-5-7 Haiku! :D
 
I'm imagining a sandwich alignment chart, but for haikus
structure purist - "A haiku must have 5-7-5 syllables"
structure neutral - "Each line in a haiku must have 5 or 7 syllables"
structure rebel - "any number of lines, any number of syllables"
The "ingredient" axis would relate to the presence of seasonal and cutting words. Most English-speaking haiku enthusiasts would fall under "rebel" there since they don't know those requirements even exist
 
Yes, I fxed up! :d
 
Let's argue about whether "waves" is a seasonal word. Without context it feels pretty summery to me, thanks to its association with beach vacations. In context it's rather season-neutral since presumably the ancient C people were seafaring year-round
 
@Aran-Fey That's Foo.__mro__
 
@Kevin "leak" is a seasonal word for sure, since they have a designated harvesting time. The german translation is even "springtime onion"
 
2:01 PM
Hmm true
 
@Kevin I have tried but did not manage yet. all builtin types are off-limits since they are objects. You'd need an extension type, but Cython only produces object subtypes. So you'd need to define your own type and object equivalents in C.
 
@MisterMiyagi naw, that doesn't exist either. This is python 2
 
yargl adjusts monocle
 
@MisterMiyagi That's what I was afraid of.
 
morning, folks!
 
2:14 PM
As an additional complication, although I don't think the interpreter gives any special privileges to type, I think it does give special privileges to object. PyObject references get passed around like candy and PyObject_SomeMethods are called for all sort of things
If MiyagiObject can't be seamlessly casted to PyObject, you're going to have a bad time
 
PyObject is not object though - it's basically just a struct with a type slot
I'd be willing to compromise and accept that as pure enough
but every time I try to do this, I end up with a LISP-like meta-language on top of LLVM inserted by import hooks
actually, I end up starting with that and having to do more important things
 
Having this problem in Tensorflow+keras :======> ModuleNotFoundError: No module named 'tensorflow.keras.layers.core'
MY versions are :--------
TensorFLow ~~ 1.14
Keras ~~ 2.24
 
Ah, if there's a distinction between PyObject and object, then there's hope
 
2:29 PM
???
 
Different conversation, don't worry about it
 
Any help, then?
Or is there a different room which handles such problems....
 
I know nil about Tensorflow or Keras, but glancing at the documentation I see no reason to believe that a core module should exist there
 
On second glance I see that object is defined by PyBaseObject_Type
 
2:43 PM
Yeah solved it. The problem was in my imports....
 
If we use list() instead of [], does the performance increase?
 
I suspect that list() is slower because it has to do a name lookup
 
[] is taken from an internal object pool
list() is a lookup in the builtins namespace and a regular call
in principle, [] is faster
 
>>> import timeit
>>> timeit.timeit("list()")
0.5218025560006936
>>> timeit.timeit("[]")
0.11293462800131238
 
@Kevin so when we use [], there is no lookup?
 
2:56 PM
I would have thought the literal would have been faster anyway, since it doesn't involve a function call
 
That's right. It compiles directly to the BUILD_LIST bytecode, no LOAD_NAME lookup required
 
actually, this might be a good dis exercise
 
I don't think I've ever used zero-argument list() in production-quality code
And if you're the type that likes to use list as a variable name, you're just asking for trouble
 
lol @ the number of times I've seen lst and lis as variable names in SO posts, to avoid calling it list
 
If they wanted to adhere to PEP8, they should call it list_ instead ;-)
> If a function argument's name clashes with a reserved keyword, it is generally better to append a single trailing underscore rather than use an abbreviation or spelling corruption. Thus class_ is better than clss. (Perhaps better is to avoid such clashes by using a synonym.)
I notice that the immediately preceding paragraph recommends the use of cls for class methods, even though it says to avoid abbreviations in this paragraph. Truly, Python contains multitudes
 
3:02 PM
the proper name is probably something related to the problem -- even if it is just a generic items
 
I find it peculiar that PEP8's advice is specifically for function arguments and not just general variable names. I guess that means clss is hunky dory if it's a regular local?
 
Looks like exactly the issue we are kinda having :D
 
Hot take: since list is not a "reserved keyword", You can use it as a variable name / function argument / public attribute while still being completely PEP8 compliant
 
I occasionally use lst, but mostly use seq
 
Yeah, I favor seq as well
Especially because I rarely write functions that are designed to work only on lists, and more often write functions that work with any type that's indexable and iterable
 
3:19 PM
What do you think guys. Is there some better solution for this. This code work but I need some faster solution because this code check every line. Do Python have some skip to line 10.000 or something like that? This is just example. I have larger and many list to implemment solution.


import itertools

def get_product_values(somelists, from_, to_):

_from = tuple(from_)
_to = tuple(to_)

gen = itertools.product(*somelists)
while True:
elt = next(gen)
if elt != _from:
continue
else:
yield elt
break
 
@Pijes Oh dear, something went wrong with your indentation. Consult sopython.com/wiki/… and try again.
You could also put your code into a pastebin, which usually does a good job of preserving formatting
 
@Kevin Ok. Thank you
 
I do think it's possible to write a variant of product that skips over the first 10,000 irrelevant values. It's an interesting problem, so I'll try to put something together
 
@Kevin Thank you Kevin
 
There's a good chance that someone will find an existing recipe online before I can write one. What a fun race we're having.
 
3:30 PM
@Pijes have a look at itertools' dropwhile and takewhile
they will run the fast forwarding and early exit without going through the interpreter main loop
if you want to stay with explicit loops, be aware that there is no point using while just to call next(iterable) -- you can just use a for loop
 
for competitive programming is python is a good choice or i will have to go with c++ ?
 
depends on the competition. If you're looking for execution performance on an embedded system, C++ might be the better choice. IF you're looking for code golfing or readability, etc, python
 
c++ is a safer bet, if only because many competitive programming sites won't even support python
 
@ParitoshSingh i am python developer but when i try to give interview for companies they ask data structures
 
@MisterMiyagi Thank you. I will try. For me at this moment it's more important to skip 10.000 lines or more if I need. The end is not that important
 
3:40 PM
@ParitoshSingh and normally they ask queue and stacks which python doesnt have
 
languages and data structures are orthogonal. Knowledge of data structures is pretty language-agnostic
 
python has queues and stacks
 
Just to clarify, competitive programming and "interviews" and two very different things as well.
 
3:42 PM
Do not take advice on one and apply it to the other.
 
thank you all guys . i got your point.
 
@Pijes you should end up with something like this:
def get_product_values(somelists, first, last):
    fast_forward = lambda item, _first=tuple(first): item != _first
    last = tuple(last)
    iterable = itertools.dropwhile(fast_forward, itertools.product(*somelists))
    for element in iterable:
        yield element
        if element == last:
            break
 
could you guys weigh in on this please?
If I have enough rep to edit a post without it having to be approved, then should I not have sufficient rep to approve someone else's edit without requiring another approval vote?
 
hold on
 
I'm gonna go ahead and agree
 
3:53 PM
No, that's what review queues are for: there's always a vote needed. You can have a binding vote with the "improve edit" and "reject and edit" options though
 
Hey, thanks @MisterMiyagi, nice rewrite! :)
 
I mean it kind of makes sense to request that, but people approve of the worst kind of crap, so extra pair of eyes is always useful...
 
but that's just the thing. I could "Approve and Improve" without really improving anything. I think that creates and equivalence, n'est ca pas?
 
compromise: In the review queue, it takes 2 votes. On the question itself, it only takes a single vote. That way you don't have to waste 5 minutes staring at unformatted code while you wait for someone to come along and cast the 2nd approve vote
 
but that's only if the edit is specific to formatting changes
 
3:57 PM
@MisterMiyagi It's slow. Still testing but slow like previous
 
@Pijes you may want to clarify that. is the time-to-first-item high, the time-between-items, or is it just slow overall?
note that you have asked for speeding up a linear scan - it is much faster not to do a linear scan, but to compute the first item. is that feasible in your real use-case?
 
@MisterMiyagi Slow like previous solution. I need to wait 10-15 min till start to get results. Can that be done faster with itertools.islice
But I get error TypeError: islice expected at most 4 arguments, got 5
 
@MisterMiyagi I suggested dropwhile a few hours ago...
5 hours ago, by PM 2Ring
@Pijes Almost. Take a look at dropwhile & takewhile https://docs.python.org/3/library/itertools.html#itertools.dropwhile
 
@PM2Ring I wasn't aware there was a history
listening to some very boring conference talks here :/
@Pijes you need to feed the product to islice, not the individual iterables
 
@Pijes, here's my take on it: pastebin.com/WF0tQmZu
 
4:02 PM
for element in itertools.islice(itertools.product(*somelists), 5, None):
    print(''.join(element))
 
I suppose it should be possible to "jump" directly to the first desired product, but I can't think of a clean way to do it using itertools.product Ah. Kevin has some code...
 
you can compute the index of the first accepted item, provided one operates on sequences, not iterables
 
My approach will quickly yield the first item, but I suspect after that point it's a bit slower than itertools.product, since I'm doing a fair bit of arithmetic
 
@Kevin Thank you Kevin. I will try your solution
 
You can probably reduce the math required with some clever use of generators, but ultimately it's still going to be O(N), so I didn't bother
 
4:10 PM
first = ('A', '2', 'a')
first_index, prev_len = 0, 1
for sub_list, sub_first in zip(reversed(somelists), reversed(first)):
    first_index += sub_list.index(sub_first) * prev_len
    prev_len = len(sub_list)

for element in itertools.islice(itertools.product(*somelists), first_index, None):
    print(''.join(element))
this computes the index of the first accepted item, and uses islice to fast-forward to it
the computation is basically the same idea as flattening a multidimensional array
 
@Kevin It's pretty clean though. I was thinking of something recursive, possibly using cycle.
 
My first design was recursive but it had a lot of special cases so I gave up on it
Basically I had four different approaches depending on whether from_ was None or to_ was None, or both, or neither
 
@MisterMiyagi This work. Thank you
 
Hmm, can islice jump to the Nth product in O(1) time? I suspected not but maybe there's some magic going on.
 
isclice is still O(n)
 
4:14 PM
@MisterMiyagi islice still has to consume the unwanted products, though. But I guess it's faster than testing each one in dropwhile
 
yes, the entire approach is basically pushing stuff into a few builtin operations
 
@PM2Ring It is really fast solution.
And suits me the best at this moment. Thank you guys. You are awesome smart people. And good too
 
I might try to defend my pride by coming up with input where mine is faster :>
Please forgive my high salt levels
 
totally unrelated sidenote: zip(reversed(somelists), reversed(first)) just pushed my desire to build a python compiler/rewriter by more than 9000 MMMUs
 
MisterMiyagiMegaUnits?
 
4:19 PM
MisterMiyagiMotivationUnits. It's part of my counter-proposal to SI units, defined entirely on the mood of my cat.
metres are so last century. The MisterMiyagiSlouch is the average distance my cat is willing to travel for food.
 
@Kevin That would be very magical, at the Turing halting problem oracle level.
 
Oops, now that I have a less threadbare testing suite, my approach is producing the wrong output
 
@MisterMiyagi One of Donald Knuth's first published works was an article on silly units for Mad magazine.
In issue 33, Mad published a partial table of the "Potrzebie System of Weights and Measures" developed by 19-year-old Donald E. Knuth, later a famed computer scientist.
 
4:38 PM
@Kevin Thank you Kevin, your solution works fast too. Now I have two working solutions, thanks to you guys. Have a nice one
 
@Pijes Please don't use my solution, it's broken for some inputs
I think this one works though. pastebin.com/Kgap2cf9
 
@Kevin OK I didn't notice
 
wim
@smci yes, search by OP rep would be a nice addition to stack exchange APIs. hope of getting such useful feature implemented is slim to none (but Aran-Fey userscript is a decent workaround)
 
@MisterMiyagi I think you might have run into index math troubles similar to the ones I had, if this test case is any indication
Who would have thought that dynamic-radix numerals would be not entirely trivial
 
@Kevin yes, the index calculation is utterly wrong indeed
what a bummer
it must be prev_len = len(sub_list) * prev_len
 
4:53 PM
morning cabbage
good news: I still have my job
 
👍
 
although, there are very likely going to be layoffs next month ;-(
 
@MisterMiyagi I think you're on to something. So like:
 
@PM2Ring sadly, I'm too old to rival this. and inventing time-travel means everything I learned goes down the drain. Lose-Lose indeed. Looks like Knuth won that round...
 
first_index, radix = 0, 1
for sub_list, sub_first in zip(reversed(somelists), reversed(first)):
    first_index += sub_list.index(sub_first) * radix
    radix *= len(sub_list)
This at least passes my one test case
 
4:55 PM
I'm confident it's off by no more than a constant
 
Anyway here is my contrived test case where my approach is somewhat faster. You may now complain about how this is not representative of real-world use cases while I stick my fingers in my ears and hum
I didn't use timeit because I can never remember how to run it on functions that have arguments without inadvertently importing main recursively forever
 
@Kevin I cannot complain about an epic battle of miyagi_product versus Kevin_product
truly epic
sticks fingers in ears and hums
a sight to behum
 
Hum something appropriately epic, like Flight of the Valkyries
Let me go get my "music for when two radical wizards shoot fireballs at one another from adjoining mountain peaks" mixtape
I must begrudgingly disclose that I lose by a wide margin when args = (somelists, (9,9,0,0,0,0,0), (9,9,9,9,9,9,9))
 
I'm almost tempted to boot up my computer & dust off my ancient MixedBaseCounter code...
 
5:07 PM
An actual purpose-built dynamic-radix number incrementer would definitely be an improvement over my approach
 
rubarb!
 
It was one of the 1st things I wrote in Python. But I have made a few improvements over the years. I originally created it for doing permutations, but I figured I might as well make it general, so it could be used for Cartesian products.
 
Guys, I want to invite a user to chat so that we don't spam the comments. But the user has below 20 reputation on SO, so I can't invite them. Am I missing something?
 
Hmm, last I checked, ordinary users can't circumvent the 20 reputation limit, even if they own the room
 
Thanks, Kevin. I guess the best way to proceed is just continue in the comments and then clean everything up?
 
5:15 PM
hold on, it might be possible to give explicit access now
 
And I have another one that I guess is implicit: is this room limited to English?
 
yes, all of chat.SO
 
Rgr, thanks. =)
 
If you created your own room, though, I don't think anyone would mind you using a different language.
 
5:18 PM
only of someone notices
> Stack Overflow (en) Chat won't allow discussion in non-English.
If people still want to discuss in non-English, they can go to chat.SE
Ideally, there's a (language).SO site already created. If there is, make a room associated with that site. If there isn't, associate it with SO on c.SE.
the problem is not being able to moderate foreign-language chat, and chat.SE has a lot more mods, and from all over the place
 
Hmm, makes sense
 
A probability problem I've been wondering about for a while: Say you have a deck of cards containing n cards. You may remove any number of cards from the deck. Then, if the top card is a specific card (1 out of n), you win. How many cards should you remove to have the highest chance of winning?
...the answer is "it doesn't matter", isn't it
 
I suspect the chance is the same regardless of how many are removed, as long as at least one is left
Remove all n and the odds drop to 0%
[I too like to live dangerously.png]
 
I would agree on "it doesn't matter" unless there are some other steps in choosing the card.
 
For some reason typing it out made it easier for my brain to process. Thanks for being my rubber duck once again, room 6
 
5:28 PM
Here is another benchmark of the sliced product problem, this time using a mixedBaseCounter that PM proposed. It's faster than my arithmetic based approach, although by a smaller margin than I predicted
Maybe the counter would be faster if it didn't use yield and instead implemented the iterator protocol directly
 
@Kevin That looks very similar to my code, although I think I use a while loop instead of for idx in range(len(radixes)-1, 0, -1), IIRC.
 
That would save a cycle or two, I expect
 
I don't know if you'll get much speed-up by avoiding yield. But I guess it's worth testing. I assume plain yield is optimized, and doesn't have any overhead related to the extra stuff that a yield expression (that sends to the generator) does.
 
5:49 PM
@wim guess the featured tag somehow symbolizes that SO endorses the post, have no idea why the would have an edit war over that?
 
Before I go on a wild goose chase, there's no matrix M that turns [x,y]*M into [x,1/y], right?
 
@Kevin Right. You can only get terms of the form a*x + b*y
 
Ah well. I can still accomplish what I want, but it's more expensive now
 
01:00 - 18:0018:00 - 00:00

« first day (3271 days earlier)      last day (1671 days later) »