« first day (3247 days earlier)      last day (1702 days later) » 
00:00 - 20:0020:00 - 22:00

12:55 AM
Here's an idiot question, but there's no short idiom/operator to do integer division of a float by an integer, and get a (duh) integer result instead of float result:

>>> 4.2 // 3
1.0
>>> int(4.2 // 3) # No I do not want to/should not need to do this constantly, it kills readability
1

Also, I would have expected '//' integer division would automatically downcast float to integer, as long as it's within range (MAXINT); or at least that the `divmod()` wrapper could offer a non-default switch to downcast ('coerce_to_int'). Because there's not much reason not to, as far as I can see (sure, t
I suppose I could subclass float and override float.__truediv__(). But equally float class could have gotten a classmethod __truediv_int__(), or a non-default flag coerce_to_int[=False]
 
 
1 hour later…
2:15 AM
cabbage
 
any matplotlib pros around here?
this line throws this error:
ValueError: Floating point image RGB values must be in the 0..1 range.
full log:
Traceback (most recent call last):
  File "C:/Users/payne/Documents/GitHub/PixelEnhancer/SR-ResCNN-Keras-/predict.py", line 175, in <module>
    plt.imsave('output/' + args.save, enhanced, cmap=plt.cm.gray)
  File "C:\Users\payne\Anaconda3\envs\ml-gpu\lib\site-packages\matplotlib\pyplot.py", line 2140, in imsave
    return matplotlib.image.imsave(fname, arr, **kwargs)
  File "C:\Users\payne\Anaconda3\envs\ml-gpu\lib\site-packages\matplotlib\image.py", line 1496, in imsave
    rgba = sm.to_rgba(arr, bytes=True)
thing is... this script used to work 100% fine. All that's changed is that I've updated my matplotlib version in the virutalenv that I use :(
 
2:29 AM
solved: enhanced = np.clip(enhanced, 0, 1)
how dumb
I guess it could be that my new model simply hasn't converged enough yet
 
 
2 hours later…
4:48 AM
How this works? Here words = [ 'ab', 'bc', 'cd', 'de']
words[1::2] = [word[::-1] for word in words[1::2]]
I'm getting what happens but not getting how
Ok got it :)
But why these 2 are not same?
i = 0
for word in words[1::2]:
    words[1::2][i] = word[::-1]
    i += 1
and words[1::2] = [word[::-1] for word in words[1::2]]
The first one doesn't change the list words
But the 2nd oneliner code does change words
Can anyone tell me how the codes are different?
 
5:40 AM
@Quark words[1::2] makes a slice of words, and words[1::2][i] changes a value in that slice. You didn't assign that slice anywhere, so it gets discarded afterwards.
 
5:54 AM
@smci // is floor division, not integer division. the conversion is consistent with the regular numerical tower (docs.python.org/3/reference/… and en.wikipedia.org/wiki/Numerical_tower)
in other words, it is highly unlikely that there is a builtin method to do what you want
use a fractions.Fraction instead of float if you want rational, not real arithmetics
 
 
2 hours later…
8:12 AM
how do you parse non - args using argparse? Like pip install xyz instead of pip --install or similar
 
parser.add_argument('install') instead of parser.add_argument('--install')
 
I thought that was a placeholder, like instead of writing install the value you wrote there got passed
also, what would be a good way of making a program both executable from cli but also from other programs? If there are too many parameters then making a function and passing all the parameters to it seems weird
 
write a library, then add a user interface to it write a user interface that depends on the library
a library doesn't necessarily need to be a single massive function with a dozen parameters
 
currently I've written a main function like `def main(input_files, targets=('vocals', 'drums', 'bass', 'other'), outdir=None, model='umxhq', no_cuda=False,
samplerate=44100, niter=1, alpha=1, softmask=False, residual_model=False):` that can be imported but also called from argparse such as `main(args.input, args.targets, args.outdir, args.model, args.no_cuda, args.samplerate, args.niter, args.alpha,
args.softmask, args.residual_model)`.
eh formatting failed
yeah um the main function calls other functions but then the same thing would need to be implemented twice if not wrapped up in a whole function
 
8:29 AM
in that case I see no better solution than having a function with 10 parameters
 
8:59 AM
i assume this means dictionaries are a pain/impossible to pass via cli?
 
not that difficult... just JSON-encode them
though that probably makes an awful user interface
 
ah i see
 
9:28 AM
cbg
 
@aadibajpai modifying commands like pip install or git commit correspond to subparsers in argparse
 
9:50 AM
oh right, parser.add_argument('install') is how you make positional-only arguments. My bad
 
 
1 hour later…
11:15 AM
cbg guys o/
Any good tutorial you would recommend on writing Unit Test cases in Python ?
<I can Google; but it might be possible that I wont get landed on a good article.>
Also where should I have test folder in my project ? My project looks like this:
Project:
    src:
       db_module //written on python
       backend_module //written on python
       frontend_module //written on angularjs
So should I have separate test folder for each python module or should I keep a common test folder which will be as part of root project folder ?
 
have a test folder on the project root that mirrors the src folder's directory structure. But instead of db_module/do_stuff it's db_module/test_do_stuff
 
Wakrimashtha! Understood!
Watermelon Arne
I have test folder there already D:
May be generated by IDE
Looks like writing UnitTest cases is way too easy job: geeksforgeeks.org/unit-testing-python-unittest
 
11:33 AM
no
don't use unittest
 
^ pytest is where it's at
 
10 times better ecosystem and way more pythonic
 
pytest ?
 
Thank you _/\_
Since the real code is in sibling(i.e. src) directory; what is best way to import them in test folder ?
import from src ?
 
11:37 AM
if you ever have something named src in an import then you should immediately rename that thing, because src is an awful name for a package
 
Ah ok. But what we should do when we can't rename it ?
btw src doesn't have the code but its subdirectories does.
 
you import your module as though it were installed; import db_module or whatever
 
your IDE and/or test framework should automatically make it importable if you got the project structure right
 
When you say "if you got the project structure right", what way is the right way of structuring project ?
Do you mean each package should have that empty __init__.py ?
 
11:42 AM
empty? Every package should have an __init__.py, preferably one that's more useful than an empty one
 
empty as in it won't have any code written.
 
Aran's sentence still stands :P
 
project/
    docs/
    project/
    tests/
    setup.py
^ that's how I do it, but I faintly remember hearing that there should be a src folder somewhere for some reason
 
project/
^ and i pray i never have to package it :P
 
@Aran-Fey Thanks for sharing that. Would you please tell me what does that setup.py do ?
@ParitoshSingh Looks like I did not understand then ^^
 
11:45 AM
So, the code inside an init file executes when you import. You can use that to "clean up the namespace" so to speak.
 
So that's how it works.
 
tbh I rarely ever write setup.pys, so I'm not exactly sure about what should be inside
haven't found a good packaging guide yet, either
 
Ok no problem
Oh boi! executing test cases is so easy; all we need to do is py.test on root directory and rest is taken care by the test apis
@Aran-Fey like I was suspecting it is not letting me to import the module
 
12:01 PM
try running your tests from inside the src folder then
 
In that case it won't find the test cases right ?
since test cases I wrote are outside src
 
pytest ../tests should work
 
Oh Ok
Not working
 
what's the error?
 
Tim
What IDE are you using, @TheLittleNaruto?
 
12:07 PM
Aran, Error is: ImportError: cannot import name
@Tim PyCharm
 
Tim
@TheLittleNaruto Give me a couple of minutes, I had exactly these problems and can explain the set-up, just need a moment to get some screenshots (and also get my head around this SO chat!)
 
@Tim Thank you; will wait.
 
Tim
EDIT: Oops, like I said, still figuring out how SO chat works
 
huh, for some reason pytest ../tests doesn't work but python -m pytest ../tests does
 
Not working either
How do we specify module which can be imported in __init__.py ?
 
12:20 PM
you can't; every file and (sub-)package in a package can be imported
 
Thinks! because I have not added modules in init.py, it is not letting me to import?
 
that is not how __init__.py works
can you import the root of the package? what is your actual import statement?
 
yeah at this point we're gonna need a MCVE
 
@MisterMiyagi from db_module.sample.utils import convertToMongoSupportedDate
 
which of these imports fails? db_module or any of the nested ones?
 
12:23 PM
I'll have to check that
 
the error message should tell you which it is
 
What is the most readable way to "pretty-print" these two lists such that the commas align:
The lists given:
[-2, 1, -3, 4, -1, 2, 1, -5, 4]
[-2, 1, -2, 4, 3, 2, 3, -4, 4]

Expected output:
[ -2,  1, -3,  4, -1,  2,  1, -5,  4]
[ -2,  1, -2,  4,  3,  2,  3, -4,  4]
 
@MisterMiyagi Sorry my import is "from db_module.sample.utils.CommonUtils import convertToMongoSupportedDate" and it is throwing error for "CommonUtils."
 
Okay, I came up with this solution, let me know if you know any better way...

for n in arr_1:
	print(str(n).rjust(3, ' '), end=',')
print()
for n in arr_2:
	print(str(n).rjust(3, ' '), end=',')
 
btw utils folder has __init__.py and it has a line of code: "from .CommonUtils import *"
Could it be a problem ?
 
12:32 PM
only if the import fails
 
import is failing only for test
 
def aligned_repr(seq):
    return "[" + ",".join(repr(item).rjust(3) for item in seq) + "]"

print(aligned_repr([-2, 1, -3, 4, -1, 2, 1, -5, 4]))
print(aligned_repr([-2, 1, -2, 4, 3, 2, 3, -4, 4]))

#output:
#[ -2,  1, -3,  4, -1,  2,  1, -5,  4]
#[ -2,  1, -2,  4,  3,  2,  3, -4,  4]
Nearly the same as what you come up with, but my two cents:
Of course, this won't nicely line up arbitrary lists with long elements such as [1,2,3333333333] and [4,5555,6]
 
@TheLittleNaruto can you please post the exact error? there are different ways how an import can fail
is there a class .CommonUtils.CommonUtils?
if not, does this work? from db_module.sample.utils import convertToMongoSupportedDate
 
@MisterMiyagi No it's not.
@MisterMiyagi Tried this one; not working
 
Is db_module.sample.utils.CommonUtils a module or a class? Module names should use snake_case
 
12:42 PM
@Aran-Fey It's just a module not a class.
 
Here is a little recipe for aligning the print representations of lists of arbitrary data. Making it work on lists of different length is left as an exercise to the reader.
 
Kevin, in your code, what is the reason for using "repr()" as opposed to "str()"?
 
MisterMiyagi, I shared error log above.
 
Because the native list-to-string method calls repr on each of the list's elements (... I think) and I want to keep the styling as similar as possible
 
Tim
project/
    docs/
    src/
        db_module/
            do_stuff.py
        backend_module/
        frontend_module/
    tests/
        db_module/
            test_do_stuff.py
        backend_module/
        frontend_module/
    setup.py
 
12:47 PM
If you change line 9 of my recipe to use str instead of repr, i.e. row.append(str(item).rjust(width)), then the first line of the result of the first sample input will be [1, 2, 3333333333, I've got a lovely bunch of coconuts]. Notice the lack of quote marks around the string element.
 
@Tim Yeah I am following this way
 
Tim
@TheLittleNaruto

Sorry, couldn't mix the project structure and the rest of the instructions, as I needed fixed-width for the project structure

On your `src` directory, right-click -> Mark Directory as -> Sources root.

Then set up your pytest run configuration:
1) Go to Run -> Edit Configurations.
2) Click '+' (Add New Configuration) -> Python test -> pytest
3) Set the following settings:
- Name: "pytest in tests" (or whatever you want, really)
- Target: Script path
- (path): path/to/your/project/tests
 
Yes, list.repr does call repr on its elements. Strike the "I think" from the record
 
@Kevin I'd be inclined to use f"{item:>3}"
 
I thought about it, but I always forget the syntax for padding using a variable width in an f string
f"{item:>{width}}" or some such.
 
12:55 PM
@Tim Thanks for the screenshots; followed and still facing import error
 
@Kevin Yes. And you can do f"{item!r:>{width}}" if you want the repr conversion instead of str
 
Tim
@TheLittleNaruto Did you mark your src directory as Sources root?
 
Tim
And is the target and working directory both the tests directory?
 
12:57 PM
The f-string docs are here: docs.python.org/3/reference/…
 
Tim
What are the import statements in the test file you're getting the error from?
 
15 mins ago, by TheLittleNaruto
Complete error log: https://gist.github.com/TheLittleNaruto/c1d56b9c45491ecadef7d441469885c9
 
Tim
Is D:\Work\MySampleProject\test\db_module\sample\utils\__init__.py empty?
 
The general format mini-language docs are also helpful: docs.python.org/3/library/string.html#formatspec
 
Tim
and I assume there's a function called convertToMongoSupportedDate in D:\Work\MySampleProject\test\db_module\sample\utils?
 
1:00 PM
28 mins ago, by TheLittleNaruto
btw utils folder has __init__.py and it has a line of code: "from .CommonUtils import *"
Yes there is python module named CommonUtils which has this method. @Tim
 
Tim
Could you try changing that import to from db_module.sample.utils.CommonUtils import convertToMongoSupportedDate and delete __init__.py (just for now, to try to narrow down the error), and run it again and see if you still get the import error?
 
erm, are you sure you are importing db_module.sample.utils from source and not the unittest folder?
path db_module\sample\utils\test_CommonUtils.py and import db_module.sample.utils implies there is a collision for everything but test_CommonUtils versus CommonUtils
 
Tim
That was the point of my instructions further up about setting up the PyCharm configuration with the src directory marked as Sources Root, and then the pytest run configuration having the 'Add Sources Root to PYTHONPATH' option checked.
 
@MisterMiyagi Ah I see;
 
Tim
It should effectively do an os.path.append("src") when it runs pytest, so all the source code should then be importable.
 
1:07 PM
even if src is importable, only one of src and tests can be imported if names are the same
 
Then what would be the way to import the source one ?
 
Tim
@MisterMiyagi Yes, good point. How does Python determine if there's a name collision like that?
 
rename your test modules :P
 
@Tim it doesn't. it just picks the "first" in the module search path
 
1:09 PM
BRB
 
Tim
@MisterMiyagi Yep, sorry, I meant "determine which one to use [if there's a name collision]", rather than detect the name collision itself.
 
e.g. test_db_module\test_sample\test_utils\test_CommonUtils
 
Tim
Presumably, if you just rename the top-level module and then always use full imports, you don't need to rename all the submodules?
 
yeah, the absolute paths have to differ. renaming the root should be enough
 
Tim
This is really helpful - I'm now squinting at my own project, wondering if I've got enough test_ renaming in my tests or not...!
 
1:13 PM
pytest only searches for tests in files that start with "test_", so if your modules have name conflicts you're doing something very wrong
 
A recent answer says, "try using an array: arr = []". I point out that [] is not an array. The answerer agrees and edits their post. "try using a list: arr = []". I'll take this as 75% of a win.
 
sounds about right
you can edit the remaining 25% against their will
 
I won't, but the fact that I can is empowering.
 
Would naming it lst really be a significant improvement, though? Still a pretty useless name
 
It's useless, not misleading. So yeah, significant improvement.
 
1:19 PM
I s'pose
 
arr causes a name conflict with the text-to-speech library I'm writing that's targeted towards pirates
I usually name my lists seq although perhaps that's wrong since tuples and strs and byteses are also sequences
 
Maybe they named the list arr in preparation for International Talk Like a Pirate Day. ;)
 
`seq[0]` --> fine, all sequences implement `__getitem__()`
`seq.append(x)` --> hold up.
 
I'm also a fan of seq, especially as a function arg when any sequence-like object is permitted.
 
I can never remember the differences between iterables, sequences, containers, collections, and whatever other fancy names there are
 
1:25 PM
I employ extreme duck typing, where I call whatever methods I want, and everyone just has to deal with it
 
@MisterMiyagi thank you
 
@Aran-Fey Same. There are a lot of them.
 
@Aran-Fey That's ok, as long as you provide enough context so people know what you're actually talking about.
 
Yeah after changing the name I am getting error for the db_module itself
 
We don't call lists arrays but sometimes we call containers collections. Pedant level: 7/10
 
Tim
1:29 PM
Yes, but it seems like TheLittleNaruto had a rather tricky specific case where they have modules with quite complicated structures, and have mirrored those in their tests directory, but only put `tests` as the root (comparable to `src`) and then `test_do_stuff.py` for the final source files at the end, but haven't renamed every directory inbetween as `test_`. So after the tests or src root name, the paths only differ at the end on the python file level.

(with me so far?)

BUT the twists then comes, that they're using `__init__.py` files to neaten up namespaces, so their imports are `from m
 
There's not much point agonizing over using precise jargon if your intended audience can't remember the differences between the terms either.
 
Tim
@TheLittleNaruto An import error, or just an error with the test?
 
An import error
So it's not able to find the module: db_module
 
Every additional point on the pedant scale requires the speaker to invest ten times as much effort into composing their message, so there's definitely diminishing returns
One only reaches the higher echelons if they love technical correctness for its own sake, or if their space probe will crash into a comet if they don't
 
I added from src.db_module... and it's working now
 
Tim
1:31 PM
Okay, that would suggest the PyCharm thing isn't working.
 
"Ah, I probably should have specified metric tons on the blueprints"
 
Tim
(as in, adding Sources Root to PYTHONPATH)
Are you running the configuration you set up as per my instructions? (i.e. pressing the green arrow)
 
Yeah it's working by running from configuration as well
 
Tim
Okay, add a file called conftest.py to tests with the following inside:
import sys
sys.path.append("src")
 
sys, you mean sys
 
Tim
1:36 PM
Thanks - edited!
 
When I get around to writing a Python distribution, I'll make sys.path a tuple >:-)
 
@Tim are you related to @Kevin? You look a lot alike.
 
Tim
Hehe!
 
@Tim Done!
 
the only thing worse than adding a path to sys.path is adding a relative path to sys.path
 
1:38 PM
@Kevin It may also have diminishing returns for the readers as well. Eg, IMHO a lot of physics & maths pages on Wikipedia are now less useful for the general reader (with a reasonable command of high-school level physics & maths) than they were a few years ago.
Well-intentioned editors have added a lot of technical detail that make the pages more accurate, but which are likely to overwhelm the non-expert. And the experts can just consult the appropriate textbooks, they don't really need Wikipedia.
 
Note: I have changed the name of all the folder to test_folder; for example: test_db_module like suggested by MisterMyagi
 
Tim
@TheLittleNaruto Does it work with just from db_module import... rather than from src.db_module import?
 
@Tim Ok checking
Not working
 
Tim
Can you try running pytest from the command line (or some terminal external to PyCharm)?
cd to your tests directory and then just run pytest
 
It's working from commandline but not from the configuration we added
@Tim Just a question: I will have to add this in every test module ?
 
Tim
1:43 PM
Okay, just comment out the contents of conftest.py and run it from the commandline again to check that it then errors
 
It'll work right ?
 
Tim
It shouldn't
 
it'll because CommonUtils has import starting from "src"?
 
Tim
Okay, get rid of that as well.
 
Not working; same error: ModuleNotFoundError: No module named 'db_module'
 
Tim
1:46 PM
Yep, now uncomment the contents of conftest.py and it should work again
 
:O
How it worked ?
Configuration is still complaining though
 
Tim
So it's working in commandline, but not in PyCharm?
 
Yes
When we did this: sys.path.append("src") What it changed ?
 
probably because adding a relative path to sys.path isn't a great idea
pycharm likely uses a different working directory
 
Tim
Ah, yes.
 
1:50 PM
my brain ain't processing anymore. Could you help me to understand why it worked ? :)
 
Tim
Change the conftest.py line to sys.path.append(r"D:\Work\MySampleProject\src")
@TheLittleNaruto Yep, sorry, keep starting writing an explanation and then another message pops up...!
 
@Tim Yeah I did that when @Aran-Fey said using Relative path isn't a great idea.
Ok I'll start driving now. Please ping me the reason that why it worked.
later o/
 
Tim
@TheLittleNaruto Python imports don't just come from nowhere. Python can only import stuff if it's discoverable from the PYTHONPATH (environment variable), which I _think_ means its parent directory directory needs to be in PYTHONPATH.

When you start up the Python interpreter, various things get automatically added to the Python path, eg anything you've pip-installed into your Python environment will be there, and also the current working directory gets automatically added. The `sys.path.append` then adds the `src` directory, so the modules in there can be imported directly, ie `import db_
 
@MisterMiyagi thanks! I'll check that out
 
2:09 PM
technically, imports rely on sys.path, not PYTHONPATH. PYTHONPATH is just one of the things that let you customize sys.path.
 
can confirm, my PYTHONPATH is empty
 
I don't think I have one of those.
 
Cabbage all
Is the IEEE 754 binary16 struct representation "relatively" new (python ~3.5ish)?
Ah! There we go
It is. Thanks for your help all :)
 
3.6, apparently
 
Aye. Now I just need to figure out a workaround
Thanks :)
 
2:25 PM
Half precision?
 
Yep. Dealing with some legacy stuff where the size for the data is a mighty 16 bits
 
If the question is "In older versions of Python, how do I replicate the ability of 3.6+'s struct module to serialize/deserialize half-precision floats with the "e" specifier?" I'm half-tempted to write a polyfill, although it would probably fall apart on any computer with a different native endianness than mine
Where did I put my float exploder...
 
cabbage
 
_PyFloat_Unpack2 appears to be responsible for turning a packed half-precision float into a regular native Python float
If you can read C, porting it to Python doesn't look too painful
 
What is the tab command in Jupyter NB to render the symbol α - I thought it was \alpha + tab
But that doesn't work.
 
2:35 PM
@ReblochonMasque You mean like LaTeX? Isn't that rendered with $$. as in $\alpha$
 
I don't suppose \greek small letter alpha<tab> would do it?
 
no, not latex - rendering of greek letters in the code
 
ah unicode
 
@TheLittleNaruto wait no, don't alter sys.path in your conftest.py. just make your code installable and install it with pip install -e .
 
something like α = 2 for instance
 
2:37 PM
that's how i'd do it in VS Code. (Not that I normally use VS Code...)
 
And don't put an __init__.py file in your src directory, that defeats the whole purpose of having it
 
@Kevin Pretty much. I wrote an IEEE 754 converter around a decade ago. I'll be buggered if I can find out where I put that code. It was on my SVN repo all that time ago. I'm trying to save time / being lazy
 
FWIW I found my double dissecter at gist.github.com/kms70847/38f404708ec58c3cb955c69c4ffc428b, although I'm not so confident that it's actually useful here
 
ok, sorry - the sequence is indeed \alpha + tab - my kernel had died! (what is the salad word for blush?)
 
@OldTinfoil perfect for a huge amount of wrong data :P
@ReblochonMasque if you find yourself having difficulties writing some code: don't do it
 
2:44 PM
@TheLittleNaruto some info on packaging in general, and testing packages in particular: blog.ionelmc.ro/2014/05/25/python-packaging
 
bbiab - things are literally exploding
 
@AndrasDeak What if it is recommended to do for the style guide?
 
Every day is a struggle writing code @AndrasDeak
What do you mean?
 
@Dair change the style guide?
@ReblochonMasque unicode doesn't belong in variable names. Very slippery slope to swift from there.
 
ok, I agree - I use it only for specific specialized stuff in Jupyter..
 
2:47 PM
@AndrasDeak Laughs in Lean
 
I know, or APL, or whatever. My point stands.
 
I was born with ASCII - Unicode is superfluous! :D
 
In a similar vein, the only things that belong in a path are string.ascii_letters + string.digits + '.-_'
 
I know most of my contributions to python in codereview, and in that community I advocate PEP 8, but not gonna lie, math notation can pay off if you make sure to use it regularly.
 
you only have to make sure that everyone ever reading your code will be used to it
 
2:50 PM
Agreed on all counts @AndrasDeak
 
Hah. Good thing no one ever reads my code! (Proceeds to cry in a corner)
 
back
 
HA! it is all into your hands @Dair
 
UNLIMITED POWER!
 
I took a shot at porting _PyFloat_Unpack2 to pure Python. It's "works on my machine" certified. pastebin.com/NPRfBKhU
Offer void in nebraska. NaNs not guaranteed to be the correct kind of NaN
It occurs to me that I could have just looked at how Pypy does it. Oh well.
Pypy's float packing/unpacking is at bitbucket.org/pypy/pypy/src/… and bitbucket.org/pypy/pypy/src/… for the curious. Not sure how easy that would be to tear out for one's own purposes, since it calls a bunch of functions whose definitions I don't feel like looking up
 
3:16 PM
@OldTinfoil Is Numpy an option for you?
 
@Kevin Nice thanks. I'm trying to stop the ringing in my ears. I'll take a look when I've figured that out
@PM2Ring Aye, we've got numpy installed
 
Hello All, I'm currently using pandas to manipulate an xlxs file. I have a column where I need to pull out information and make a new dataframe. The issue is need to grab the delimiter in this case "oz" and any digits attached to "oz". Does anybody know a way to do this?
 
@OldTinfoil In that case, use it! Numpy can do 16 bit floats
 
@Aran-Fey Ok, but what happens for this - words[1::2] = [word[::-1] for word in words[1::2]]? Here also words[1::2] is a new different slice of words. Can yu tell me how it changes the entries in words?
 
That was my first attempt - using the float16 function, but the protocol can only handle integer representations - so I was planning to use struct to pack/unpack the data so that python passes it the unsigned 16 bit integer representation of the IEE 754 half precision floats
feels like he's becoming an XY problem
 
3:30 PM
@Quark Yes, words[1::2] is a slice, but you're assigning to that slice. In words[1::2][i] you're not doing that - you're assigning to an index in that slice
 
@OldTinfoil Yes. I'm a little confused. :) So you're trying to pass float16 data through a protocol that only understands int16. Is that right?
 
Correct
well.
Almost
 
It doesn't feel like an XY problem to me, although I'm not clear on what part of the problem can't be solved with some combination of numpy and struct
 
I'm conversing via modbus - the protocol can handle larger sizes, but each storage register is only 16 bits. I need to update a single register (not multiple) I have to therefore convert the value to be of the right size
Pymodbus' write_register() function uses struct to unpack the data, and it's expecting an integer. Pymodbus has useful utility functions for converting various types of data to 16/32/.. etc sizes - but doesn't have a float16 for some reason
The version of python that I have is 3.5.3 - so I can't use un/pack("e") on the end platform
One option available to me is cross-compiling a newer vsersion of python, but I'm trying to avoid doing that because I might need to update other devices in the field and I don't want to start distributing pre-compiled python binaries if possible
 
@OldTinfoil .view(np.int16), tadaaa it's an int now ;)
 
3:38 PM
err meh gerrd
 
3:54 PM
(fwiw @Kevin - I'm trying to use/adapt the unpack function from the pypy module you linked me to)
 
cbg all... I've figured out my problem
 
What was the problem/solution?
 
the problem is that I can't use a decorated function in multiprocessing/concurrent.futures.ProcessPoolExecutor. The solution? I haven't figured it out yet
I'm open to suggestions...
 
you cannot pickle them?
 
How difficult is it to undecorate them?
 
4:08 PM
@OldTinfoil would very much prefer to not. The undecoration would be far too difficult - not impossible, just hilariously difficult
@MisterMiyagi That does appear to be the case: AttributeError: Can't pickle local object 'contracts.<locals>.decorator.<locals>.wrapper'
 
you may want to try using dill
AFAIK importing it is enough to install the appropriate hooks
 
yeah, I thought about that, but here's why I thought it wouldn't work: well that shoots down my counterarguments
 
@OldTinfoil I'm curious what you're communicating with? All of our PLCs are on 32bit and it's quite old machinery.
 
so import dill did not fix this
 
4:27 PM
@roganjosh Not sure. I'm just going by documentation just now.
 
if you feel like it and your decorator is stable, you can modify __module__ and __qualname__ (__name__ for older protocols) to point to some place where the function will be eventually.
for example, copy __module__ and __qualname__ onto the wrapper, store the function on the wrapper and change its own __qualname__ to __qualname__ + '.__wrapped__'
plus, account for typos from typing on a train WIFI
 
Urgh. Hate numpy/scipy documentation - I find it to be both clear and obtuse at the same time. Trying to work out what views() actually does and whether I can trust it reliably as a struct pack/unpack replacement
 
oh apparently I need to import dill as pickle. Rerunning. Will re-report
 
@OldTinfoil Half serious suggestion: just test it exhaustively on every possible float16 value. There are only 65,536 of them, after all.
 
Sod it, why not?
Cheers Mr Kevinsson
 
4:34 PM
@OldTinfoil Quite often, views does almost nothing. It just changes the way the bit pattern is interpreted, so it can be extremely fast. But it's smart enough to know when it actually has to make a copy of stuff.
 
import dill as pickle does not work. Any other ideas? I'm dying here
 
From a C point of view, it's like using a union. You write a float16 to the union and read the bit pattern as if it were an int16.
 
I can confirm that ProcessPoolExecutor does not like to run decorated functions on my machine either. I'm unclear on why this is the case.
I'm going to guess... "closures"
Solution: boot up Wine and run your program in an environment that has actual fork() capabilities
 
apparently one fix is to set the decorated function's __module__ to "__main__", which is gonna be difficult for me to do
 
Oops, I got that backwards. Wine lets you run windows stuff in Linux. You need something that lets you run Linux stuff in Windows.
 
4:39 PM
@Kevin running on AWS, so that shouldn't be an issue
 
Windows 10 comes with a Linux environment. It's a shame that I find the rest of it to be a steaming pile of poop so it's not enough for me to upgrade
 
Oh, that's surprising. I figured forkable envs would have an easy time moving functions around without pickling them.
 
@inspectorG4dget you can set it to any place where pickle can find it. see above
pickle just looks for the object at __module__.__qualname__ - and you are free to put objects wherever you like and change __module__.__qualname__
 
@MisterMiyagi Whoa! That is some next level python hacking. Gonna try to implement this
 
4:56 PM
At some point it becomes more cost-effective to just run the program on a single core
 
5:12 PM
cbg all
 
cbg pmg recbg room6
When I comment something like this:
I write Please provide a [mcve]
But everything inside of me tells my fingers to type "Please provide an [mcve]"
It is a big effort to write a instead of an
 
What about just "Please provide [mcve]"?
 
I think that feels worse
 
Try to condition yourself to read the acronym as "muckvee"
 
^ that's a solution with legs
 
5:19 PM
There was a great first-time question posted the other day about pyparsing - included a pared-down parser that showed the problem, the input text being parsed, what they got and what the thought they should get instead, all in about 3 paragraphs. Puts them in the top 5% of all pyparsing questions right off the bat.
I have used dill in the past for pickling parse actions. But somehow this has vanished from my unit test suite.
 
I imagine that user was a "long time reader, first time asker"
 
Instead of pickling code, maybe you could pickle a callable class. Then import the class before unpickling. It seems I have done this to "compile" some sort of scripting language using skeleton classes without implemented code, then later unpickle with implemented classes imported so the the unpickled pieces could be "executed".
A gaping security hole to be sure, but if using pickle, I assume we are already past that
 
5:38 PM
I am still at the deep end of the ThreadPool. I think I have a generator-compatible version of my own, but I'm curious about other opinions from the room re: multiprocessing.pool.ThreadPool's treatment of generators as iterables. To recap: ThreadPool will, if given a generator as its iterable to process using the target function, will reify the generated values into a list, and then start parceling out entries to the various workers.
 
@Kevin I always think of John McVie.
 
Which is a major pain for me when I have a generator like Path.rglob('*') that will pull in many many file entries, when I would prefer to have TP just pull from the generator as needed.
 
Like how I think of Peter Noone when anyone forgets the space in "no one"
"Noone documents their code thoroughly". "Noone strives for 100% unit test coverage". This man is a champion among developers.
 
Florida Man of programming?
 
Some say... That Noone is as powerful as Jon Skeet.
 
5:42 PM
Noone says that anyone is as powerful as Jon Skeet
 
Noone is an island
 
Someone, somewhere, is called Noone and their life is a sitcom
 
 
1 hour later…
7:10 PM
@OldTinfoil as PM said, view typically just reinterprets your data. Probably not terribly helpful in your use case, but who knows. You might be happy to hear that there's a huge project going on where tech writers help revamp the numpy documentation. It's going to be big, from what I can tell, although it's still in the early stages.
 
Wake me when it's done
 
7:26 PM
Oooooh yam. I've been on and off looking for a bug in my code for months. Yesterday when I got to it again I realized that one of my input files were wrong. Rerun proves that the code was working fine all along...
 
@Andras that old chestnut hey... strangely pleasing yet frustrating at the same time :p
 
you said it
at least in the first days of looking for a bug I did find a relevant bug that needed fixing :'D
 
00:00 - 20:0020:00 - 22:00

« first day (3247 days earlier)      last day (1702 days later) »