« first day (3055 days earlier)      last day (2117 days later) » 
01:00 - 21:0021:00 - 00:00

01:52
docs.python.org seems to be down. Anyone know how long that's been going on?
02:09
It seems to be back.
Have seen a few various big services down today actually. Wonder if it's a certain cloud provider or something
 
4 hours later…
05:43
cbg
 
5 hours later…
10:23
list multiplication reference dupe stackoverflow.com/questions/54882206/…
^ closed
10:47
Thanks
11:03
cbg folks,
any idea why I'm getting this error:
  File "IntroDL8PP.py", line 71
    featureset.append(features)
             ^
SyntaxError: invalid syntax
from this code:
one good and dirty tip when trying to understand syntax errors is just running the code line by line if possible.
featureset = []
    labels = []
    counter = 0
    with open(fin, buffering=20000) as f:
        for line in f:
            try:
                features=list(eval(line.split('::')[0]))
                label=list(eval(line.split('::')[0])
                featureset.append(features)
                labels.append(label)
                counter+=1
            except:
                pass
    print(counter)
    featureset=np.array(featureset)
as for the error, the line above is missing a closing bracket.
I see lots of evals
i suppose thats tip number 2, most syntax errors are either on the line itself, or something left unclosed on the line before it.
11:07
good point, noted
oh gosh. uh, 1 eval is 1 too many. that too while reading a file and running an eval on it.
avoid evals. theres almost always a better way.
(evals open you up to a world of hurt, just 1 wrong command from a malicious source is all it takes)
first I'm trying to get the example code up and running and then I'll tweak it a bit since evals are so nasty
11:46
@Skyler nope nope nope
ast.literal_eval if anything
and you can use str.partition for an efficient single-split
also except: pass -> nope nope nope
Hey guys, I have a very basic query.

I have a text file full of C arrays, whose contents are hex values. They are of varying size in a text file and I need to store each of them into a Python list of sorts. The end goal is to convert each array into an image using matplotlib.

A small snippet of the text file: https://pastebin.com/MFg3ZVVF

What would be the way to go about this? The basic flow which I thought of was to read till a newline occurs and store the values in a list, but I couldn't figure out how to do that. Any help would be appreciated, thank you.
You're saving a textual representation of RGB image data? That's 18 bytes per pixel. That's a lot.
The textual data I obtained was from a pcap file. What I am essentially trying to do is convert pcap files into images.
read till an empty line or the end of the file instead of a newline maybe?
12:06
I'll try that, thank you.
Wait, so all you have is a dump of sequential RGB bytes? Do you even know the dimensions of the images?
Those are not RGB bytes, I think. I could be wrong. What I am trying to do is make grayscale images.

What I did was dump a PCAP as C arrays. That gave me a bunch of arrays with hex values. I used the first 100 values to make images of dimensions 13*10 as a trial and ended up with something like this:

https://imgur.com/a/jTBF9FY
I don't think you're going to have much success with that approach. PCAP files contain things like HTTP headers and such. Displaying HTTP data as an image doesn't make a whole lot of sense
A naive approach would be to split the file by 3 \ns then get rid of new lines within those, then parse it to get your byte values in a tuple, something like:
import ast

with open('your file') as fin:
    lists = [ast.literal_eval(el.replace('\n', '').strip()) for el in fin.read().split('\n\n\n')]
I filtered out the HTTP headers from Wireshark to only include the data. That'd be relevant, right?
12:16
oh, alright then
Thanks Jon, I'll check it out.
12:44
CBG
13:10
@coldspeed don't happen to know what the equivalent of .drop_duplicates() is but also returns summary info... eg.... I want to group by all columns and get a count...
 
1 hour later…
14:16
Design puzzle. From a third-party library, I have a class FancyCollection that represents a collection of objects. The class has a method, visit(self, callback). This method iterates over each object in the collection and calls callback(the_object). The class does not support any other way of accessing the objects. I would like to write a generator function, iterate_collection_objects, that yields each object in order.
I can do this fairly easily if I'm willing to use O(N) memory:
def iterate_collection_objects(my_fancy_collection):
    objects = []
    my_fancy_collection.visit(objects.append)
    for obj in objects:
        yield obj
But is it possible to do this without creating a list and waiting for visit to complete executing before I can yield the first value?
i dont understand callback.
so, it is something that takes a function?
maybe dumb question, but cant you write a function that takes 1 argument, and just yields it as is?
The intended type of callback is indeed function, if that's what you're asking
my_fancy_collection.visit(silly_func_yields_whatever_is_passed)
@Kevin is for obj in myfancy_collection.visit(lambda L: L) viable?
I am legally allowed to pass a generator function to visit, but it won't yield the objects to any context that I have access to
@JonClements I'm afraid not. visit itself returns None.
14:20
ahh... that makes a difference then...
class FancyCollection:
    #remember, you're not allowed to change anything about this class, because it's from a proprietary third party
    def __init__(self, items):
        self._items = items
    def visit(self, callback):
        for item in self._items:
            callback(item)

my_fancy_collection = FancyCollection([4, 8, 15, 16, 23, 42])
my_fancy_collection.visit(print)
Here's a small implementation for testing purposes
Here's a theoretically more space-efficient approach that uses threading:
import threading
from queue import Queue
def iterate_collection_objects(my_fancy_collection):
    items = Queue()
    done_signal = object()
    def populate_queue():
        def visitor(obj):
            items.put(obj)
        my_fancy_collection.visit(visitor)
        items.put(done_signal)
    t = threading.Thread(target=populate_queue)
    t.start()
    while True:
        item = items.get()
        if item is done_signal:
            break
        yield item
If the main thread consumes objects as fast as the child thread visits them, then this uses O(1) memory. But if the main thread consumes objects slowly, it's worst-case O(N) memory.
I suppose I could set up some Locks to ensure that the child thread only executes while items.get() is being executed in the main thread. But I already don't like this solution and I don't want to spend too much time putting polishing a turd
14:39
@Kevin can you do something with a generator and send'ing into it maybe?
That's what I'm playing around with now. visit is the immovable object here that I need to route around, because it is not a generator and does not play nicely with generators
@Kevin visit presumably does something more complicated than just iterate over something and callback though? I mean - it iterates in some pattern that isn't straight iteration.... ?
Yeah. The real FancyCollection is more like a directed graph than a list.
stackoverflow.com/questions/9968592/… is quite close to my problem. Two of the answers suggest threading/Queue and one answer suggests list.append. The "Generator as coroutine" answer I think doesn't satisfy my requirements because the values are only accessible inside process_chunks and can't be yielded to main or main's caller
kudos to brice for suggesting Queue's maxsize=1 though, that would fix the threading approach's worst-case behavior without me having to write any Locks
process_chunks is interesting in that values created during one callback can be accessed during later callbacks. That's a fun trick. But it's not instrumental towards my goal.
15:25
I think my dream goal, yielding from callback to iterate_collection_objects while skipping over and/or suspending the execution of visit, is impossible.
yield has much flow-control magic, but I require finer control over it than what the language provides
@Kevin wonder if you have a blocking put then you can do something similar to docs.python.org/3/library/asyncio-queue.html#examples - but not sure that's much different than your Queue example...
If there's a solution that uses async, I would consider that distinct from a threading-and-queue-based solution.
15:47
cbg! My goal is to super quickly check if now is more than 0.5 seconds away from "2014-11-07T08:19:27.028459Z". I must convert this to datetime/pd.Timestamp first I think. Which one would perform this check faster?
@isquared-KeepitReal UNIX timestamp comparisons are integer comparisons, that makes them about as fast as is possible. (I can't speak to pd.Timestamp implementation details, though.)
@amcgregor so I can just check two str UNIX timestamps with < or >?
@amcgregor but what I said above would not tell me if it is 0.5 seconds away
ahh. Strptime and difference?
import time
time.time()
# wait a moment
delta = time.time() - _
delta
# 49.81281…
@isquared-KeepitReal timestamp() and difference
Notably, actual integer difference, not string comparison or anything like that. String comparison might appear to work, but only if they're of the same length, due to the lexicographical sorting behaviours of ASCII.
"3" > "10"  # True, because 3 > 1.
15:54
IIRC ISO-compliant datetime strings are specifically designed so lexicographic comparison is the same as numerical comparison. Not that this is useful for determining whether two datetime strings are within half a second of one another.
Se so. ISO is good people. But doesn't get the desired answer of the difference in time.
nope, it does not
so, which solution is better, timestamp() (pandas?) or time()?
I was more warning about the difference between, say, "1551196249" (roughly now) and "951196249" (2000 and earlier, one digit shorter).
@isquared-KeepitReal I always prefer built-in native solutions, where possible. time is the module for handling UNIX timestamps.
Perhaps vaultah is referring to docs.python.org/3/library/…
Or, hmm, maybe not, since that doesn't seem all that useful in this context
time it is then
Melon!
15:59
Already in progress… >:D
In [1]: import time
In [2]: import datetime
In [3]: then = 951196249
In [4]: now = 1551196249

In [5]: %timeit (now - then) > 0.5
117 ns ± 4.24 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

In [6]: %timeit (datetime.datetime.fromtimestamp(now) - datetime.datetime.fromtimestamp(then)) > datetime.timedelta(seconds=0.5)
2.4 µs ± 208 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
So, uh, using datetime construction, and timedelta comparison is ~50x slower than just using the integers. (Remembering that they'll need to be floats to get sub-second accuracy.) ;^P
@amcgregor beautiful! built-in, home-grown, organic!
Eco-friendly and green, too, given the reduction in CPU cycles.
laurel haha
need to convert that str into float first though
Let me add this result to my big-ass'd collection of timeit tests. :D (I'll clean it up and match the other single-line invocations later.)
:not-bad: slack emoji!
16:05
Originally wrote that when I was hacking on my own pure-Python C10K capable HTTP/1.1 server. Some differences are weird, like split with a limit actually slowing split down, and partition just trumping split completely.
I had to Google to understand what that means lol
Or in a Python REPL: help(str.split) and help(str.partition) :D
(See also mentions of split/partition in the timeit test results, there.)
no, that part is OK :) the C10K thing
Ah, yes. 10,000 simultaneous connections. (my attempt back in the day succeeded in not failing a single client request, but could only actually serve ~6K/second.)
(Silly naive kernel-level incoming socket allocation pools.)
I have an array values = np.array([12,56,92,114]). I'd like to reduce the array to the two values that are closest to the value b, lets say b = 60. Is there an easy way to do that? I can only think of convoluted loops
So final_values = np.array([56, 92]) would be the desired output here
16:15
If you had asked the same question about a regular list, I'd say:
>>> seq = [12,56,92,114]
>>> import heapq
>>> heapq.nsmallest(2, seq, lambda x: abs(x-60))
[56, 92]
Knowing nothing about np arrays, I can't say whether there's a better approach
What I can come up with is subtracting b, sorting according to absolute value, selecting the lowest 2, and adding b again, but that seems convoluted
I'll have a look at heapq though, thanks!
@user129412 there's also stackoverflow.com/a/18691983 and a pandas.Series has nlargest/nsmallest methods...
Note that the order of the two numbers in the output will not necessarily be in the same order they were in in the original list, and they won't necessarily be ordered smallest-to-largest
>>> heapq.nsmallest(2, [1, 50, 61, 99], lambda x: abs(x-60))
[61, 50]
Yeah, that'll definitely work. I'll look for a bit if I can resolve it with just numpy, but efficiency isn't key here, so not too big of a deal :)
16:32
Here's a nice change of pace. Instead of my project refusing to build even though it reports 0 errors, now it's successfully building even though half the lines in this file are marked in red.
@isquared-KeepitReal Interesting weird note related to benchmarking things as high-performance as C10K… BSD socket exhaustion. (After closing a connection, the port isn't immediately made available, because there may still be packets in-flight that need to be rejected upon arrival, so it's put into CLOSE_AWAIT. At 10,000 requests made per second, that could exhaust all available port numbers in ~5 seconds.)
Best-case scenario, this code file is being used nowhere else in the project, and its myriad errors don't matter because nothing depends on it.
cbg all
@amcgregor I will keep that in mind :)
I just looked at time.strptime, it looks like it does not support the microseconds
@isquared-KeepitReal %f, no?
16:43
import time
latest_time = '2018-03-22T21:11:11'
time.strptime(latest_time, '%Y-%m-%dT%H:%M:%S')

works fine
let me check
@isquared-KeepitReal rgruet.free.fr/PQR27/PQR2.7.html#timeModule — 2.7, but these format strings are unlikely to change for 3. ;P
don't see %f in their docs
ahh 2.7
it did not give me the errors :) I guess that's a good sin
I don't it takes account of it
@isquared-KeepitReal Huh, seems not? stackoverflow.com/a/531220/211827 is reasonably not-inelegant. ;)
(That is: it isn't pretty, but it's not the ugliest thing in the world.)
It may be easier to just create a datetime object directly, rather than create a string and convert it to a datetime object
@amcgregor ;( oh well. But how can you refuse an answer of a person who has a Mandelbrot set as their avatar
16:52
According to [this](https://stackoverflow.com/a/698279/211827) it sounds like it ought to. :/

time.strptime('30/03/09 16:31:32.123', '%d/%m/%y %H:%M:%S.%f')
latest_time = datetime.datetime(2018, 3, 22, 21, 11, 11). Something like that.
So this:

import time
import calendar

latest_time = '2018-03-21T21:11:11.000Z'
a = time.strptime(latest_time, '%Y-%m-%dT%H:%M:%S.%fZ')
new_latest_time = '2018-03-21T21:11:11.5Z'
b = time.strptime(new_latest_time, '%Y-%m-%dT%H:%M:%S.%fZ')
float(calendar.timegm(b)) - float(calendar.timegm(a))
returns 0.0
Or even just latest_time = 1521767471.0. Depends on how much you value clarity.
@Kevin we have string to begin with
@amcgregor Python 2.6. I am on 3.6
What I'm hearing is "in my example code I'm creating latest_time from a literal, but in my actual code I'm getting it from an external source, so I can't convert it by hand". In that case, strptime is fine.
16:56
@Kevin correct. But, I am getting back 0.0 as in example above. It looks like it ignores the microseconds. And when you look at time structs that get created (a and b above), they do not have microseconds either
Hmm. I know that some OSes don't have mircosecond-accurate resolution when you call time.time(), but I wouldn't expect that to affect datetime arithmetic.
oh, you meant datetime. I was hoping to do it with time. But it looks like I cannot. I am on macOS
maybe it is because I am not padding my microseconds with zeros to the right in the above example
nope
Sorry, I was only half-reading your code and didn't notice that you were using time and not datetime. Pretty much everything I've said so far applies to both, though.
struct_time certainly appears to not support microseconds.
17:04
So I don't think you can get time.strptime to do what you want, regardless of how you adjust the format string and/or input string
@Kevin agreed
@Kevin I will try datetime.strptime
Had to read that again... Perhaps stack and then group on the index and call value_counts?
I'm optimistic that datetime.strptime will work better
lol, this does not look promising: datetime.datetime(2018, 3, 21, 21, 11, 11)
yes! works
I guess this is as fast solution as is possible since time is not supporting the microseconds
need to time it
19.9 µs ± 1.17 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
not too bad!
thx Kevin
Sounds like you've got it, but here's what I had in mind for the record
import datetime
latest_time =     datetime.datetime.strptime('2018-03-21T21:11:11.000Z', '%Y-%m-%dT%H:%M:%S.%fZ')
new_latest_time = datetime.datetime.strptime('2018-03-21T21:11:11.5Z',   '%Y-%m-%dT%H:%M:%S.%fZ')
print(new_latest_time - latest_time)
print(new_latest_time - latest_time >= datetime.timedelta(microseconds=5))
17:11
starred. Thx
@coldspeed so given:
df = pd.DataFrame({
    'A': [1, 0, 1],
    'B': [0, 0, 0],
    'C': [0, 1, 0],
    'D': [1, 0, 1]
})
Running .drop_duplicates on that gives me:
   A  B  C  D
0  1  0  0  1
1  0  0  1  0
I want that first to have a new column of "2" because there were that many rows which were a combinatino of 1, 0, 0, 1.... and the other row have a count of 1...
Umm... does nunique take an axis...
m8_
m8_
Afternoon! I'm trying to prepend a str to values in a column that are NOT in a list. Trying df['col'] = df[~df['col'].isin(list)].apply(lambda x: x.str.cat('FIX – ', x)) but says type object 'str' as no attribute 'cat'`
inds = ~df['col'].isin(list)
df.col[inds] = 'FIX - ' + df.col[inds]
does that not work? ^
hm, I was actually told once to star if it is useful. Now I am asked not to star lol
17:22
> SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame
guess that's not a good idea
@isquared-KeepitReal useful or interesting to a broader audience
if you look at the starboard you'll see the kind of things that get starred and stay that way
@JonClements df.groupby([*df], sort=False).size().reset_index(name='Count')?
or df.assign(Count=df.groupby([*df]).A.transform('size'))
m8_
m8_
That did the trick, thanks @AndrasDeak! Quick question though, what if my header has a space in the middle? df.Type One[inds]? Do I have to change my header name or is there a workaround?
@JonClements or more complicated...
i, r = pd.factorize([*zip(*map(df.get, df))])
df.assign(Count=np.bincount(i)[i])
@piRSquared humph.... could have sworn I tried that....
17:27
cbg
@piRSquared cbg :)!
cbg all, cbg @isquared-KeepitReal
and now we have two squares in the room
@piRSquared that's what I was going for... not sure what I did when I thought I tried that... sighs
I'm guessing you tried df.groupby([*df]).transform('size') without referencing one particular column after the groupby
17:29
@Code-Apprentice his is a bit different though laurel
@m8_ df.col and df['col'] are interchangeable, the latter is more general and applicable if the column name is not a valid identifier name
but don't do what I wrote; you should get a warning about setting a value on a copy of a slice. Even if it works, don't do it and read the docs pandas.pydata.org/pandas-docs/stable/…
m8_
m8_
Ok gotcha. Can't I just make a copy of the df and then perform the code?
that's probably not the point but I don't know pandas very well, so read the docs and see what it tells you
m8_
m8_
Ok thanks
@m8_ when a new column is added to a dataframe, Pandas monkey patches an attribute (or hacks it via __getattribute__, either way doesn't matter) with the same name as the column as long as that column name satisfies a bunch of conditions. I would consider accessing columns via the dot operator a convenience and not depend on it in general.
m8_
m8_
17:38
so the space won't throw it off?
"Col One" will not qualify to access via the dot operator. df['Col One'] is fine df.Col One is not even valid Python.
@piRSquared they want to access a column that has a space in it. You somehow convinced them of the opposite of what they need as an answer :P
df['Col One'] is the way to go
m8_
m8_
lol, I was just asking for clarification. I am using df['Col One']
thanks!
There's some interestingly weird behaviour going on from what I'm seeing of Python's object model.

class Xyzzy: ...

a = Xyzzy()
a.foo = 27
setattr(a, "bar baz", 42)
dir(a)
# [..., 'foo', 'bar baz']
getattr(a, "bar baz") == a.__dict__['bar baz'] == 42

That's weirdly JS-like. Inb4 `[object Object]` gets used as a database table name. ;)
17:46
Yeah I don't think the object model actually cares if your attribute is a vaild identifier. Spaces, reserved words, unicode emojis, whatever, just stick em in the dict.
I don't see the "weird behaviour"
It sets the attribute :P
I'd almost expect setattr to enforce accessible attributes. setattr('foo bar', ...) is paradoxical as foo bar can't be an attribute via attribute dereferencing notation.
@amcgregor inb4: you can put anything in globals()
The behavior seems reasonable to me but I would also consider it reasonable if setattr crashed with a "that's not a valid identifier" error
I'd hope for at least a warning. ;)
17:49
Python's footguns are as silent as they are deadly
Or awesome.
Our footguns are the best. Everybody knows this.
class Popeye:
    _inst = None

    def __init__(self):
        if self._inst is None:
            Popeye._inst = self.__dict__
            return

        self.__dict__ = self._inst
I am Popeye of Borg; you will be askimilgrated.
(There's a reason I like to quip to developers in other languages that Python is "dicts all the way down". ;)
17:52
I seem to recall someone once suggesting that d = {'a': 1, 'b': 2, 'multi word key': 3} used in some_function(**d) shouldn't be allowed...
Generally, I don't want my perfectly cromulent code to get slowed down by an extra cycle in order to prevent silly developers from doing silly things
wim
wim
^ that
no convincing enough reason to forbid it, extra work to do so
On the gripping hand, Pypy JIT guards. (Alice's Law #56: To go faster, slow down.)
On the other hand, when I do something silly, I would like an instructive error message. Therefore the devs should detect exactly and only the dumb mistakes that I personally am likely to make.
@Kevin something like: "Nuclear arsenal successfully launched. Hope Armageddon works out for you!"
18:02
@JonClements The essential problem of consultation contracting: giving the client what they need, not what they think they want.
> Arthur Dent: What happens if I press this button?
Ford Prefect: I wouldn't-
Arthur Dent: Oh.
Ford Prefect: What happened?
Arthur Dent: A sign lit up, saying 'Please do not press this button again.'
Oh.... I don't mind that... I make it very clear in writing what they think they need isn't and spec. out what they want... if they're not going to listen to my advice then I just do what they say they want - get paid for it - they then realise a little bit later I was correct to start with and they pay me again for the work they do need :)
I like my "you're doing it wrong" errors (warnings, exceptions, etc.) to not just describe what went wrong, but to also point out how to fix it wherever possible. And I unit test those messages to verify they mention both what it was, and how to do.
Whoa! dicts with only strings as keys are faster?
I think I'll look in the source to see this string optimization for myself. Most likely I will get hopelessly lost and give up~
github.com/python/cpython/blob/master/Objects/dictobject.c#L62 at least confirms that the optimization is still present in modern builds
18:12
@amcgregor I couldn't get all the way through alice's laws. I'll have to pick up where I left off another time (-:
@piRSquared There are a lot of 'em, I'll admit. Semi-organized into loose groups (personal, programming, paramilitary, business). ^_^;
18:35
cbg
@Kevin I see no issue with having modes that don't run said helpers - we already can run optimised disregarding asserts (and I assume other less publicised things)
@toonarmycaptain if __debug__ elision is a feature I make use of extensively. E.g. I wrap all calls to any_logger.debug in that, so in production the lines are removed from the codebase entirely. Additionally, my web framework will only verify the endpoint arguments can be supplied to the endpoint in development. (404 if mismatched in development with a helpful enumeration of available choices, 500 in production due to the uncaught TypeError.)
Now that there's an official "development mode" flag, those types of "expensive" checks have a place to live. \o/
wim
wim
@piRSquared because they are used for namespaces
@amcgregor There I go again, Kevin'd by core-devs.
@piRSquared Strings-as-keys also seriously benefit from string interning. That is, string equality checks can be pointer comparisons. They are both hashable, and immutable. Very convenient.
I'll keep this in mind. I've assumed that all dicts were made equal. Now I know that some dicts are faster than others.
wim
wim
18:49
no change to complexity, only the constant factor
At one PyCon (2016?) I believe I overheard a core dev: "If we made dictionaries any faster it'd open up a wormhole and the universe would implode."
wim
wim
it's kind of weird - a string optimized key lookup function is used up until you check for a non-string key. when that happens, it permanently changes the lookup function to a slower and more generic function
Indeed; I approve of "guards". Assume fast, until it's not, then go safely forwards. :D
I would like to point out that string_a == string_b does not necessarily imply string_a is string_b
@wim that is good to know as well. Otherwise I'd assume my lookup was wormhole worthy when it wasn't
18:53
@Kevin An important note. There are cases where strings aren't interned, such as strings that are the result of "complex" calculations.
I suspect that the average dict key string tends to be short enough to be interned, so it's still entirely possible that lookup-via-id may hit more often than it misses
''.join(['f']) is ''.join(['f']) # True; single-character strings are special cased.
''.join(['f', 'o', 'o']) is ''.join(['f', 'o', 'o']) # False; too complex.
'a' * 20 is 'aaaaaaaaaaaaaaaaaaaa' # True
'a' * 21 is 'aaaaaaaaaaaaaaaaaaaaa' # False
'foooooooooooooooooooooooooooooo' is 'foooooooooooooooooooooooooooooo'  # True because literals.
@Kevin I suspect it's less about length for most dictionary keys. Most would, I hope, be defined as literal constants in the source, and those are collected and interned on module import as a whole, AFIK.
19:49
you guys are too funny
20:12
On typo-laden posts, I often comment "you almost never want to have an unconditional return inside a loop". The "almost" is a CYA measure, but I wonder if there ever is a time you do want an unconditional return in a loop.
for foo in bar:
    baz()
    return qux
Exercise: come up with a justification for this code.
can qux be set somewhere in buz? Like if it is global
Sure.
im having trouble with a bash script getting a called process error
this is hard. Why would you want to loop through foos in bar if the very first thing you do after baz() is return. hmm
could be an elaborate debugging scheme perhaps
20:15
anyone seen this error before `/usr/local/lib/python3.6/dist-packages/google/colab/_system_commands.py in check_returncode(self)
136 if self.returncode:
137 raise subprocess.CalledProcessError(
--> 138 returncode=self.returncode, cmd=self.args, output=self.output)
139
140 def _repr_pretty_(self, p, cycle): # pylint:disable=unused-argument

CalledProcessError: Command 'dataAugment () {`
like running a loop with a break just to test one iteration
except with a return
good point
@AK0101 that's not a full traceback
@AK0101 context? Any chance that the cmd to be executed was that 'dataAugment () {' string?
`CalledProcessError: Command 'dataAugment () {
image="$@"
target=$(basename "$@" | cut -c 1-200) # avoid issues with filenames so long that they can't be appended to
suffix="png"
# nice convert -flop "$image" "$target".flipped."$suffix"
nice convert -background black -deskew 50 "$image" "$target".deskew."$suffix"
nice convert -fill red -colorize 3% "$image" "$target".red."$suffix"
nice convert -fill orange -colorize 3% "$image" "$target".orange."$suffix"
what were you running, where did that error come from, etc?
20:18
I am running the dataAugment script to augment a directory of images
And that dataAugment script is a bash function?
i think so
does it work when run directly from bash?
I am running it on google colab with %%shell
in the cloud
python 3.7
it doesnt work currently
Where did the bash script come from?
you have to debug the bash script first before you expect python to be able to execute it
20:20
it was from /r/machinelearning one second Ill get the link
I don't need a link, it's bash
you'll have to fix it yourself
bash is outside of our area of expertise, alas
if bash is the problem, then a different chat room will be more helpful for you
ok ill take a look there thanks
just wondering if its a vm problem as google colab run on python 3.7
notebooks
and all other code is in python
20:23
maybe it's a permission thing and you cannot execute bash there
wim
wim
def first(iterable):
    for element in iterable:
        return element
    raise ValueError
I think you can use %%shell stackoverflow.com/questions/52343308/…
at the top of each cell
I would expect a jupyter %%shell to be as good as running in the shell. If you're not sure the shell script works in raw shell then there's no point in trying to debug python. if you run the script in your shell and it works but it doesn't on colab, we can try figuring out the issue.
@wim Not bad, I can almost imagine seriously using this in a code base.
iterables need __iter__, but do they all need __getitem__ as well, or can it be omitted?
20:27
think generator
msg from the future
I might be inclined to do return next(iter(iterable)) instead although that raises a StopIteration not a ValueError. But I am perpetually ambivalent about exception typing.
@Kevin if it excepts like a duck...
20:29
then i'd just return a duck!
ha, beat the system.
@JonClements You have probably already received satisfactory answers from piR and the others. I would have done exactly what was suggested, df.groupby(df.columns.tolist()).size()
wim
wim
ValueError chosen to be consistent with max([])
@AndrasDeak cool hack
I occasionally see people warning that file should not be used as a variable name. file was a built-in variable in 2.7, but it isn't in 3.X. Should we continue to warn people for the sake of version intercompatibility? Should we stop warning people on Jan 1 2020? Should we have never warned people in the first place because the kind of person that would overshadow file wouldn't be doing anything fancy enough to require the built-in file?
Personally I've used file as a variable for years, even though the spectral shame dog appears before me every time I do
I remember someone saying that they use file on purpose. Probably Antti.
Or me, disguised with a mustache
Kevin and his alter egos, Kelvin/Calvin/Caoimhín, all recommend file
20:44
those three guys happen to fit in one big trench coat
Yes, and it triples their programming power for as long as they can retain their balance. Which is usually about eight seconds.
        bids, asks = {}, {}

        for open_position in self.open_orders:
            if open_position.side == 'buy':
                bids[open_position.price] = open_position.order_id
            else:
                asks[open_position.price] = open_position.order_id
is there a better way to do something like the above
bucket = bids if open_position.side == 'buy' else asks
bucket[open_position.price] = open_position.order_id
perhaps
Seems decent enough to me, why bother changing it.
wim
wim
stomping the name file is OK. because the docstring for file says not to use it anyway.
20:52
@andras I was just typing that up. Now let's debate about whether it should be a ternary or a complete conditional statement.
ive picked my side already :P
I'm OK with either :P
bids, asks = {}, {}
for open_position in self.open_orders:
    if open_position.side == 'buy':
        bucket = bids
    else:
        bucket = asks
    bucket[open_position.price] = open_position.order_id
For comparison.
bucket
I like this better lol
I'm not enormously fond of creating variables inside a conditional even though rationally I know that the scoping works out fine
wim
wim
20:55
@AndrasDeak bad for coverage.py
I believe you
wim
wim
better to have an explicit branch there for the coverage report
plus more lines in the commit means more work well done
Third possibility: instead of bids, asks = {}, {}, do buckets = {"buy": {}, "sell": {}} and then buckets[open_position.side][open_position.price] = open_position.order_id
just add a jupyter notebook. Even better
20:58
Of course this changes what happens when the side is neither buy nor sell
that is not possible
that is a very good recommendation! Thanks
Anything is possible... Except for turning a visitor callback method into an iterator without using threading
01:00 - 21:0021:00 - 00:00

« first day (3055 days earlier)      last day (2117 days later) »