« first day (3989 days earlier)      last day (942 days later) » 

1:47 AM
cabbage
 
 
4 hours later…
5:46 AM
Anybody know how to shut up mypy on a ` error: Value of type "Union[Iterable[Any], Any]" is not indexable` ?
 
6:41 AM
Hi I've written code to search coordinates (geo reverse coding). I tried scatter plot to show case these coordinates in a world map. It didn't work. Does anyone else know a way to showcase coordinates in a wold map. It would be great, if it shown as regions.
Following is the code that I've written so far
import reverse_geocoder as rg

for i in range(len(data)-2):
coordinates = (data['LocationLatitude'][i+2], data['LocationLongitude'][i+2])
print(coordinates)
results = rg.search(coordinates)
print(results)
 
7:16 AM
@Mikhail take a look at the definition of Iterable in collections.abc. There's no __getitem__ method, meaning iterables aren't necessarily indexable. If you know that this object has definitely got a __getitem__ method, then Iterable is the wrong annotation for this object. If you don't know that this object definitely has a __getitem__ method, then Mypy raises a fair point.
Definition of collections.abc.Iterable is here in the source code: github.com/python/cpython/blob/…
 
 
2 hours later…
9:35 AM
cbg
Just entered the UK after two years ^^
 
9:48 AM
cbg folks
 
cbg
 
in other python news, have you all heard about discord.py shutting down? Apparently it's a very big python usecase, and the discord bot community is in a state of panic.
 
@ParitoshSingh is discord.py a library?
 
aye, the so called "go-to default" for building discord bots in python
 
I always see a bunch of discord bot posts on main
@ParitoshSingh huh, you don't usually see major libraries just pull the plug. I assume there's some drama involved...
 
9:51 AM
@ParitoshSingh Yam, just starting farming on that tag D:
 
@AndrasDeak yeah, the tl;dr is that apparently discord has made a lot of breaking changes to their APIs and imposed restrictions on the bots, and has generally not been very developer friendly. the guy maintaining discord.py was tired and couldn't find a successor he trusted to maintain the repo. i believe this is the link but.. apparently gist has been blocked as well here, yam
 
Is there a canonical for wrongly-indented methods?
 
@MisterMiyagi probably
 
9:58 AM
@ParitoshSingh yup, that gist looks good, thanks
 
In the Q I am looking at the indent is syntactically correct, but the method is local to __init__.
 
oh
that's going to be tricky to find, considering the other more common indentation errors.
 
typo?
 
yeah i'd say dont bother duping it, explain and typo sounds fairly reasonable
 
 
2 hours later…
12:46 PM
map(lambda num: num ** 3, filter(lambda num: num % 3 == 0, range(1, 21)))
How is this evaluated? Does range propogate values to filter, and then to map, 1 item at a time?
 
That's one way to think about it, sure
 
So the whole line is executed 20 items basically? Or does range fully evaluate first, and returns the whole iterable (20 items) to filter, which then evaluates and returns a 20 item iterable to map etc
 
Alternative: range returns an iterable. filter takes an iterable as input and returns a new iterable. map also takes an iterable as input.
@AstralWolf No. The filter applies its lambda function to 20 integers, and then the map applies its lambda function to the remaining ones
 
Just to clarify, does filter receive the 20 integers from range at once, or does it receive one integer at a time
 
Umm... filter receives a range object as input. It then iterates over the integers in that range 1 by 1, and only on demand (when someone else iterates over the returned filter object)
Try this:
def filter_func(num):
    print('filter:', num)
    return num % 3 == 0

def map_func(num):
    print('map:', num)
    return num ** 3

for num in map(map_func, filter(filter_func, range(1, 21))):
    print('for loop:', num)
 
12:58 PM
"Stay a while and listen. About higher order functions and coroutines."
@AstralWolf Have you toyed around with generator functions yet?
 
```
path("<slug:url>", ProductDetailView.as_view(), name="product_detail")
return reverse("product:product_detail", kwargs={"url": self.product.url})
```
NoReverseMatch at /nesiojami-kompiuteriai-priedai/
Reverse for 'product_detail' with keyword arguments '{'url': '/nesiojami-kompiuteriai/nesiojami-kompiuteriai/nesiojamas-kompiuteris-acer-sf314-14-i5-8265-8256gb--9618707.html'}' not found. 1 pattern(s) tried: ['product/(?P<url>[-a-zA-Z0-9_]+)/$']
why it doesn't match it?
 
Why would it match it?
 
cuz the model itself has url field
and you can filter by it easily
 
Does it matter that "product/" doesn't appear anywhere in the url? I'm not familiar with whatever library/framework you're using
If you can write an MCVE that just uses regular regex, I'd be happy to investigate further
 
1:14 PM
I'm using Django, basically we have few apps, so this url file is just a part from the root
but when I switch the fields from url to pk, in the path and reverse itself it does work.
 
I also notice that the character set contains letters and hyphens and so on, but not "."
 
In [3]: Product.objects.filter(url='/nesiojami-kompiuteriai/nesiojami-kompiuteriai/nesiojamas-kompiuteris-acer-sf314-14-i5-8265-8256gb--9618707.html')
Out[3]: <QuerySet [<Product: Nešiojamas kompiuteris Acer SF314 14" i5-8265 8/256GB MX150 W10>]>
 
it's alright because the string ends in a literal slash :P
 
Hmm, yep, this is definitely outside of my area of expertise (such as it is).
 
Thank you Aran, took a while but I got it
 
1:18 PM
idk why, but the url setup seems correct, because you can specify the type of the value. In this case its a slug, so we use slug:url, the url itself is the model field, but the model field is TextField, maybe that's why?
 
just for the record: I'm not the only one who has no idea what these code snippets mean, am I?
 
nope
 
@user9745220 Please take a moment to put yourself into the shoes of someone who has no idea what you are working on, nor what you are working with. Do you think the information you have provided would be enough for them?
 
Another question: how wrong would these statements be?
 
I could tell it's a django question because we get these all the time :P
 
1:20 PM
@MisterMiyagi I see two string literals, and one of them follows valid regex syntax, and the other one is almost a valid match for that regex. All the other code and technical talk is just noise to me.
 
yeah Aran, it is :D
 
Everything about TextFields and slugs and Querysets, I'm basically ignoring
 
1. An iterator is an iterable that implements __ next__. 2. An iterable is an object that returns an iterable that implements __ next__
 
@Aran-Fey I reserve the right to be educationally stupid. :P
 
@AstralWolf Consult tinyurl.com/urnzp7k for tips on keeping the markup engine from turning __next__ into next
TLDR: backticks
 
1:23 PM
@AstralWolf "An iterable is an object that returns an iterable"? How does that work? Does that mean an iterable can be called like a function?
 
Hm, good point. :/
 
I sense generator functions sticking in between some gears
 
Also, while "An iterator is an iterable that implements __next__" is indeed correct, that's only one of a few things that are necessary for an iterator to be an iterator. An iterator must also return itself from its __iter__ method, among other things
 
@Aran-Fey Perhaps they got that from docs.python.org/3/reference/datamodel.html#object.__iter__. "Iterator objects also need to implement [__iter__]; they are required to return themselves."
Oops beaten
 
Alright, thanks for the clarification folks
 
1:32 PM
Admittedly, things generally don't break if you mess up Iterator.__iter__
 
I almost never define custom iterators, and I'm thankful I don't have to fiddle with __iter__ definitions. yield is sufficient for me 99% of the time
 
1:51 PM
I have another one, how does an iterator maintain its state? Is that behaviour implemented in its __ next__ method?
 
it's implemented in the iterator.
some iterators don't have state at all.
 
Like, how does an iterator know that it is currently at the 2nd element of its iterable
 
Perhaps it maintains a self.index value
@MisterMiyagi I believe you, but my imagination is failing me. Do you have an example?
I vaguely suspect network I/O is involved
 
Nah, you can have a __next__ that just returns random.random(), for example.
Or a constant, for the functional crowd in the room.
 
Those would be examples of iterators without state right?
 
1:56 PM
class StatelessIterator:
    def __iter__(self): return self
    def __next__(self): return 5
 
Ah, true. I suppose itertools.repeat could also be stateless, when you call it without a times argument
 
If you ask "But where does it store self?", please help yourself to your own cookie.
 
All selves are stored within the Platonic Realm of Pure Forms
 
Hi all, I'm using the tsai library and I don't know how to change the # outputs of the Fully Connected layer from 2 (default) to 3 (# classes I have). If anyone can help, that would be great.
I checked the documentation, but I'm still confused: timeseriesai.github.io/tsai/models.FCN.html
If anyone could help, that would be great.
 
2:14 PM
Am I undercaffeinated, or is the example code on that page invalid syntax
    (convblock1): ConvBlock(
                ^
SyntaxError: invalid syntax
Ok, it's not just me then
 
^ Well, the thing is the FCN can classify -- it's just not that accurate, as you can imagine.
In fact, the maximum accuracy it can get is 2/3 = 66%, because it only outputs 2 classes.
 
Have you been following a tutorial or anything? I may be more useful if I can get some context
 
And what do you know? After 1000 epochs, the accuracy settles off at 66%
 
"what do you know?" -- I know very little, typically :-P
 
No, I was just looking at the documentation, but I'll keep trying and if I still can't understand, I'll return. Thanks @Kevin
 
2:28 PM
The Internet tells me that you can hook up multiple binary classifiers in order to create a multiclass classifier. For example, suppose you wish to classify a rubber ball as either red, blue, or green. First create a binary classifier that classifies a ball as either "red", or "not red". Then take all not red balls, and feed them into a binary classifier that classifies balls as "green" or "not green".
Now you have a bucket of red balls, and a bucket of green balls, and a bucket of not-red not-green balls, and hopefully all of those are blue.
 
Plot twist: the ball is yellow!
 
The truly diligent might add a third classifier, whose "blue" output goes into the third bucket, and whose "not blue" output goes into the incinerator
 
I preferred "pink" given that it doesn't exist... You're giving AI a free pass by changing to "yellow"
 
I don't always put our AI overlords into a categorial dilemma, but when I do I avoid cute colours.
 
Most colors don't exist when you get right down to it
 
2:35 PM
Plus, I suck at colour mixing. :(
Real life twist: the ball is transparent!
 
This is about qualia, isn't it?
 
The ball takes the shape of the viewer's worst fears
 
Outer Limits twist: the ball is painted by an evil AI trying to classify you.
 
@AndrasDeak I don't know enough about the subject but from a quick scan, pink could be qualia?
 
@AndrasDeak I like it, but I'm basically saying that only spectral colors are real, and all of the rest are fake. In other words, only colors composed of a single wavelength are legitimate.
In pop science, magenta is often singled out as the poster boy for non-spectral colors. But you won't find pink on a rainbow either.
 
2:40 PM
Plot twist: magenta is pink.
@Kevin but is it really a colour if we see it differently?
 
en.wikipedia.org/wiki/CIE_1931_color_space#/media/… depicts* all colors that humans can typically see. Only the colors along the tickmarked edge are spectral. The area of a line is zero, so the area of this chart composed of fake colors is almost exactly 100%
(* to the extent that a computer monitor can display them. I believe quite a bit of the greenest greens are absent, so use your imagination for those)
@AndrasDeak I'll take the pragmatic stance and say "yes". Even if the red you perceive is different than the red I perceive, it's still useful if both of us can obey traffic signals
 
3:04 PM
assuming that perceptions are consistent :P
 
If the red you perceive looks like green on alternating Thursdays, I suggest taking the bus
 
Are all Python's magic methods implemented in C?
 
I feel as though there are a number of ways to interpret this question
"When I write `a + b`, and Python decides that this means it should call `a.__add__(b)`, is that decision written in C code?" -- yes
"When I write a custom class and want to define `__add__`, do I have to write it in C?" -- no
"Are all the dunder methods of all builtin types written in C?" -- quite a lot of them are, but I bet there are some that aren't
 
Interesting answers, though I meant to ask the 3rd version
Do you know of any which aren't?
 
No, but I might poke through the source and see if anything turns up
 
3:45 PM
Preliminary results: everything I looked at was in C.
I think it would be more efficient if I said "100% of all builtin type dunders are implemented in C, no exceptions" and then a lurker will leap out of the bushes and say "Well Actually, memoryview.__matmul__ was implemented in Python from versions 3.7.9.8 to 3.7.9.9"
I bite my thumb at thee, lurker in the bushes
 
4:07 PM
morning cabbage
 
morning cbg
@Kevin I regularly leverage the "Let me tell you..." individuals for anything that isn't covered by 80/20 rule :)
 
Hi everyone.
Can someone help me understand what c_in and c_out means in FCN(c_in, c_out, layers=[128, 256, 128], kss=[7, 5, 3])?
There's no explanation in the documentation what c_in and c_out mean. Are they the # classes for the input / output?
 
4:23 PM
@Kevin IIRC some of the interactive helper builtins like quit and help are implemented in Python.
 
If anyone can help, that would be great. I'm still struggling.
 
@rb3652 We would struggle too. No docstrings and * imports. I'm not even going to bother trying to trace it back. You just have to start at the source
 
@roganjosh Thanks. I'm trying to understand each parameter.
kss=[7, 5, 3] contains the kernel sizes for each of the 3 Conv2D layers; First layer has kernel size of 7, second 5, third 3.
 
4:38 PM
I know what you're trying to do. I'm telling you why we also don't have any advantage over you in understanding this, because the library owners didn't see fit to actually document anything here
 
@roganjosh Right. Just to clarify, if I have a kernel size of 7, that means I have a 7x7 filter going over my data, correct?
 
I can't clarify anything. I have no idea how that library works
 
OK, thanks for trying.
 
Hi everyone, I am asking a question which might not be related to python directly. I am getting a lot of data from BigQuery and would like to analyze the data from the same. However we also use AWS(for our backend databases). In such a scenario, would it be prudent to use pandas for processing and storing the data in EC2? or should I look forward to tools like Kinesis or DataStudio ? or some other tool like Spark/Kafka?(the data generated is around 5 GB per day)
 
spark and kafka aren't data storage tools?
 
4:45 PM
no, I was asking for the processing part @roganjosh
 
Nor is EC2. I don't understand the question
"to use pandas for processing and storing the data in EC2?" then you need to re-phrase the question
Kafka isn't going to process the data for you. You've given a jumble of different technologies
 
sorry , yes, let me rephrase it. I am just trying to know what would be the best option considering the data size per day. Presently I am using pandas but the data generated is around 200 MB at the moment. However I know it will scale up to around 5 GB per day later. What would be the best option for me then to tackle this? @roganjosh
 
is it possible that python can do some kind of fitting to experiment data?
I have an external program that accepts input file which contains arbitrary parameters, this program returns or outputs a result file that could or could not fit the experimental data (depends on the parameters)
I plan to make a python script that finds those arbitrary parameters.
 
@RaphX There is no definitive answer to that. What does the processing involve? In our AWS pipeline we have Redshift to store the data and we move from there
@EnthusiastiC Yes it can do that
 
@roganjosh It currently involves flattening of embedded JSON records and then creating funnels for visualization later in dashboards. The files are conceived in JSON format. So do we need to build a data warehouse then?
 
4:52 PM
Both of you need to clarify your points atm because you're asking abstract questions
@RaphX I didn't say you needed to build a data warehouse. You're communicating this to me like a sales person ("creating funnels for visualizatation") rather than a programmer
 
Sick burn
 
It wasn't really meant to be a burn. If I can help, I will, and there's no good chat to redirect people to. But it needs to at least explain what's going on
 
But those values are arbitrary do not follow certain rules or law
 
5:07 PM
In which case, they can't be fitted?
If you're saying that it's totally random then that implies there is no trend... so you can't fit a trend to it (with any confidence)
 
if I do it manually I have to keep guessing those values till the fitting becomes acceptable
 
Do what manually?
 
guess values and store them in a file
that file is fed into a program
 
We have absolutely no idea what the data is, what you're trying to achieve, what you're actually doing, what...
 
I get a result and I compare to a pdf file that contains the experimental data.
 
5:11 PM
It says FCN(c_in, c_out, layers=[128, 256, 128], kss=[7, 5, 3]). Now I know that kss is kernel square size, but what exactly does the 128, 256, and 128 in the layers mean?
 
@EnthusiastiC Great. Take a step back for a minute and think about what you're asking us to answer here (given that we have no knowledge of the data). What are you expecting?
 
Is 128 the # nodes in the first Conv2D? And 256 in the second? And 128 in the third?
 
37 mins ago, by roganjosh
@rb3652 We would struggle too. No docstrings and * imports. I'm not even going to bother trying to trace it back. You just have to start at the source
 
@roganjosh I know, I looked at the source -- pulls out hair it's a dead end!
 
I could be wrong, but I don't know anyone in the room that has used that library
@rb3652 that perhaps says something about it?
 
5:16 PM
Yup, it's obscure, but it does everything I need so simply. What it doesn't do, however, is have good Docs.
 
I think it'll be a fun evening of source code exploration for you, then :)
 
Well. There goes my evening.
I'm going through their tutorial notebook, and I'm wondering if anyone here could help me understand something.
^ How do they get the loss before they train?
I got it!
 
Fair disclosure: I'm about done with these obscure questions from, now, 3 people concurrently. No code, no context, no explanation, just an expectation that people are going to respond and help
 
@roganjosh -What I am doing;
1) fed an input file that contains random parameters into a program which in turn outputs results as files.
2) I compare the results to a pdf file that shows experimental data.
3) I re-type the values of those parameters if I didn't get a good fitting and do the same steps again and again until some confidence is gained.
- What I intend to do
1) Write a python script(s) that let me find those values for which the difference between results of that program and experiment is minimal.
I hope that makes us in the same frequency.
I feel that's a pain to reach! several steps need to be done
My approach to a solution,
What a python should do is,
- Access and set the random parameters in specific lines for any input file
- Redirect the input file to the program for calculation
- Read the results from output file
- Read the experimental data from pdf file
- Compare
- Change the random parameters to other values
- Loop over the above steps until convergence
the problem resides in the behavior of changing values.
 
5:55 PM
Hi all.
 
Ok, and how do you think that happens in relation to "is it possible that python can do some kind of fitting to experiment data?"? The plan seems fine, so I guess that answers your questio
 
Here's my problem:
I have a CNN that's supposed to classify a graph as Linear, Quadratic, or Cubic.
My CNN has 44.6% accuracy after 5 Epochs
But that's not what I'm suspicious about.
^ The CNN classifies all the Quadratics correctly!
What's going on? It's obviously doing something bad, but this is just so weird I can't quite tell what's happening.
What, is it biased towards quadratics?
If anyone can help, that would be great.
Here is the code: https://colab.research.google.com/gist/oguiza/d85ffc445b528d23799a5d19f2709c23/copy-of-time-series-multi-class-classification-using-tsai.ipynb
Again, if anyone could help, that would be great. This CNN's been popping up with all sorts of problems for the past week.
Even the loss is super weird:
I'm still trying.
Confusion Matrix is even worse now.
 
6:40 PM
cbg
I see folks are having a jolly good time.
 
I don't even understand my problem.
@MisterMiyagi Sorry to ping, but do you have any ideas?
 
Well, if you don't understand the problem, do you think you could explain it?
 
@rb3652 don't ping random users with your problem. Your filling the room with walls of noise is bad enough
 
@AndrasDeak Sorry, my fault
 
indeed
 
6:46 PM
@MisterMiyagi Sure, the problem is that my CNN classifies all the quadratics correctly, but misclassifies all other classes. It's very weird.
 
my magic 8-ball says you're overfitting
 
^ Yes, of course I considered that
That was my suspicion the moment I saw the large gap between the validation and training loss
 
@rb3652 please stop posting messages mid-sentence
 
@AndrasDeak OK
 
My general advice is to throw domain knowledge at the issue until a cranial impact event occurs on the plateau of work.
 
6:49 PM
I've been trying the whole week, in fact, to resolve this CNN. I tried changing the architecture, changing my training set, changing the hyperparameters ... nothing seems to work.
@AndrasDeak Even if it was overfitting, wouldn't it be strange to overfit after only 5 epochs? And furthermore, to classify solely one class 100% correctly?
 
I don't know anything about neural networks
 
I see.
 
But someone might note that your confusion matrix says that everything gets classified as a quadratic
I'm sure there's a Data Science Stack Exchange or something where you could ask a well-formed question about this
 
Magic 8-Ball says the data isn't properly normalised, seeing how the bias of the result matches the bias of the data.
256 Slotties make 1 Slurm.
 
you know that being actually helpful will keep them pinging you :P
 
7:00 PM
It's all part of my master plan to silently sabotage the ML department.
 
heh
 
@MisterMiyagi Hm
 
Cbg everyone, I hope you're all doing well! Just stopping by to say that I graduated today and I am finally an engineer! Thank you all for the support!
10
 
@AlexandreMarcq 🎉
@MisterMiyagi Signs point to No.
 
@AlexandreMarcq congrats
 
7:18 PM
@AlexandreMarcq congratulations!
 
7:45 PM
Hi All. Python/Django question. I have an API built that writes to a database. 2 of the fields are "start date" and "end date". I'm doing a query filter to check if there is already record(s) between those dates already. For example, if I'm posting a record with dates 9/1-9/10, I want to do a filter to make sure there's no records with: 9/1-9/10 OR 9/2-9/9 OR 9/3-9/12 OR 8/25-9/5. Pretty much can't let the dates overlap. Any query filter ideas?? Thanks!
 
Why do that vs. some identifier of the user?
 
Aren't you reinterpreting the problem there? What does the user have to do with whether any records exist in a given time frame?
 
It breaks REST principles to get what you want. Maybe you can store the state locallly
 
I'm not quite following what is the issue here. Ask for records starting after the start and before the end date or ending after the start and before the end date?
 
@Aran-Fey probably
 
7:58 PM
If you get a record for 9/2-9/10, is it valid if you happen to have a record fro 9/1-9/11 already?
 
But the question basically suggests there is no primary key (or at least an identifier) so it's nonsensical to me
 
8:13 PM
MisterMiyagi
Not Valid
If you get a record for 9/2-9/10, is it valid if you happen to have a record fro 9/1-9/11 already? - Not Valid
 
Not Valid isn't a Python error
 
Okay. So extend the rule of "start is in the range" or "end is in the range" with "start is before the range and end is after the range"?
 
9:01 PM
Cabbage all
I had a go at adding a description to the python-typing tag a few days ago and spent a fair bit of time on it — might someone with a little more rep be able to take a look at it? stackoverflow.com/tags/python-typing/info
 
 
2 hours later…
10:34 PM
@roganjosh Start and End could be the PK here...but user is trying to add business layer in a query to avoid writing logic layer policy? Bad in general, esp if pushing restful backend
 
Why would you have a datetime as the primary key?
The entity is the user. I'm so confused
 

« first day (3989 days earlier)      last day (942 days later) »