« first day (3818 days earlier)      last day (1130 days later) » 
07:00 - 20:0020:00 - 00:00

7:19 AM
Funny enough a VS code question of mine of how to show function usage like in Pycharm is how I got my most rep. It feels like a scam. Ask well done, highly researched mvce question get 1 upvote. Ask simple how to do x without any research question get 50upvotes. But well what you gonna do
 
 
2 hours later…
8:59 AM
What would one suggest to learn object oriented programming in Python? And similarly, how to learn gui programming on Ubuntu and Python?
 
9:35 AM
Having dragged myself through OOP, I'm really skeptical about whether a good guide exists. I'm certainly not a guru on that side, but I think most of my learning has come from going through smaller projects on github and seeing how people structure their code
I mean, you have to be selective in the projects you look at. All that's on github isn't gold
I really wanted to push harder on the "all that glitters ain't gold" saying, and it only takes dropping a letter to make it work, but also not work simultaneously :/
 
10:04 AM
Cbg mates, just noticed I had 9400 rep and then after getting an answer accepted my rep went down to 9386 instead of going up. There is no notification about -15 anywhere either. Is it a glitch?
@JaakkoSeppälä For GUI you can start with tkinter and slowly elevate to PyQt5 and Kivy.
 
probably a routine deletion i suspect. maybe one of your answers got axed by some question prune. just a guess on my part
 
@ParitoshSingh They would still leave a message right? Do the routine deletion happen at this time though?
 
Don't think so
 
dont think you get informed if a question you answered on gets nuked
also cbg!
 
It's simple to find out - just cross-reference your answers with your mental index of everything you've ever answered and you'll find the culprit :P (being facetious)
 
10:08 AM
@ParitoshSingh I usually get a 'User removed -10' message
@roganjosh Was going to do it xpp
 
Oh, then it's just that the user selected a different answer
 
also i assume that message only kicks in when a "user/account" is itself removed, and not on questions
 
OPs can do that at any time. They just swapped their tick to another answer to their question
 
Damn that hurts ;) But if thats the case it means I lost 29 rep.
@ParitoshSingh Hmmm maybe, its the first time happening though.
 
-29 is rough to explain. You lost an accept and an upvote, which will get you -25, then... err
2 downvotes?
 
10:13 AM
@MisterMiyagi hi, I do not know how to thank you. Would you mind posting this comment as an answer to my question? Your comment made my code work as a charm!
and I have to mention, your help was a huge motivation to me. I was almost gonna stop working on my code.
 
Just delete the question, honestly. It's not a good question, and even if it was, it should be closed as duplicate. There is no scenario in which it should be answered.
 
@Aran-Fey I understand your concern on the quality of the question. But let's just understand the weight of the words we choose when we give comments to users, specifically the ones like me who may not be as skilled as you. Using works like "it is not a good question", or in your back comments I felt like I am being pressed by saying you have several obvious bugs... these make new and lower-skilled users upset and unmotivated.
Just, please comment using neutral words to keep everyone motivated.
 
I'm not sure what you mean by "neutral words". I wasn't rude, and everything I said was true. Isn't that exactly what "neutral" is?
 
I think that's a fair ask @enthu. I hope you don't lose motivation over it but also I think Aran was just trying to be pragmatic - that question could potentially get you downvotes. Occasionally feedback can be direct because of the nature of the profession where code always has very-well-defined logic so it can be interpreted as a bit too direct some times
 
@roganjosh All those in a nick of second. Seems so weird. My bets are on 'a bug' and I hope it is too, cant afford to lose those many ;p
 
10:23 AM
@Aran-Fey I am not saying you are rude. You are extremely kind and tried to answer my problem with utmost care. I am saying, we can approach weak users with lower skills with the words which make them more motivated.
 
It is a fair demand. It is not hard to be nice. Sometimes seeing such questions can be triggering considering the amount of times it has been repeated and maybe some foolishness in it. But we all have been there and got to be better. So its fair ask to respect and stay 'neutral'.
 
@Aran-Fey As an outsider, I could easily look at that comment and think that it perhaps neglects the fact that the OP did actually have effort and motivation behind it, even if the premise was slightly off
It certainly isn't in the camp of the people asking "do this for me"
 
I am saying, even if it is duplicate, we can raise duplicate flag on it with sufficient comments on the question. Duplicate suggestions can also be a motivation for OP to read more. Sometimes sharp true comments make weaker users extremely unmotivated. They are themselves under huge pressure when they bring their problem here. Trying to face sharp words will add to their stress and make them sad. Specifically when they are young and not professional.
 
@roganjosh Anyway is there any place to register complaints? Stackoverflow Meta is fine?
 
It's a fair request, but I'm afraid I no longer have the patience to tiptoe around anyone's feelings. Stating the truth in a factual matter is as friendly as I can get nowadays
 
10:27 AM
@enthu FWIW I don't think they're sharp words, just very objective. I do think that your best course of action is to delete the question to avoid further issues
@Aran-Fey That's also fair. To be clear - I don't think you've done anything wrong. I'm just not surprised that someone could reply in the way that you got
 
@enthu I can relate, we just have to live through it at times, ignoring most of it. Stackoverflow gets a lot of critics too, for being a den for condescending people and bla bla. Just talking neutral here, not at Aran or anyone.
 
A lot of people on SO are jaded like me :D
People who spend their free time on trying to build a high quality Q&A site + people who have no clue what they're doing and only care about finding solutions to their problems = recipe for disaster
 
Whats wrong with "People who spend their free time on trying to build a high quality Q&A site"?
 
LOL
It doesn't make money for SO?
 
@roganjosh Hah makes sense :p
 
10:36 AM
In any case, I think this was a useful, but probably uncomfortable, experience @enthu. I've known Aran-Fey long enough to know there was no malice in the comment, but I can also see how you'd feel it was dismissive. As a Room Owner here, I'll do my best to keep things cordial, but it's useful to be exposed to this kind of thing and be able to not take it personally
There are many people that take the feedback too far, and I'm guilty of it myself in times of high stress. But you will have to face this to varying degrees, and you will be better-placed if you can tolerate objective feedback without taking it to heart, and then you'll recognise the people that are going overboard
 
It's easy to feel attacked when someone's criticizing you, even if that someone is perfectly friendly and doesn't mean any harm. There was an SO blog post about it a while ago.
 
At the end of the day, sometimes you'll need blunt feedback to be able to move forward without things being ambiguous. Please don't feel demotivated by it - the intention behind the comments is to help you improve
 
...Is there even a way to browse old posts on the SO blog? No clue what the designers were thinking when they made that website
 
Cool
 
I appreciate your comments. Although I feel that direct and sharp words may discourage younger and low skilled users; when all of you are standing on Aran's side; I accept your side of view. I have some years professional experience and I am not taking things personally. But I believe that motivating less experienced users will help SO as well. BTW, I again thank you @Aran-Fey and @roganjosh and any other person here who always help me with my codes.
Also, I deleted my question, older users are more aware of how things work here and how quality should be maintained. We should respect the culture here.
Thanks all :)
 
10:54 AM
@roganjosh Thanks for the backup, by the way
 
It's fine. I think I've just played attorney for both sides :P <internal conflicts intensify>
 
@roganjosh more of @Aran-Fey 's side btw :))
 
Well, only for the fact that I think that you still get something useful out of that @enthu
I am still sympathetic to how you read the initial comment
You have to keep in mind that we have a revolving door of people that just expect things to be solved by other people, so sometimes the default stance of people in this room (experts in many cases) is quite defensive to keep sane
 
sure but I never expect things to be done by others @roganjosh
 
I know :)
I wasn't suggesting that you, in particular, were. I'm just saying that we have to handle a lot of people that we don't know and they occasionally cause issues. So the regulars have to have a stance against that. You might have been caught up in that sorry
In any case, I think someone needs to post a kitten picture now or something
Kitten, or maybe a bunny would be more seasonally-appropriate
 
11:06 AM
 
Hmm which map projection should I use
 
Mercator?
 
Equirectangular is literally zero effort because it just maps lat/long straight to x/y. And it makes the US look bigger than it really is, that's a plus for my Americentric users. I only wish it was even bigger and more important...
Ok perfect
 
..... wow
 
I'm just giving the people what they want
 
11:13 AM
Slightly miffed at being relegated to like 5 pixels in the UK!
 
Hmm, perhaps I could scale each country relative to their US diplomatic friendliness.
I've seen some neat maps that do that except with, say, manufacturing exports. It's pretty neat. I wonder if there's a library for that.
For example this population map worldmapper.org/maps/grid-population-2020 which bulges in a rather uncanny way
 
@CoolCloud If you wanna go the Meta route, I wipe my hands of any consequences btw. It's a silly place.
 
@roganjosh Experienced it myself ;)
 
Google tells me this is called a "cartogram". python-graph-gallery.com/cartogram claims that geopandas can generate them, but their example gallery is currently empty.
 
Feels like could have got a better response - meta.stackoverflow.com/questions/406318/…
 
11:26 AM
My default assumption is that pandas can do literally anything unless proven otherwise, so I'll take their word for it
@CoolCloud If the question is "what sort of events could make me lose reputation without a notification?", the linked post suggests that common causes are "your post was removed", "the voter reversed their vote on the same day", and "SO background processes, such as the one that merges accounts"
Apparently you can confirm or rule out the first one by including removed posts in your reputation history with the checkbox at the bottom
I agree that the commenters could have been nicer to you. Ah well.
I think questions along the lines of "why did I get downvoted?" are very very common on Meta, and many of the askers are the "I want to speak to your manager" type, so readers are already in a confrontational mode when they lay eyes on your post title
TLDR: it's a silly place.
 
Lol I agree
 
it's also a duplicate
 
11:42 AM
Probably yeah, unless it's a brand new bug that caused the invisible rep loss. Odds are low... But never zero >_>
Half the time I post on Meta I think "depending on what's causing my problem, this will get dupe hammered, and I will deserve it". It's still worth finding out though
Downvotes and close votes on Meta should be taken even less personally than ones on SO because they can mean more things than "low quality" and "do more research please"
 
Yea true.
@AndrasDeak No it isnt, the link did help, but I also mentioned the reputation peak wasnt shown there.
 
I'm curious - does https://stackoverflow.com/reputation display a complete report of all events without exceptions, or does it follow the same rules for hiding reversal events like the reputation tab does? — Marty Cagas Jun 14 '20 at 13:53
I'm curious about this too.
 
there are certain events that alter history
like when question upvotes were raised to 10 rep (ugh) or when serial votes are reversed
 
I think CoolCloud's first comment on his post suggested that he didn't see any -14 events there either, so I guess there are indeed exceptions
I wonder how many other rep-affecting "background processes" there are besides serial vote reversal and account merging. Let's say, excluding one-shot events like changing what upvotes are worth sitewide.
 
12:07 PM
@Kevin Hmmm
 
Bah, this documentation says I should be getting an array of arrays of floats, but it's giving me a list of one-element-lists of tuples of floats. Incorrect in both concrete type, and in dimension.
 
morning cabbages, folks
 
cbg
 
12:24 PM
Ah, I forgot, a polygon as defined by this spec is not just a list of coordinates.
I spent an hour yesterday talking about "polygons with holes" so that should have occurred to me. Can't define the inner and outer boundaries of a donut with one list.
Style poll:
#approach 1
if x == 1:
    a()
elif x == 2:
    b()
elif x == 3:
    for _ in range(10):
        b()
elif x == 4:
    c()

#approach 2
if x == 1:
    a()
elif x in {2, 3}:
    for _ in range(1 if x == 2 else 10):
        b()
elif x == 4:
    c()

#approach 3
if x == 1:
    a()
elif x in repititions := {2:1, 3:10}:
    for _ in range(repititions[x]):
        b()
elif x == 4:
    c()
3 I think is unnecessarily fancy. 2 is borderline fancy, but deduplicating the b() call is valuable to me.
Especially because in my real code, I'm not calling b() but rather executing a suite of statements inline. I could make it a function, but it would be a bit awkward
 
12:39 PM
Looks to me like a piece of code that would benefit from refactoring. But given those 3 options, I dislike #2 the least
 
Thanks :-) Once my real code's functional, perhaps I'll post the actual thing and re-evaluate
 
# approach 5
calls = {1: a, 2: b, 3: (lambda : [b() for _ in range(10)]), 4: c}  # I'd rewrite #3 with the itertools consume recipe
calls[x]()
 
Hmm I see I see
 
Finally figured out what happened.
Under the graph section, it showed this.
Feels kind of dumb because I did toggle that switch but I guess it was elsewhere
 
whoopsy daisy
I hope you still enjoyed your venture into meta
 
12:53 PM
Hah, im never going back there again xpp
 
Sure does.. ;p
 
I'm glad you figured it out :-) I was quite curious what was causing it.
 
But why the delete though. The answers could've still stayed there.
 
1:10 PM
huh?! I just found a bug in SO Main. I was in the middle of answering a question. As I was typing out my answer, the question was dup-hammered by someone else. Out of muscle memory, I hit <kbd>Tab</kbd><kbd>Enter</kbd> to submit my answer, which then took me to the homepage. I hit the back button to see the cached version of my answer/post, which I was now able to submit, even though the question had already been dup closed
 
They intentionally give you wiggle room for posting answers a little after a close occurs, but it's interesting that navigating away from the page doesn't disqualify you.
 
* As I was typing out my answer, the question was dup-hammered by someone else (and I saw the orange banner "no more answers are accepted")
does that change anything?
 
It increases the interestingness level but the underlying principle is still the same, I'd say
The intended design is to be forgiving of people with patchy connections that might not receive async notifications like the one that causes the orange banner. We don't want them to become frustrated that their submission won't go through and no explanation is given. But if the orange banner does appear, we should feel free to refuse the submission.
So I guess what happens in your case is, when you press Back, the cached page is from before the async notification arrived. When you click submit, the client-side validator checks for an orange banner -- nope, none found -- and submits to the server. The server trusts the client, assumes you're a user with a patchy connection, and accepts the post
 
that's interesting. Thanks for sharing :)
 
I don't think I've ever been able to submit an answer after a question was closed, but seen it happen to dozens of other people, so... the algorithm seems to be very capricious
 
1:24 PM
Perhaps it could do a better job remembering that you've already seen the orange banner, by...
- saving a cookie when the banner appears, and checking the cookie client-side during post submission.
- asking the client to send a "seen" receipt when it first gets the async notification, and cross-checking this value server-side during post submission
- sending a "seen" receipt when you navigate away from the page after the banner appears. (is this possible? idk)
- not letting you return to a cached page, or ensuring the cache is of the page's state after the banner appeared (also not sure if
I suspect some of these have state-wrangling problems of their own. For example, making sure these cookies and server-side flags continue to have sensible values even if the question is reopened
 
@Kevin 4 hours
 
File "improved state tracking" under "would be nice to have, but there's not much clamor to implement it"
 
@Aran-Fey need a lack of server-side validation. Something like losing network connection?
 
If it fell out of the sky into our code base, great. If it requires N>0.25 man hours... eh
 
I keep forgetting the details but it's harmfully dumb
 
1:30 PM
that sounds about right
 
<clamor has increased by 2>
 
Coming soon: A browser addon that blocks a tab's communication with SO while you're writing an answer
 
Even with my proposed designs, it only provides a (marginally) better experience for well-meaning users. Naughty users can still circumvent each of them to maximize their 4 hour grace period.
 
grace period should be 5 minutes
 
1:34 PM
Agreed, but I'm not emotionally invested
Make them implement mixed-format multiline messages in chat first
 
With syntax that matches main
 
Anyway. Here is my actual code corresponding to the abstract style question I posed earlier. render_poly is the b() call I was trying to refactor out.
One of the reasons I didn't want to make it a function is because it makes use of the global variable draw, and I usually feel awkward referring to a name on a line higher than where it is first assigned to.
And I also feel awkward assigning non-constant globals above my function definitions.
 
You know this would all be easier with matplotlib.basemap :P
 
Oh, that's much less complex than I anticipated. You can just pass draw as an argument, no problem at all
 
Hmmmmmmmmmm ok. I'll go with that for now. I might need to pivot to something more OOPy as I add features.
@AndrasDeak Acknowledged. I'm exploring language-independent approaches for the time being.
 
1:46 PM
I figured
 
Kevin, haggard and dying, crawls through the desert. He reaches a sign:
all-in-one module solution: 5 miles east.
reinventing the wheel: 10 miles west.
He begins to crawl west with renewed vigor.
In hindsight I think I subconsciously blocked out the "pass draw as an argument" approach, because in 95% of my image rendering projects, that's a frustrating design because there are multiple layers of function calls between draw's assignment and any invocation of draw's methods
I almost always have a single draw object whose lifetime is roughly the same as the program's, so just making it a global saves a bunch of clutter in my function signatures. I guess that's not an A+ justification, but it is what it is.
 
which direction was K crawling in at first? If North or South, then cool. Otherwise... he was already travelling in the direction of one of the solutions
 
He may have come from the east, so he already passed by the module, and was still like "eh... Maybe later" even when it was within arm's reach
"I don't want to fill up on cool life-giving water now, in case I run into cool life-giving vitamin water, enhanced with electrolytes and a hint of lemon"
 
Reminds me of the old adage, "never leave fish to catch fish"
 
2:02 PM
I should be more mindful of when I'm choosing interesting failure over boring success. I'll probably still pick failure a lot, but it still ought to be something I'm aware of.
 
2:17 PM
Ah, nice, this shape file lists Lesotho after South Africa, so it renders them both properly even if I incorrectly fill in South Africa's interior boundaries.
I wonder if it does that for every enclave. I see one or two others dotted about.
 
Since Manjaro isn't cooperating lately, does anyone have recommendations for a rolling-release linux distro with GNOME? (I was thinking of using vanilla Arch, but that seems like a pain to install)
 
Theoretically, it's not possible to define a partial ordering of enclaves, because you could have three countries that are all enclaves of one another. I don't think that happens in real life though, even in that really fiddly bit in the Netherlands
 
@Aran-Fey why don't you like Ubuntu?
 
Not sure tbh, it's been forever since I last used Ubuntu. Guess I could give it another chance
 
I've been a user for nearly three years now. Programming with this OS is a dream. I was a lifetime Windows user prior. All of my systems are dual boot windows/ubuntu
 
2:25 PM
Fig 1. A three-way exclave system that cannot* be properly rendered with a single pass through the country list that renders opaque polygons
+-------+-------+-------+
|       |       |       |
|   +-+ |   +-+ |   +-+ |
| A |B| | B |C| | C |A| |
|   +-+ |   +-+ |   +-+ |
|       |       |       |
+-------+-------+-------+
(*unless you rearrange the country list so that the little exclaves of A, B, and C are separate entries from their larger territories, and appear at the end of the list. But geometrically testing for enclaveness is its own can of worms)
How fortunate that I won't be using an approach that depends on opaque polygons
 
@Kevin YAGNI?
@Dodge I switched to debian because of Canonical's shady data selling practices
@Kevin and why can't you? Have a different layer that masks the same country with transparency, and then merge down. Will that break?
 
2:43 PM
@AndrasDeak Interesting, I'll google that at some point if I remember. I always opt into anonymous data collection for development purposes. Coming from the major OSes, the data collection is probably not much different.
Nothing pops up on my DDG search. They're good :)
 
@Dodge it used to be opt-out with the opt-out option stored safely in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying "Beware of the Leopard." And at around the same time they put the Ubuntu Manifesto right next to it.
I think they sold your file and search miner in dash(?) to amazon
 
No such thing as a free lunch I guess
 
@AndrasDeak I'm imagining a quite primitive image library that does not do transparency, full stop. If you want to merge layers, you can blit opaque rectangular sections onto other opaque rectangles.
 
@Dodge that's a pretty sorry excuse for an open-source-like OS selling you out
@Kevin OK, I misunderstood what you meant by "opaque polygons"
 
This is quite unrealistic, but as you say, YAGNI, so the triple enclave problem is only interesting from an academic viewpoint
I did a sketchy job of explaining what i meant in the first place, so, understandable
 
2:49 PM
It's quite clear in hindsight. I just slept about 10 hours in the last 2 days :P
 
@AndrasDeak I'm not convinced that I am being sold out. I also don't think what I said is as much an excuse as it is an unavoidable aspect of human existence.
 
3:10 PM
@Aran-Fey ubuntu is typically not rolling. If you want ubuntu I suggest debian instead. I've been using debian testing for 4+ years without any issues, and there's no corporation hovering above the distro, and the packages in testing seem a lot less stale than standard ubuntu repos, if memory serves.
 
I googled a bit and found there's a way to enable rolling release in Ubuntu, but on second thought, I wouldn't mind trying a fixed release distro again. Maybe those won't cause as many problems
I now regret allocating only 1 partition for temporary linux installs. I need about... 3
 
3:25 PM
I wonder if it's possible to re-partition an existing partition into multiple partitions, assuming there's still enough free space to go around
 
@Kevin it is
 
Without deleting the current partition's data? An impressive feat, yet plausible
 
in principle it might even preserve data on a partition if you can carve out new partitions from unused space, but most sources usually say "it might work or might not, so do back up everything"
I think other partitions are not supposed to break at least...
@Kevin well, resizing partitions is a thing. But again, not too reliable I think.
 
I imagine chopping empty blocks off the end of a partition is feasible, but there might be less space there than you'd like if your disk is fragmented
 
yeah, I bet there's a step where the OS tries to reshuffle everything
 
3:28 PM
1 Gb available, sprinkled throughout memory in little 10 kb pinpoints
 
I tried to shrink an existing partition once, I don't remember the details, but I do remember that it didn't end well
 
there are threads like ubuntuforums.org/showthread.php?t=1590522 which say "it should work fine (as long as your dog doesn't pull the power plug from your PC)"
 
Maybe there's a leaky abstraction somewhere, so the "empty" space at the end of the partition isn't really truly empty
 
considering how even decent dist-upgrade tooling will tell you to back up before you go forward, this is not too encouraging to me
 
Oops, the undocumented part of the bootloader was saving an important configuration setting at 0xFFFFFFFF, but it's gone now.
"If 0xFFFFFFFF is 0, display the Ubuntu logo in black and white. If 1, display in color. Otherwise, undefined behavior. Let's see... the value is 'Q'. Undefined behavior it is! Executing command HALT_AND_CATCH_FIRE."
 
3:39 PM
Resizing a filesystem/partition via the OS tools is generally safe. A backup is generally advisable to protect against human error, not tooling failure.
 
that's just what Big Dataloss wants you to think
 
The tooling was designed by a human too, just saying ;-)
 
"This software was built by our benevolent AI mastermind" just isn't good publicity. :P
 
"The car accident was caused by human error"
"The driver was going straight and all four wheels came off simultaneously. What error did he make?"
"Not that human. I mean Frank, the lug nut guy at the car factory. His error was forgetting to tighten anything all last Friday"
Frank says "my bad", we trust this settles your emotional damage claims
 
Error number one: trusting Frank.
 
3:44 PM
Mistake two: trusting Steve, the guy in charge of not trusting Frank and checking his work
 
3:58 PM
In the UK, Frank was a name used in a huge multi-year campaign about drugs btw. I don't know whether the service still exists. Frank had issues and was simultaneously a listening ear
 
We continue to employ Frank because his great people skills make the team feel like a real family. The increased morale and productivity cancels out the occasional lawsuit.
 
Just wow. Apparently that was a decade ago. I've lost track of time
 
Frank was there when Deborah left me and took the dog. Frank helped me move. Frank saw me kill a man. Wait, forget that last one.
 
well this is an awkward conversation to walk into.
 
Hiya PiR :P
 
4:02 PM
You saw nothing, pi
 
o/ rj
I keep hitting refresh on my newegg order tracking waiting for 2 dims of memory. I crashed my computer yesterday answering trying to answer a ProjectEuler question. I thought to myself "Better get more memory so I can make bigger mistakes"
 
ulimit is cheaper :P
 
Hmm, I mentioned my map rendering project to the boss and now my v0.2 requirements document is 30 lines longer
 
Wife sees order email and asks "Hey, did you need that?" I said "Mhmm"
 
I better get 30 brownie points for this at least
 
4:09 PM
What you wanted was a work sanctioned hobby project.
 
<_<;;
 
"If you could use python 2.6 support IE5 that would be great"
 
[I take 99 points of psychic damage]
I can support IE5 as long as all the countries in the world are squares
diagonal lines weren't invented until IE7
 
AAB
hi all,
have a django app, I need to send users a notification or email when I have price drop.
I have created a python script, and run a cron job at certain times in the day to get the list of userid to which I need to send notifications too
 
Ok, sounds like a reasonable design so far
Very similar to the design of one of my projects long past
 
AAB
4:18 PM
@Kevin thanks, I use scrapy to collect data.
 
New SO cookie popup is crazy
 
AAB
I was wondering if I can make code changes in scrapy to do acehieve the above
 
Note to self, invent or find a browser extension that accepts all cookie banners so I never have to look at one again
 
AAB
scrapy crawl runs every day and insert a new price was wo0ndering if it would be better to get userid during scrapy operation itself
 
@Kevin Can we name it Santa Clause?
 
AAB
4:21 PM
Also to send notification to specific userid I need to create a mapping between userid and tokens/regid
should I do this myself or go for some service that will do this for me? I heard onesignal allows sending notification to specific users
 
I'm interpreting the first question as "should I continue to have a cron job that sends emails, or should I just make my web crawler do it when it notices a change?". I think both designs are feasible. I already mentioned that my own project did it with a cron job. Handling both web crawling and email in one script is surely possible. I would just be very careful to keep the crawling and email code as separate as possible.
import crawler #todo: write this module
import emailer #todo: write this module

def main():
    user_ids = crawler.insert_new_prices_and_get_user_ids_whose_price_changed()
    emailer.send_emails(to=user_ids)
Here, the crawler has no knowledge of the emailer, and vice versa. Only the main function passes data between them.
Running the emailer immediately after the crawler is nice because the emails will get sent out quickly after a price change has been detected. If you had a cron job for the emailer, it might only run at, say, 6:00 PM even though the crawler finished at 1:00PM.
In my own project, it was fine if the email was five hours late, since the data wasn't urgent
You might be thinking "what if I have a cron job for the crawler at 1:00 PM, and a cron job for the emailer at 1:05 PM? Would that ensure that the emails are quick?". Perhaps not. If the crawler takes longer than five minutes to run, then some or all of the emails will be delayed by an entire day. Network I/O speeds can vary for many reasons, so it would be hard to confidently estimate how long the crawler should take to run.
Your other question seems to be about push notifications, which I'm not very familiar with. If it were me, I'd be happy if I found a service to do the hard parts.
 
AAB
4:40 PM
@Kevin thanks a lot :)
for the crawler approach would it be better to use some temporary storage like say a queue or somehing that will store partid that have price change, then based on values present in queue I read and send user email/notifications
Even for sending mails sercices like send grid exists shoudl i think about integrating with those apis if i get more user traffic?
or do it now
 
@Kevin why not decline all?
 
5:29 PM
Alright, we'll make it a configuration setting.
@AAB A queue would work, sure
I don't think you need to use SendGrid right now. if you get consistent userbase growth, you should be able to predict when it would be good to switch to an email service.
If you want to think ahead, then you could design your email code so that it will be easy to eventually switch out for a service. I mentioned modularity before, that's a big help. You could even write an interface that's similar to SendGrid's.
 
Would this be the place to ask for help, if I'd like a short look on my code to see if it's done the "right" way?
 
5:48 PM
Sure. Ask away.
BTW, does anyone know about the Google's coding competition?
 
@PoornaChandraR which one?
 
Here's the short code + a picture on Imgur in the end when debugging.
https://pastebin.com/tuJnRFfM
I intend to put it into a neural network later (a time-series signal and a lot of misc values), but I'm not sure if putting everything (misc values, two-dimensional signal(16 channels each having many datapoints), labels) in a dataframe is the best way to do it.
Currently manipulating it when putting multi-dimensional objects in a column confuses me a bit now and then :p
And to test, I'm trying to "allData.loc['EEG'] to get the EEGs out again, but it keeps giving KeyError(key) for 'EEG'
correction: I do test = allData.loc["EEG"] hoping the test would become the column, but unfortunately not
 
AAB
@Kevin thanks kevin, I think I will hold off on using sendgrid or some other service for now. As to making it modular, yes definitely need to look into it.
 
6:10 PM
@PoornaChandraR I've heard of Code Jam but I'm not familiar with the others
 
@TDuncker Unfortunately we can't run that because it requires files local to your computer
I appreciate the effort, but there's another level of abstraction needed here
 
My dataframe experience is limited, but I'm suspicious of any dataframe that doesn't have a dtype more specific than "object", or doesn't have a compelling reason to be a dataframe instead of something else, such as a dictionary
 
Oh, okay. But looking at i.imgur.com/ijP10me.png, shouldn't I be able to do test = allData.loc["EEG"] and it would extract the column called "EEG" which is three long, filled with objects of 87022x16?
 
To be honest, I don't know what loc does.
 
@Kevin What would it be better I work with, considering I have a 87022x16 object, plus some 16 different values of integers, all pr. "set" of data?

loc should extract the named column :( iloc should do it based on index of column (in pandas)
 
6:20 PM
pandas.pydata.org/docs/reference/api/pandas.DataFrame.loc.html makes it sound like it extracts a named row rather than a named column
Maybe I'm looking at it sideways... (not a joke)
 
@Kevin [Scottish accent]"It wisnae me!"
 
Ah yes, the good ol' XY problem - knowing whether a given 2-dimensional thing is arranged like [x][y] or [y][x]
 
@holdenweb See, this is why we hired Daniel, to make sure you're watching Frank.
 
I suspect Daniel and Frank are in collusion and have reverse-MitM'd me out of the chain of command.
 
@Kevin it can do both
 
6:26 PM
Hmm interesting
 
Think I maaaybe found out the error.
 
>>> import pandas as pd
>>> df = pd.DataFrame({'a': [1, 2, 3], 'b': [4, 5, 6]}, index=[7, 8, 9])
>>> df.loc[7]  # uses index, gives a row
a    1
b    4
Name: 7, dtype: int64
>>> df.loc[:, 'b']  # uses column name, gives a column
7    4
8    5
9    6
Name: b, dtype: int64
and there's df.iloc to use integer ordinal indices for the same
 
I'd love to give some concrete advice about storing complex data, but I don't know much about machine learning in general or the problem at hand in specific, so that leaves me with mostly platitudes
You can represent nearly any kind of data with just lists, dicts, numbers, and strings. Don't make your data structures more complicated than they need to be. Identify which parts of your code need to be fastest, perhaps by writing it the easy and slow way first.
 
Looking at that variable explorer screenshot it seems like some cells in the dataframe might be python lists, so meh
 
Don't take any wooden nickels
 
6:37 PM
Atm little of it has to do with speed. It's mostly just trying to not confuse myself, and start good practices
The last element of the dataframe is a list of the EEGs (16 channels of 87022 datapoints)
 
if you want good practices remove non-scalar cells from your dataframe
 
Non-scalar = multidimensional? So keep the list of EEGs separate in a different object?
 
yes
multidimensional as in 1-or-more-dimensional
numpy and pandas work best (fastest) if types are scalars: numbers, bools, or in case of pandas, strings
having other objects lying around in a dataframe sticks out like a sore thumb
 
I was trying to say something like that when I mentioned dtypes, but I think I ended up, not saying that.
 
Hey,
I have a couple of Inputs that I have collected into a JSON object & a File, They're to be sent to the server.

I want the server to process the Json object and add its data to the database first before I link the file to the database as I'd need to rename the file to match the serial number/ Id that results from progressing the Json object.

what is the best way to do this ?
should I separate the upload file Function from the Json Object function and just use async between them ?
 
6:44 PM
@LoopingDev So when you insert a new row into the database, you use that row's primary key as the new filename for the JSON file? Maybe you can determine what the next id will be before you actually add it. I think Oracle has a function for that...
 
@Kevin yeah, dtype is the correct terminology here
 
👍
 
@Kevin what if 2 users uploaded at the same time, one of them could bet the other depending on the upload file size.
 
What would be a way to ensure I don't accidentally scramble them? Like, the first set of values is associated with the first EEG in the list and its 16 channels.
Like, it'd mean everything breaks if I accidentally associated value set #1 with EEG #2
 
@TDuncker put the EEGs in a dict of arrays, where the key matches the index in the dataframe
in more general cases you could use any unique key in the dict of EEG arrays, and you could store that key in your dataframe where you used to store the whole data
This might be the same concept as a foreign key in a database, I'm not sure. Maybe not. I don't know anything about databases. And little about pandas for that matter :P
 
6:50 PM
sounds foreign-key-like to me.
@LoopingDev Hmm, true. Never mind that idea then. What library are you using to insert values into the DB? Maybe it can insert the data and tell you the new file name with a single operation.
Lazy solution: don't make a file and just store the whole json string in the db too
I can't remember if this is the actual recommended way to store file-like data or not
Extreme laziness: don't store the json anywhere, just reconstruct it from the other columns if you need it again
 
@Kevin
flask-sqlALchemy

I only decode Json with this line
`data = request.get_json()`

and the file with that one :
`file = request.files['file']`
I've seen a post that you could add jsonObject to a form data as an array. using jquery.

but it looks like it would work with JS vanilla as well.

https://forum.jquery.com/topic/upload-file-and-json-data-in-the-same-post-request-using-jquery-ajax
 
google tells me that in sqlAlchemy you can get the id of a row you just inserted by using the row's id attribute. Try using that to name your file.
I'm becoming less confident that I understand what the goal here is, but I feel good about this .id thing
 
ok let me draw you a picture of what I'm planning to do, it would make it much simpler to understand
 
7:10 PM
Is there always exactly one file for each json object, and exactly one json object for each file? Maybe you can do this with one table instead of two.
Maybe I'm jumping to conclusions. What kind of columns does your File table have?
 
well, there are multiple tables that would require a physical filing to comply with the local government laws like Tax, Invoices ... etc.

so I'd need files for many tables and for now Yes it's just 1 file for each row but as we start actually getting into it, there might be several others for each process.

I think it would be the best to just separate those files in a different table and just link them with a foreign key.
 
Ok, reasonable.
 
@Aran-Fey what did you decide as far as OSes go?
 
It may be possible to have SqlAlchemy create one row for the json data and one row to the file, and it will fill out their foreign keys at the same time when it inserts them. I don't know specifically how though.
 
ugg...learning a JS framework that I only learned existed like a month ago...
 
7:17 PM
@Code-Apprentice should be a pro by now ;)
 
I think that will depend on the db setup @Kevin
 
yes Its but wont it be the best to separate the file in a different function instead of calling it even if there hasn't been a file attached to the formdata?
 
That makes no sense to me
 
If putting your logic into a new function makes things more difficult, then don't do it
 
@Dodge Debian was first on the list, but I gave up mid-install. (It was gonna spend 3 hours downloading stuff, and I really didn't want to wait that long.) After that I didn't have any more motivation to try another OS, but maybe I will tomorrow
 
7:19 PM
I meant like for the example Step1 in a function and if there formData has file in it, then it can call a new function to process ( step 2 )
 
So you assign the file upload a UUID and then store it under that name, and put a reference in the DB, no?
 
I usually don't bother having separate functions for each of my steps unless it's substantially better for readability
 
@Kevin that is the same reason why I want to do that , the upload function requires (7-10 lines ) , the json function would require (10+) , it would look a bit messy in my opinion if I left both of them together.
 
I wrote a thirty line function today, my power cannot be contained
 
@roganjosh yes Exactly , because I've just noticed that flask would overwrite the file if it has the same name,
I'd better make sure it has a unique name.
 
7:23 PM
The json function? SQLA can do all of this, but not in a single step
@LoopingDev well, there's no such thing as a "json function" so are we even on the same page?
 
@roganjosh what do you mean ? could a json object contain a file ?
 
@Aran-Fey if you do try ubuntu, I would be interested to know what you think about it given that it sounds like you have some experience with different unix like OSes. rhubarb, off to get vaccinated
 
@LoopingDev You seemingly have a function to receive the file and process it. It would be a good start if we can see that
Rather than debate whether a file can be serialized to x format
 
@roganjosh

at the moment I used the default ones
which you could find on the flask's page
https://flask.palletsprojects.com/en/1.1.x/patterns/fileuploads/
 
Sooo... what are we answering again?
Because, posting code from a guide is not too helpful if you haven't tried to apply it
 
7:32 PM
@roganjosh I applied it and it worked nicely.
as I mentioned, when I tested with two files of the same name, it just overwrites the old ones.
 
So give the file a UUID when you save it or a UNIX timestamp
I think that's all your question is about, then?
If you wanna update it in SQLA then you need to show the code that isn't working there
 
not sure what UUID is , but for now I am planning on rename it with ID of the newly added row of that was the result of the JsonObject.
 
A universally unique identifier (UUID) is a 128-bit number used to identify information in computer systems. The term globally unique identifier (GUID) is also used, typically in software created by Microsoft.When generated according to the standard methods, UUIDs are, for practical purposes, unique. Their uniqueness does not depend on a central registration authority or coordination between the parties generating them, unlike most other numbering schemes. While the probability that a UUID will be duplicated is not zero, it is close enough to zero to be negligible.Thus, anyone can create a UUID...
 
Good luck if anyone decides to reset indices of the DB
 
TLDR: it's a value that almost certainly won't get overwritten by another one
 
7:36 PM
I think it's much simpler than that, I just need to send both of them in the same request to be able to keep track of the new row's Id
 
> The term globally unique identifier (GUID) is also used, typically in software created by Microsoft.
*sigh*
 
so either I process the request as a JSON with FormData - file attached to it ( if possible )

or a formData with a JsonObject attached to it
 
No, now we're on different pages again
 
If the problem is simpler than generating a UUID, then why have we been trying to determine what the problem is for an hour
I don't think changing how the data is sent from the web page to your server will have any impact on how your server saves the data to the database
 
you helped me already. I know now what I have to do , am I ? ^^
 
7:38 PM
@LoopingDev you're jumping between backend storage and handling frontend submissions so you need to pin this down in what, exactly, you're asking
 
@Dodge Hi, I am not a professional linux user. I have it on my system and I play with it occasionally. The reason is that most of engineering softwares and the companies are windows based so I can only work with linux at home. I tried to install many linux OS's. The reason I did not go with ubuntu was that it seemed to be a little heavy on my laptop. I went to debian and it gave me best experience. It has no demands on my hardware. I am planing to learn enough to use more of its terminal.
 
@Kevin it wont, but I need to know the row's ID and then use that Row ID to store the file, if I splited the request into ( json and formdata with the file in it ) I don't know how am I going to pull the ID correctly in a different request.
 
@Dodge I've been avoiding this part of my code base for a reason
 
If you know what you have to do, that's good. My understanding is largely unimportant, because your program will run whether you explain it to me or not.
There is an old saying, "You don't truly understand something unless you can explain it to Kevin", but it's not mandatory
 
memory upgrade complete... numpy broadcasting, look out!
 
7:42 PM
@LoopingDev Sounds hard. Try to put them in the same request if you can.
 
lol!
I wasn't sure if it's the best approach, It's always better to hear the expert's before making a decision on something.
@Kevin yeah , that is what I'm saying , thanks a lot for the help :)
 
@enthu I'm unsure why you're replying to a message that wasn't directed at you or about your issues?
 
I think that was just ubuntu feedback
 
I'd put my database knowledge a couple pegs below "expert", but people listen to me anyway ¯\_(ツ)_/¯
 
@roganjosh you are right , I just wasnt sure if it was possible but then I asked around here and there and found out that it's possible to attach json object to a dataForm and If i'm not wrong, dataForm could include an input( with type: File )
 
7:45 PM
Getting import error when I try to import sqlalchemy from flask_sqlalchemy
 
@LoopingDev WTForms is aptly named, as far as I'm concerned
@RohanSingh ok. Why do you expect that to work?
 
@roganjosh it looks legit as well, yes. I should take a look at it as well. Thanks :)
 
Oh, it's certainly legit. I think you missed my point
I don't use WTF because it gets confusing
 
@roganjosh tbh I do tend to agree to an extent lol I have abandoned wtf for the most part other than for standard flask only forms as no need to but when combining with react its asier to go manual
 
8 messages moved to MetaPython
@piRSquared 128 GB or bust
 
07:00 - 20:0020:00 - 00:00

« first day (3818 days earlier)      last day (1130 days later) »