« first day (3078 days earlier)      last day (1870 days later) » 

2:00 PM
I wonder if there actually is a language out there where int(x) turns x into an int.
Got assigned a task today: "every database query made by the Welcome page should be properly parameterized". A reasonable requirement. But the Welcome page does not do any database queries.
 
sounds like an easy day
 
I can either 1) mark the task as complete, having made no changes to the code, or 2) get clarification from the client about the task. If I do #1, I'll get interrogated when the deployment process fails to find any check-ins by me. If I do #2, then I'll get interrogated when I claim that the Welcome page doesn't touch the DB, because it does several things that look like they ought to touch the DB
 
make a change to the code that is simply a comment mentioning the situation and then the deployment process will find a check-in
:)
 
2:16 PM
I'm not feeling Lawful Evil today, alas
I'm leaning more Stupid Good, so I'll probably do #2
 
Today I lost 2 hours of my life trying to import a csv file into a program. Living in the 21st century has never felt so great.
 
Management says: we're trying to be compliant with [obscure acronym] standards, which assume every page talks to the database. It's silly, but just... Add a query in, and make sure it's parameterized.
It's worth adding an extra thousandth of a second to Welcome page load times if it means The Auditors don't have any footholds
 
2:54 PM
cbg
im confused, the line
`df = pd.read_sql("SELECT * FROM parent_reply WHERE unix > {} and parent NOT NULL and score > 0 ORDER BY unix ASC LIMIT {}".format(last_unix,limit),connection)` returns the error in the next comment

even though I've confirmed that the table parent_reply does in fact exist
pandas.io.sql.DatabaseError: Execution failed on sql 'SELECT * FROM parent_reply WHERE unix > 0 and parent NOT NULL and score > 0 ORDER BY unix ASC LIMIT 5000': no such table: parent_reply
 
Let's try simplifying the problem. Does pd.read_sql("SELECT * FROM parent_reply", connection) also crash?
 
pd.read_sql("SELECT * FROM parent_reply",connection)

pandas.io.sql.DatabaseError: Execution failed on sql 'SELECT * FROM parent_reply': no such table: parent_reply
 
This at least tells us that the problem wasn't in the .format call or the resulting query string
 
the snippet you suggested required I add back in the connection, which is defined here: connection = sqlite3.connect('2009-09.db')
 
(insert typical disclaimer here about how you almost never want to use .format to parameterize sql queries, since the database library probably has a safer alternative)
 
3:00 PM
I'm still just barely learning SQL so I was sticking with the structured commands since I'm learning the queries as I practice it in python
sqlite3 2009-09.db
SQLite version 3.11.0 2016-02-15 17:29:24
Enter ".help" for usage hints.
sqlite> .tables
parent_reply
oops
 
Are you sure 2009-09.db exists? If it exists, are you sure that it's in the current working directory?
 
@Kevin i wish someone gave me that disclaimer when i started working. Never knew about it till very recently
also, cbg
 
maybe im being a retard and when I cloned my terminal i opened in a different dir
 
It can happen to anyone. I make CWD mistakes approximately one million times per project
 
i'd be surprised to find someone who hasnt made a mistake like that alteast once in their life while coding
 
3:03 PM
ok, looks like that was it, when i cloned my terminal in visual studio it navigated to the parent dir of my virtual env
 
nice, sounds like that's sorted then!
 
ty
 
@ParitoshSingh Cabbage. Meet Bobby Tables.
 
haha, yes indeed. We recently met. Cool guy.
 
btw guys, is there a nice way in to do something like this with numpy and pandas np.savetxt(f, df['parent'].to_numpy())
this works with df but df['parent'] is a series
 
3:14 PM
looks nice enough. df['parent'].values can give a numpy array directly if it is stored that way.
actually, if im not mistaken, pandas is basically stored off numpy arrays always.
 
yea, which is why i thought that would work, however I get this error: AttributeError: 'Series' object has no attribute 'to_numpy' when I type df['parent'].to_numpy()
ooh you said values
 
yep yep. full disclaimer though, i am not familiar with pandas much. Used it sporadically.
 
youre right about numpy arrays being a part of it though, which is what im trying to bank on here
 
@PM2Ring It was actually a conversation in this chatroom that mentioned parameterized sql queries. I've since then found this article that mentioned it, but the article linked there about how it all could be used was what finally drove the point across.
 
@ParitoshSingh That 2nd article looks great.
 
3:29 PM
I remember exploiting an SQL injection on my school's website and (after several steps) ending up with solutions to a large country-wide STEM Olympiad
 
@Aran-Fey >1 second per row is pretty slow. How big / complicated are these rows? Are you reading this with the csv module, or Pandas? I don't know Pandas, but I've never experienced slowness like that with the csv module.
 
maybe they're really BIG rows or something?
 
@vaultah Oops! How did the school react to that?
 
roughly 1k characters per row. It wasn't me who was parsing it, though, it was a terribly written website. I aborted the process after about 20 minutes, I don't think it would have ever finished
 
ah yikes.
 
3:35 PM
@Aran-Fey Ah, ok. So maybe it was using a hand-made csv parser, possible written by a cargo-cult coder...
 
I sense catastrophic backtracking
 
@PM2Ring well, I wasn't expelled :D But they were not happy, and soon they launched a new site on a new server and with a different CMS that had more-or-less adequate security. (because of how bad their old setup was, we also gained access to ~80 sites that shared the same hosting provider and CMS)
My friend who tried to use the solutions got caught, but he wasn't expelled either
 
@Kevin Sounds plausible. Although a little surprising when processing line-structured data...
 
Meanwhile there are students that get punished for "hacking" when they use CTRL+ALT+Down to rotate the display 180 degrees
At my high school they had to restrict access to cmd because we kept using net send to make popup windows appear on other computers on the network
 
@vaultah They should've been grateful to you for discovering the security flaw..
But I guess that depends on how quickly you informed them. ;)
 
3:52 PM
off to take the first drone flight of the season, got a good feeling about this year... rhubarb
 
guy, im trying to run a program in anaconda and after I click run it just jams the console
I think for some reason it cannot run if __name__ == '__main__':
is anyone familiar with this problem?
 
I am bothered that the word "should" is overloaded with mutually exclusive meanings. "The server should be up right now" can mean either "Based on available evidence I expect that the server is up" or "I observe that the server is down, but in a perfect world it would be up"
 
@Kevin :facepalm: That sort of thing tells the kids who do have actual hacking skills that the teachers are so clueless that they can probably get away with almost anything that doesn't actually demolish the system.
@Kevin You should be bothered by that.
 
I can, must, and will be bothered
@PM2Ring Yes, but first they have to change the command prompt styling to black text on white background, because white on black looks too hackery to any supervising adult.
 
does anyone here use vs code?
 
4:03 PM
@AdamG occasionally
cbg all
 
do you happen to know how to get it to format python on paste?
format as in indent properly
 
@Permian I don't know Anaconda, but that sounds a little odd. if __name__ == '__main__': is the standard "guard" used to execute code in a file only if the file is run as a script but not if the file is run by importing it.
 
I tried editor.formatOnPaste but it does nothing.
 
@Kevin you need to stop using English, then
@AdamG maybe it's doing as well as it can ;)
 
Generally speaking, it's impossible to automatically correct indentation
 
4:06 PM
false
 
@Permian Im guessing you mean spyder? I can run programs like that just fine. first thing to check i'd say: can the program run as a script without issues? Also, is the file saved somewhere? and you're using the button to run the script? (since you can run a python program in different ways in spyder. as a script using the run button, or in the interactive kernel)
 
*sigh*
 
@AdamG Is the original text indented correctly?
 
yeah, it just seems to be going back a tab instead of forward lol
 
@AdamG I suspect you need a MCVE
 
4:08 PM
of what?
the code? or my editor settings?
 
@AdamG Is the code using actual Tab chars, or tabs converted to spaces?
 
@ParitoshSingh I go on the IPython console (which is the only one now in newer versions of anaconda)
and click play
 
oh. hmm
 
i used to have this problem before and i would just run it in the python console, but this is not an option now
 
i am not quite sure what that refers to. Is that basically anaconda prompt?
 
4:14 PM
no Spyder IDE sorry
 
@AdamG whatever you need so that folks can replicate your indentation problems. Arguably, the smallest code sample that exhibits the failure-to-indent.
(both code in your file in vscode, plus whatever you're pasting)
 
ok spyder, got it. its weird. can you test a very simple python code, something that prints "hello world"
 
if they can't repro by copying and pasting then very clearly there's a settings/version mismatch
 
you shouldn't really have any issues unless the script itself is mucked up. FWIW, i've used if __name__ == "main" guard in spyder without issues.
 
if __name__ == '__main__':
print("hello world")
this works
@ParitoshSingh ^^^
 
4:16 PM
yeah, my guess is, something with the script is wrong
just a hunch, does your script use parallel processing by any chance?
i'd start commenting portions of the script away if thats an option, and seeing what breaks it.
 
@ParitoshSingh no but i know that is a problematic with the ipython console
 
yeah, i was hoping for an easy-to-spot culprit. I think you probably need to figure out if the script is the culprit. It may very well be.
 
def merge_the_tools(string, k):
    # your code goes here
    arr = string.split()
    for ch in arr:
        ch = "".join(set(ch.split()))
    print(arr[0])
    print(arr[1])
    print(arr[2])

    return

if __name__ == '__main__':
    string, k = input(), int(input())
    merge_the_tools(string, k)
 
(unless you can get it to run in some other method without issues. ) Why isn't running it through the terminal an option?
 
it all looks fine to me
@ParitoshSingh i have never done that lol
 
4:18 PM
^^ Ctrl-K to get fixed font, please
Not to late to edit previous post
 
yeah, just click edit, select all your code text, and ctrl+k. It fixes indentation in chat.
 
@ParitoshSingh done
 
No need to even select your code text, ctrl+k formats the whole post - fixed font is all-or-nothing on your post
 
Even faster: hit the Up arrow key to edit previous post(s).
 
Now we know who spends a lot of time in chat...
 
4:21 PM
haha
@Permian The script runs just fine. you might have been thrown off by the lack of "prompts". There's two input calls in your code. the code is waiting to accept inputs from the user
 
@PaulMcG that doesn't always work, I'm told
 
@ParitoshSingh so it just hangs?
 
Try this @Permian
def merge_the_tools(string, k):
    # your code goes here
    arr = string.split()
    for ch in arr:
        ch = "".join(set(ch.split()))
    print(arr[0])
    print(arr[1])
    print(arr[2])

    return

if __name__ == '__main__':
    string, k = input("enter the string:"), int(input("enter some number"))
    merge_the_tools(string, k)
 
would it be correct to say that the time complexity of Bellman-Ford on a positively weighted graph is O(|V||E|)
 
Maybe that will make it clearer to you. The code is waiting for input, because you (or your code) specifically asked it to.
 
4:23 PM
Unfortunately, my standard phone keyboard doesn't have arrow keys. And the Hacker Keyboard app only shows the arrow keys in landscape, which is not convenient, and takes up too much screen space.
 
@ParitoshSingh thanks
 
Another trick: when editing, you can link a post as a reply. Sometimes. ;)
 
no worries
 
That merge_the_tools function is a bit weird. Or was that just a silly example?
 
looks like a typical coding challenge thing, i think Permian is probably picking up coding through some website.
Thats probably an attempt in progress to solve some problem statement
 
4:36 PM
How cloud I insert a line break?
When I press enter it sends the message...
 
@X4748 On a normal keyboard, use shift-enter. Or compose the post elsewhere & paste it in.
 
@Aran-Fey Hmm. Certainly I agree that removing indentation is not unambiguously reversible. But I wonder if it would be possible to write an indentation corrector that merely tries to return a syntactically valid file.
#so
if True:
f()
g()

#gets corrected to
if True:
    f()
    g()

#and this may not necessarily match the original intent (maybe g() was outside the conditional),
#but we're happy as long as it doesn't crash at parse-time
 
@PM2Ring Thank you guys, It's a normal keyboard :D
 
oh, that's definitely doable
 
I wonder if you can do it in one pass? I suspect just greedily indenting as much as you can at every step may lead to dead ends when you encounter an else or a yield or something.
 
4:41 PM
making it valid is easy... making it correct is hard :P
 
Or maybe it's lazily indenting as little as possible that leads to dead ends. Or both!
 
Hmm. I don't think anything can go wrong if you indent at every line that ends with a colon
 
I haven't come up with a counterexample after thinking about it for 100 seconds so I might agree with you
 
If you encounter an else: or elif: or except: or similar you may have to drop more than 1 level of indentation, but there should still be a way to make everything syntactically valid
 
I think even just making it valid requires backtracking, you can't just process it line by line.
 
4:44 PM
Is that possible to import files from a subdirectory like this following tree?

/project
/helpers
__init__.py
stringHelper.py
databaseHelper.py
index.py

When I import liks 'this import Helpers.databaseHelper as DB' It says

ModuleNotFoundError: No module named 'Helpers'
 
1) Capitalization matters
2) Is `helpers` on your PYTHONPATH?
 
more inits everywhere!
 
I want to say "try putting an __init__.py in the project dir too" but I'm not confident that does anything
 
@Aran-Fey Sry, I forgot to capitalize the first letter, helpers directory is Helpers
@Aran-Fey Hmmm, No
@Kevin I have done that before, It doesn't matter
 
the question is, where are you attempting to import it from?
 
4:49 PM
@X4748 Is /project empty, and all those .py files in /Helpers ?
 
is that import statement in the index.py file? or somewhere outside?
 
Hint: try using ctrl-k to ensure a multiline chat message retains its indendation
 

Sandbox

Where you can play with regular chat features (except flagging...
 
 
C:\Users\Kevin\Desktop\project>tree /f
Folder PATH listing for volume OS
Volume serial number is 6E1D-CD3F
C:.
│   main.py
│
└───helpers
        databaseHelper.py
        __init__.py


C:\Users\Kevin\Desktop\project>type main.py
import helpers.databaseHelper as DB
print("If this line executes, there wasn't an import error")
C:\Users\Kevin\Desktop\project>main.py
If this line executes, there wasn't an import error

C:\Users\Kevin\Desktop\project>
On my machine, a single __init__.py is sufficient to be able to import the directory as a module.
 
4:55 PM
/project
    index.py
      /Crawler
         Crawler.py
         __init__.py
            /Helpers
               databaseHelper.py
               stringHelper.py
               __init__.py
@ParitoshSingh that's in Crawler.py which it's a module itself
 
If I put helpers inside a directory Crawler, then I can't import it from the main project directory unless I do import Crawler.helpers.databaseHelper as DB
... But I can still do import helpers.databaseHelper as DB if I'm doing it from a script in the Crawler directory.
Since my project directory seemingly matches yours, I'm confused that mine works and yours doesn't.
 
@Kevin This is exactly what I'm trying to do, But I get ModuleNotFoundError: No module named Helpers ):
 
Times like this I wish it wasn't a pain in the butt to create MCVEs that involve multiple files and directories
 
I'm having a problem with a matplotlib plot, one time it does not use tex even though it's specified but it does use it on a second plot. On that first plot (where it does not work properly), it seems it does not use it for the first graph ticks for both axes neither the labels, but it seems it uses it for the consequent ticks after the first one. Any ideas?
 
5:10 PM
@Kevin It worked, Thanks
 
@Kevin this is why I'm somewhat forgiving about MCVE in . Just the nature of android dev requires like 10 files just for the simplest complete app.
 
With a little bit of elbow grease, one can write a single-file script that creates some directories, creates some files in those directories, and populates those files with text. By uploading this "project generator" script to pastebin or similar, other users can easily reproduce your project directory.

Project idea: create a project generator generator that examines an existing project directory, and creates its equivalent project generator.
For example, pastebin.com/HLNVT36X is the project generator for the project I created for X4748's problem. Writing it manually took me about five minutes; it would have saved me some time if there was an automatic approach.
"But surely you could just upload a zip or tarball?" you say. But those formats are 1) not human-readable and 2) require specialized software*
(*never mind that said software is unbelievably widespread and easy to use)
 
5:29 PM
@Kevin Even in this simple example, you have to guess whether g() should be indented or not. Colons only suggest when indentation should be increased, I'm not sure what hints you could follow to know to outdent
 
I have a Python script that prints an indented directory tree listing. It can also read such a tree listing to recreate the directory structure wherever you want, all the new files are empty. But it's a few years old, so it probably doesn't use pathlib, it may even be Python 2...
 
"outdent only when you encounter else, elif, except, finally in order to match them with their accompanying if/try tokens" catches the majority of cases I think
if you're trying to do this in one pass you'll need to maintain a stack of all the blocks you're inside so you can efficiently find the closest if/try
I would go forward with this project generator generator idea if there was a version of repr that returns triple quoted strings
I want triple_repr("foo\nbar") to return '"""foo\nbar"""'. That's right, the newline isn't escaped.
 
Now I get "TypeError: 'module' object is not callable" l:

Google says:
The message TypeError: 'X' object is not callable means that you wrote expression(some_arguments) where expression is an instance of the X type, and this type doesn't support to be used in a function call syntax. Most of the time you wrote this because you thought expression was a function or some other callable type

It was working perfectly before importing and putting in a separate file, I try to pass arguments to the __init__ function
 
@X4748 If you're doing import Crawler.helpers.databaseHelper as DB, followed by DB(), then that will certainly raise a "module object is not callable" error, because DB is a module and not a class or function
if you have a function or class named DB inside the module, then perhaps you want to do from Crawler.helpers.databaseHelper import DB
 
@Kevin Are you having a conversation with yourself?
 
5:37 PM
I solved my issue by calling the plt.rc commands before the figure, if anyone cared...
 
Only one third of the conversations I'm in right now are with myself.
@mariogarcc Sorry we couldn't help much. Next time for sure!
 
@Kevin The class name is the same thing as the file name
 
I just tabbed over to see multiple messages from you, Kevin
 
Yeah then I definitely advise using the from whatever import thing syntax
 
@mariogarcc I was going to ask for code, but I see you've fixed it
for future reference, plt.rc stuff set defaults, which is why they need to be called/set before doing anything it should affect
 
5:40 PM
@Code-Apprentice [plate spinning music intensifies]
 
What qualifies as "plate spinning music"?
 
Dancing music from "Can-Can" - duh, duh-duh duh-duh, duuh, duuh, duh-duh duh-duh, duuh, duuh, duh-duh duh-duh, duh, duhduhduhduhduhduhduh (repeat <) (where '<' is the musical crescendo instruction, not the mathematical operator)
 
Sabre Dance is the canonical tune but I might also accept breakfast machine or maaaaaybe Powerhouse
 
@X4748 The (modern) Python convention is to make module names all lower case. That stops you from confusing modules with class names, which should be CamelCase, but it doesn't stop clashes between module & simple function names.
 
@PaulMcG ambitious
 
5:45 PM
Note to self: along the lines of JPEG->ASCII conversion, write sheet music -> ASCII conversion function
 
@PaulMcG It may already exist. See ABC notation
 
Sabre Dance: duh-duh-duh-duh-duh-duh-duh-duh, duh-duh-duh-duh-duh-duh-duh duddle-it-duh-duh, duddle-it-duh-duh, duddle-it-duh-duh, duddle-it-duh-duh, duhduhduhduhduhduhduhduhduh
 
I'm half tone deaf but I'm tempted to accuse breakfast machine of being a cover of Sabre Dance with different instruments
 
May need more ML to automate conversion of "duddle-it"s, etc.
And when should be "duh" vs "nuh"
 
I lied. My code only handles directories. stackoverflow.com/a/33914896/4014959
 
6:03 PM
@PaulMcG Well that was a wild ride.
Did I forget Flight of the Bumblebee? You can spin a mean plate to that.
 
@Code-Apprentice Looking back, is this also an inquiry as to just what is "plate spinning"? Raise your hand if you know what this is AND are too young to have ever watched Ed Sullivan.
 
"I have 5 folders with 11 .json inside and around 19 jsons inside the .json files"

wat
 
"19 dicts inside the JSON files", probably
maybe
who knows
 
that's what im imagining..
 
Too many people think they've got json when they've got JSON Lines
 
huh TIL that's actually a thing
 
6:24 PM
One more last question, Why am I getting this error ("name 're' is not defined" ) when I have imported re already?
 
[1]\n[2] is not valid JSON but it is valid JSON Lines
 
@X4748 need to see your code.
 
@X4748 Wild guess: you're trying to use it in a file other than the one that imported it.
 
Michael Fairley says:

You need to have the import statement in mapfn itself. mapfn gets executed in a different python process, so it doesn't have access to the original context (including imports) in which it was declared.
 
what does your import look like
 
6:25 PM
If a.py imports re and b.py; and if b.py tries to use re.match, then b.py will crash because it can't access the modules that a.py imported
 
each module can only use the things it imports or defines itself.
 
"process"? Uh oh, I don't know a thing about multiprocessing.
 
import requests, re
from Crawler.Helpers.databaseHelper import DatabaseHelper
from Crawler.Helpers.stringHelper import StringHelper
from Crawler.Helpers.dataLogHelper import LogData
from bs4 import BeautifulSoup

    def extract_emails(self, text):
        emails = set()
        # emails.update(new_emails)

        try:
            emails.add(re.findall(r"[a-z0-9\.\-+_]+@[a-z0-9\.\-+_]+\.[a-z]+", text, re.I))
        except Exception as error:
            print("[extract_emails]: " + str(error))
 
what is actually calling your extract_emails function?
 
@AlexanderReynolds It's called in init function
class Crawler:

    def __init__(self, data):

        self.data = data
        self.databaseName = 'crawler.db'
        self.databasePath = self.data['basePath'] + "/" + self.databaseName

        # Save link
        DatabaseHelper(self.databasePath).insert({'link': self.data['target'], 'status': '1'})

        # Get plain links
        print(self.data['target'])
        content = self.get_content()
        links = self.extract_links(content)
        emails = self.extract_emails(content)
 
6:28 PM
does your folder of current working directory have a python file named re.py by any chance? EDIT: nvm. that wouldnt behave that way.
 
Ignore other lines
@ParitoshSingh Nope
 
this is SO's canonical "how do I make objects of my class iterable" question? Really? Am I bad at googling or this is some kind of sick joke?
 
Is there any possibility that you have a circular import? a.py imports b.py which imports c.py which imports a.py? These can produce truly baffling behavior.
 
@Kevin Hmmm, I think yes, I'm running a module from the index.py, In that module I import another class which has that error
 
To be clear, it's OK for index.py to import foo.py which imports crawler.py, as long as crawler.py doesn't import index.py
 
6:34 PM
Sry, Error accures at Crawler.py:

/project
    index.py
      /Crawler
         Crawler.py
         __init__.py
            /Helpers
               databaseHelper.py
               stringHelper.py
               __init__.py
and I sent you Crawler.py
Crawler.py is being called at index.py
 
Hmm. With just the information available to me, I don't see anything obviously wrong.
 
wim
camelCase is obviously wrong :P
 
wim
@Aran-Fey hm, that looks like a reasonably good description of the iterator protocol to me? what's the problem?
 
@wim I fixed it lol Because I sent you this one I decided to dont't change it
 
6:41 PM
@wim __iter__ and __next__ are different things and don't need to be on the same object of course, so I do find it lacking in that understanding.
A cursory read of that might be "oh I need to have an iter and a next method and the iter should return self"
Meh I didn't even describe that very well but I think you can get my point. The top answers don't do a good job of separating iterators and iterables.
 
@wim Not only is it unclear (what the heck is an "iterative function"?), but it's also asking two different questions. It should either ask how to write an iterator, or how to write a generator function, not both.
 
The answer to "how do I make something iterable" isn't "make it an iterator" IMO.
@Aran-Fey Well you can propose an edit to the question, no? :)
 
wim
@AlexanderReynolds true
 
... Is it bad that I typically just yield values straight from __iter__
 
you do you kevin
 
wim
6:46 PM
I think it's ok, because calling it will return a generator which is an iterator
 
To be clear, I'm not saying that a question asking "What are all the possible ways to create something iterable" shouldn't exist, I'm just saying that it shouldn't be our canonical dupe for "how do I make my class iterable" questions
 
@Aran-Fey I agree that it is a different question.
 
wim
well, I sometimes just return return iter(self._mystorage) which seems equally as dirty
if you need a concise one for difference between iterator and iterable I have this
 
I'm asking too many questions I'm sorry, But how could I stop all threads when something did happen? using the threading module, I have searched a lot about it but I didn't find a straight answer!
 
You could create a thread communication object, such as an event, which you set whenever you want all threads to stop. Then every thread could periodically check the state of the event and stop when it's set.
This requires you to manually insert a bunch of if stop_event.is_set(): sys.exit(0) all over the place
"But what if I don't want to rewrite all my threads? Can't I just kill them without their cooperation?" you ask. stackoverflow.com/questions/323972/… says, "not really no"
 
7:04 PM
just pull out the power cord.
 
wim
Q: how many psychologists (programmers) does it take to change a light bulb (to stop a thread)?
A: only one, but the light bulb (thread) has to want to change (stop)
 
@Kevin I have a simple thread that keeps running till Queue.qsize() > 0, But threads never stop on time
 
when i load a plugin, or a file using a CDN, the CDN provider get the IP of the my website or the IP from the machine that's requesting?
 
I might expect this to happen if the thread that suggests username/passwords is distinct from the thread that tries username/passwords to see if they work, but that strikes me as a very unusual design
 
I tried to read status from a file before running the loop, But didn't work either
 
7:57 PM
@PaulMcG Yes, I know what plate spinning is. I'm too young to have watched Ed Sullivan when it was on air, but not too young to have watched reruns.
so my question is what music is appropriate while spinning plates?
 
Depends. Can you headbang while spinning plates?
 
sure
I mean the plates might fall, but I can still headbang while doing it...
 
oh. Well, some music that goes well with the sound of shattering porcelaine might be appropriate then...
dubstep, maybe? idk
 
8:24 PM
Mates how can i convert a python django view to a string .
covert django view to json string
How can i pass a python django view to a json data .
 
Hey, So I have a very dumb question since I'm not horribly well verse with Excel and I'm working with Python to decipher data between two files. Does anyone here have reasonable experience with openpyxl to help?
 
I have used it several times. Whether that constitutes "reasonable experience" is doubtful. You would be better asking the question and seeing if anyone responds
 
I've opened and saved a few workbooks with openpyxl, does that count?
 
Do you know how to compare two books for relevant entries that have Zero comparable formatting. They have the same types of Columns but the exact location of the columns is not the same as in my initial workbook. Like I want to search workbook 2 for entries that possess similar data values for the Job and Job ID, but have different Statuses.
What I mean by formatting is in workbook 1 the columns are arranged in one order but are not arrange in any similar way in workbook2
 
That part wouldn't matter in itself, but "similar" raises numerous issues. How do you intend to compare similarity?
 
8:37 PM
Trying to work with data that's stored in an unknown format sounds all sorts of problematic
Why do you have to work with a workbook that has its data in an unknown location? If these are 2 specific workbooks, I would open them up, see where the data is stored, and hard-code those locations into my program. If you're trying to support arbitrary workbooks, I'd recommend not trying and forcing them to adhere to a certain well-defined format instead
 
Its not arbitrary just for whatever reason the source of Workbook2 has things arranged differently in terms of order of columns and the presence of new ones.
I want to search Workbook2 Row-by-Row and find relevant matching Jobs and JobIDs and then perform a function based on the JobStatus. I have what columns these are but I feel like I cant just iterate through it like that.
 
Well, wouldn't job ID be expected to match, not be similar?
 
There are multiple JobsIDs for one Job and sometimes the JobID varies by a digit or two. 21 is diferent from 210 and 021 or 0210
 
So there is a pattern where there may be a leading or trailing zero, but that would be the only difference?
If the difference can be narrowed down to some reasonable rules, you may be able to match them with only a few false positives
 
Yeah, My previous use of the word similar was meant more towards the arrangements of the columns and the presence or lack there of specific ones.
I apologize.
 
8:48 PM
Where are you up to with the problem right now?
As an aside, I really think what you're trying to do is well beyond the scope of openpyxl. You will need that to read the files in in some reasonable format, but after that I think its work is done and you need other tools
 
All I need to do with the entries after this comparison is increment a counter for display depending on what was found and delete entries from Workbook1, If Job 54 with ID: 0001 is marked as a C I delete the corresponding Row from Workbook1,
 
This is a classic Step 1, Step 2, all I need to do, profit???
I was asking first where you are actually up to in trying to solve the issue. Do you have your files read in, as a starting basis?
Also, are you able to use Pandas here?
 
I can use Pandas I haven't used it before, and Im at the step for checking the Rows but I've done comparisons and sorting in a limited sense with a single files and the use of Sheet.iter_rows() I'm not clear on how to iterate through the second.
 
@Aran-Fey re: your meta question, considering the spirit of the question is working with broken JSON data, I think a better idea for a title to that question would've been "How to deal with a JSONDecodeError when parsing JSON data?" or something of the like.
That would communicate intent more clearly.
 
Ok. First thing, use df1 = pd.read_excel('excel_file_one.xlsx') and same for the other file. Then print df1.head() and same for the other. If the files don't have tonnes of columns then perhaps post the output here, but if it's going to be over loads of lines then use something like pastebin for the results.
 
9:05 PM
I got a unicode error
"df1 = pd.read_excel("C:\Users\USERDesktop\Reports\untimed_fixed_dates.xlsm")"
 
Try df1 = pd.read_excel("C:\Users\UFJUDFM\Desktop\Untimed_Reports\untimed_fixed_dates.xlsm‌​‌​", encoding='cp1252')
 
Add encoding='latin-1' as an argument to read_excel, perhaps that'll do it.
or that ^^
 
still errored*
(unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UNXXXXXXX escape
 
With both of those encoding options?
 
Yes
df1 = pd.read_excel("C:\Users\UFJUDFM\Desktop\Untimed_Reports\untimed_fixed_dates.xlsm‌​‌​", encoding='cp1252')
 
9:08 PM
openpyxl must also have the same issue?
I was sure that was the underlying library used by pandas
 
Its not had that issue I iterated through the file before to count the number of jobs with a certain program before.
 
whats a better IDE platform for python: Pycharm or Anaconda?
 
@MartijnPieters what am I missing here? What protection is an f string giving?
The table name and columns must be formatted into the string, but that requires validation; those could still be vectors of attack, no? I'm more taking issue with just throwing string formatting at the entire query, where several of the values we know nothing about
 
@roganjosh it is no different from the %s string interpolation the OP was already using.
They are interpolating a table name and headers. You can't use SQL parameters for SQL object names anyway, the point of parameters is to prevent data from being used as anything other than data.
 
Exactly. And I repeated the comment under the question too. Maybe I'm misreading your comment, I thought you were saying I was incorrect
 
9:15 PM
So either they use a 3rd-party library (like SQLAlchemy), quote object names manually, or just trust the data.
 
The table name and columns can be cross-referenced with the schema before allowing the formatting. I was half putting together an answer and then stopped at your comment :)
 
The point is that the format string is no different from what the OP already had, no need to berate the answerer for a mistake the question-asker is making, which is not split the values, probably.
 
I wasn't intending to berate the answerer. They posted a continuing issue all over the SQL tags. I'm not sure how I've commented incorrectly.
A genuine question; how is my tone any different to yours on a tab I happen to have open? My comment was for a genuine security risk.
@RobertFarmer What version of Pandas and openpyxl do you have?
 
Latest Ones i believe
Panda I downloaded today and openpyxl monday
sap = self.wb_sap.active
wt = self.wb_wt.active
for saprow in sap.iter_rows():
for wtrow in wt.iter_rows():
if (saprow[3].value == wtrow[4].value):
counter+=1

Would this work for what I want?
 
xlsm means that the file contains macros. I don't see how they could be screwing with the reading but possible
@RobertFarmer not quite, because it doesn't handle the possible issues of JobID having leading or trailing 0s
 
9:26 PM
Yeah this case its comparing the JobTitles which should be the Same Strings with Numbers like IT1435723 or AR223084 or EX39458 and so on
 
Hold on, so JobTitles can be expected to have exact matches?
 
Yeah, Theres a JobTitle but it Has x rows with different IDs and Statuses
 
Or you're just going to catch the ones that happen to have exact matches?
 
I want to search and find the Exact Matches for the Title and ID and keep/delete the Entry in Workbook1 based on the Status in Workbook 2
I have an Idea about the IDs but I have to go, It's the end of my Work period.
I hope I can pick this up sometime Tomrrow with you or someone.
 
listing_count = {}

sap = self.wb_sap.active
wt = self.wb_wt.active
for saprow in sap.iter_rows():
    for wtrow in wt.iter_rows():
        if (saprow[3].value == wtrow[4].value):
            listing_count[saprow[3]] = listing_count.get(saprow[3], 0) + 1
Try that to see if it can identify repeated JobTitles
 
9:32 PM
Oof Its either chugging along or it did nothing
Theres approx 8000 entries i the files..
 
8000 * 8000 values to get through, but it's no more complex than what you already had
 
Yeah
its not doing anything, I restarted it and put a print in to make it print a period during every loop. not ther.e
 
There are more efficient ways of doing this, I'm more curious first to confirm that you can actually get matches
 
how do i input code?
With formatting I mean.
 
for more than a handful of lines use a code paste service
 
9:34 PM
its 9 lines
I want to post my function
 
OK, then just format it according to the link (complete with practice in the sandbox if necessary)
 
def check_rows(self):
    listing_count = {}
        sap = self.wb_sap.active
        wt = self.wb_wt.active
        for saprow in sap.iter_rows():
        for wtrow in wt.iter_rows():
            if (saprow[3].value == wtrow[4].value):
                print('.')
                listing_count[saprow[3]] = listing_count.get(saprow[3], 0) + 1
    print(listing_count)
Boom!
Success for me for once.
 
mind. blown.
 
I count this as a win, I don't get many so Ill take them however small they may be.
 
for what it's worth formatting in SO chat can be a pain, so this is always an achievement
 
9:37 PM
My suspicion is that you forgot to call the method. If you don't think you did forget, put a print('hi') or something as the first thing it does
 
@RobertFarmer you have an IndentationError though
 
If it isn't a case of forgetting to call the method, then sap is likely empty
 
C:/Users/UFJUDFM/Documents/untimed_report_latest.xlsx
C:/Users/UFJUDFM/Desktop/Untimed_Reports/Work_Tracker_List_March212019.xlsx
these are the directory I get when I connect to the files im using askopenfilename to click them.
Okay Its not doing anything in the If the IF statement isn't catching any matches
 
I think this is going to be too much to keep chugging through on debugging here
 
Hold Up I figured it out..
Row 1 in Workbook is used as Headers and in and Column E in Workbook 2 is not the same Column E as Column 4
So I need to iterate starting at Row 2 -> X and then Figure out why Column E is not 4
 
9:46 PM
That might be a non-trivial task. This sounds like my hell trying to parse spreadsheets a month or so back. I think some guesses are taken at the format of sheets that have, say, a row with a single value vs. all the subsequent rows, or merged cells
 
There iteratating through the same workbook.
 
Somehow it ends with a 2D list but it's not always clear how it generates it because spreadsheets can have all sorts of weird structures
 
Workbook 1 is iterating through the first row which is all headers and then wtrow which should be iterating through the second excel doc is iterating through work1's correspond column#
 
You would only need to iterate a couple of lines through the second file before breaking out to see what column 4 refers to
 
I dont get why im its limiting me to workbook1 when trying to iterate
Im never entering workbook 2 to compare
I really have to go now, We will have to pick it up tomorrow . I apologize.
 
9:53 PM
Good luck
 
@coldspeed At the time I just wanted it to have an accurate yet completely useless title so that nobody would ever find it again. Feel free to change it.
 
10:08 PM
Ah, if that was the intention, I'll leave it alone :D
 
Well, it's improved a lot
*shrug*
 
It really has improved a lot
Nice effort kicking it off, Aran-Fey
 
Thanks :) There are a lot of popular questions that I'm unhappy with, but that one was in a completely different league. Leaving it alone would've been a crime
 
10:33 PM
This question reminds me of delete an element from a dictionary, in which the original intent of the question was asking how to delete an item without altering the original (i.e., implicitly return a copy), but became popular because people just wanted to know how to delete items period.
That one never made it to meta, but I do remember @smci bringing it up here on chat.
 
In that case, though, the title is clear and the copy part is an "Additionally," so it's far more useful in search results
 
Yes, that was edited in later, after said discussion with smci. The original question remained misleading for years...
 
Ah, sorry. I should have looked at the history first considering you were drawing parallels and this is all about face-lifts :)
 

« first day (3078 days earlier)      last day (1870 days later) »