« first day (3854 days earlier)      last day (1079 days later) » 
03:00 - 22:0022:00 - 00:00

jrh
3:01 AM
Coming from development that's been outside of stuff that involved ORMs, is the idea of frameworks like Django that I store all application state in the database? Like I stop using my own classes and turn everything into a Model (and therefore a relation/table)? Or is it more like a "use the database to store information that you'd normally store on disk in a desktop application" thing?
I read up on normal forms and it seems like it'd take something like 3 tables to store configs that reference a list of ordered items, it's been an interesting puzzle to figure out, but I can't help but think it'd just be a class with a string array field if I did it without using models, and I'm not sure if anyone would really want to trace through 3 layers of IDs. I can see why databases would be amazing for bulk storage, but for a handful of 8 char strings?
 
@jrh Yeah I wouldn't be inclined to use it for that, unless that list of strings is custom for each user or something, in which case...user data should probably be in a database?
 
jrh
@toonarmycaptain there's no real 'user data' at all, I'm looking to make an app to configure the way the server works, as sort of a web frontend to it
there's more to the server than just a web server
I was thinking something like JSON strings would probably be good enough for ordered lists of strings
academically it's not really using the database correctly, but pragmatically the overhead of the DB would be very massive compared to the benefits of good relational DB design and the model is uh.... bureaucratic and hard to understand; you can't just look at one table and see what's going on at all even though it's pretty much just entries = ['one', 'two', 'three', 'four']
It's a bit weird for me because usually I think of DBs as massive storage spaces for long term important data of uniform shape, but the tutorials make it sound like Models are just helpful abstractions to retrieve page data and show it; in my case some data is important and I'll want to store it, but it doesn't feel right to make all my data structures using Models. It's like I'm trying to fit my program into an excel table.
I'm finding more relevant search results now, maybe I wasn't searching for the right thing, or Google magically started being helpful
A 2-D array is a great example of something that wouldn't fit well within a model and something I might need to store, good to see I'm not the first one to run into this
 
 
3 hours later…
6:48 AM
Hi everyone. I have a quick question. I will have a singleton class that is expected to store variables that can be accessed anywhere in my project. To be clear, I will read lots of arrays from different files and pass them to the functions that will modify these data on my purpose. Since I don't want to make these heavyweight computations over and over again, I want to store then in a singleton class. So what should conventionaly I choose a name for this class?
(e.g. 'class DataContainer' or something else)
I guess there is no consensus over this topic in PEP
 
7:00 AM
is this just a class that acts like a container? or an actual Singleton class? in any case that name looks fine to me
 
@NickAlexeev When a and b are mutable sequences, then a[:] = b syntax means "replace everything in a with everything in b. That implies resizing a to hold exactly as much as was in b.
See Builtin types: Mutable Sequence Types. It's the case s[i:j] = t, with i and j implicitly being start/end when omitted.
@black You might just want to use a module for that. Modules are inherently singletons.
 
hey guys anybody have eperience with tika and python? keep receiving this error (was functional yesterday and not sur whats gone wrong dpaste.com/GCKAPZGCM
error seems to be coming from within the library itself
not sure how to debug it
 
@Kwsswart Not in specific, but did you create the fpath yourself?
It looks like tika expects you to give it a JSON, but what it finds isn't a JSON.
 
yeah when printing the fpath it is C:\Users\name\Documentos\workspace\dbug\files\2014\BK002735_5410506_95156026.pdf
the file is pdf as I am parsing pdfs
the error is arising from here within the tika code github.com/chrismattmann/tika-python/blob/master/tika/parser.py
within the def _parse( function
which I can only think that for some reason it is not receiving the json from the tika api
but as said was working yesterday so really confused
-,- just realized tika cannot be accessed via proxy
 
@MisterMiyagi I know modules also works well for this usage but I will use some features that 'class' provides. (using @property, @setter, adding some other methods etc.). So I decide to use 'class' instead of module. Thanks for your answer.
 
7:13 AM
@Kwsswart Yeah, that's true. I'd recommend attaching a debugger to see what the response is. (Unless the proxy issue is already the cause.)
 
@MisterMiyagi Yeah disabled the proxy and functioning perfectly
had active for other reasons lol
 
 
2 hours later…
8:49 AM
I found the culprit of my program not terminating on sigterm from supervisor. It's ros2 again, ros2 launch doesn't somehow relay signals to the nodes it starts, I assume it creates a new pid for each node. Then supervisor closes the launch file which then leaves the node hanging. Well luckily I could just call the executable directly. The less ros the better
 
Silent Orphan Zombie Child strikes again!
 
9:04 AM
execute_children comes in handy again
 
9:15 AM
@AndrasDeak Gosh. We have someone run that on our prod cluster. Too bad it's a shared account and they execute all children.
 
 
1 hour later…
10:39 AM
"execute" is a nice auto-antonym, since it can mean both "put into motion" and "kill"
I executed my program, but after it entered an infinite loop, I had to execute it
 
11:02 AM
Is there a word for imposter antonyms, like "flammable/inflammable"? Words that should be antonyms, but aren't.
 
anto-antonyms? ;)
 
english.stackexchange.com/questions/19197/… lists some other inflammable-esque word pairs, but I don't see a formal term for the group
 
Trying to think of others. "genius/ingenious"?
 
"press/depress" --ooh, I hate this when I'm binding events to GUI buttons
 
release?
 
11:07 AM
My tech writer brother always cringed when getting raw material from engineers describing a critical part: "Do NOT unloosen this bolt!"
 
"release" is fine, but I have to navigate through the maze of depress/de-press/unpress each time to get there
 
ravel/unravel is a lightbulb for me because I had this vague confusion about that (there's a ravel in numpy, which does what I thought was unraveling)
I never articulated this confusion to try and investigate the meanings
 
I like Ravel's music, but I don't like Unravel.
 
Unravel's famous Unboléro?
(I never knew there was an accent in there)
 
Similarly, I'm irritated that "undoable" could (in principle) mean both "an action that can be reversed" and "an action that cannot be done". In our line of work it usually means the former, but there's always a chance...
 
11:10 AM
Or the unlistenable Pavane for a Live Princess
 
And I get dizzy whenever I need to implement "an action that cannot be reversed". My brain says I should refactor class UnUnDoable(Action): into Doable(Action). No!
 
hello, does anybody know about the issue with google drive api when redirect to localhost?
 
A recently evolved near-auto-antonym: once colloquially understood as "A limp, ineffectual, or uninteresting person", the word "drip" has taken on the additional meaning of "(of an outfit) cool, fashionable"
 
@dmitriy_one hello. Please don't ask for help here with fresh questions on the main site as per our rules
 
11:30 AM
ugh, closing your browser window with 20 important tabs and then realizing you have another open with 1 useless tab and then on reopen instead of opening all the tabs it just opens that 1 and you have to reopen all the tabs, so annoying, I wonder if there is a fix for that
 
Trying to find a cleaner symbolic solution to my matrix rotation problem... atan(sin(theta) * cos(theta)) doesn't simplify any further, right? If it helps, 0 <= theta < 90 degrees
 
@Hakaishin does ctrl+shift+t not work across windows?
 
@AndrasDeak sadly no
 
@Kevin probably not. That's atan(sin(2*theta)/2) which is barely better.
 
Ok, thanks for the sanity check 👍
 
11:34 AM
It is, however, suspicious that you ended up with this formula
 
It does strike me as odd, but the numerical results have looked good so far, so
Here is my MCVE, if you're curious.
I'm a bit surprised that sympy didn't simplify sin(Q)**2 + cos(Q)**2 to 1 in the few places it appears
On reflection I guess this is a "draw the rest of the owl" algorithm since I didn't explain how I derived the haruspex's formulae
 
@Kevin that's very close to a bug that was fixed a few years ago... it would simplify 0.1*sin(Q)**2 + 0.1*cos(Q)**2 to 10 :I
I don't see sympy.simplify anywhere, so no wonder it doesn't simplify
 
Basically I took shear_y(a) * shear_x(b) * scale(c,d) and then, uh, inverted it or something?
(* here being matrix multiplication)
@AndrasDeak Why would I ever want to do things the simple way ;-)
#Ok ok, I'll throw in some simplify()s...
scale_x: cos(Q)
scale_y: tan(Q)
shear_x: -sin(2*Q)/2
shear_y: 1/cos(Q)
skew_x: atan(sin(2*Q)/2)
skew_y: -atan(1/cos(Q))
 
11:49 AM
better
 
Aw, Desmos got me excited because atan(sin(2*Q)/2) looks very similar to sin(2*Q)/2 when you're zoomed out
Presumably atan(x) ~= x for small x
 
yup
sin(x) ~ tan(x) ~ atan(x) ~ x for small x
sinh(x) ~ x too for that matter ;)
 
12:41 PM
guys anybody know of a way to host private libraries? I have made one that I want to have access for the company I work for but cannot put it on Pypi as its public
 
Private git repos so that only the people who can see it can clone it
 
hmmm but would this allow the likes of pip install for functionality or no?
 
No, if I remember correctly, pip install works on pypi. There might be a way to configure it though
 
You're right that pip works on pypi, but it also works on more than just pypi
For instance I know for a fact that it can install .whl files from your local drive
pip.pypa.io/en/stable/cli/pip_install says it can install from "Local or remote source archives". I suspect a git repo counts.
 
pip supports local directories, git and other sources for packages. But you can also spin up your own warehouse.
 
12:48 PM
If your company already has an existing version control system, you should probably use that
 
currently using bitbucket but no way to organize using pip to install custom libraries and would be useful
 
So you can configure it...
 
Thats very useful thanks guys
 
jrh
1:08 PM
After a night's sleep, thinking a bit more about Django I think it might be ideal for websites that display either A) static content or B) content entered in by users and stored in a table, but that's not what I'm looking for. It seems like Django didn't really design its Model API to easily handle classes that don't fit into a SQL table (e.g., a 3d array of doubles); it might be telling in that some blogs say data like that should go in static files on the server
I certainly could store serialized numpy data/etc. in static files, but if the whole point of using a server is to store the data and information in a way that's related to some key, in an efficient, scalable manner, as in, "look up the results of this test", it doesn't quite add up to say "this can't be a class / we can't use numpy because it doesn't fit neatly into a rectangular table".
 
yeah I think that is the status quo for image storage for django sites. But I think you'll have a hard time finding a web framework designed specifically for your example. In django you store the URL or filepath with the metadata in the model
 
jrh
There's a famous blog I ran into the other day that said never store images in databases [no exceptions], but there was one job I worked on (not ORM, not Django) that involved storing the machine vision camera pictures of parts with labels showing the measurements of each component. The image wasn't an icon or logo or something, it was literally associated with the record of the engine.
 
What is the longest string that can be stored in a database?
 
jrh
@Dodge Quite long, more than long enough for any data I'd need to store I think
 
1:16 PM
MySql's LONGBLOB type goes up to four gigabytes
 
@jrh Wonder how bad performance is from storing your numpy stuff as string representation and converting when needed
 
jrh
@Dodge yeah I'm starting to get the feeling Django was coded with some very specific use cases in mind, and I wonder if something like Flask would be a better fit
though I've never used Flask; I did some ASP.NET MVC and a bit of Node.js
 
@jrh pinterest is (or was) a heavily django laden code base. Lot's of images there
 
jrh
@Dodge yeah that's what I don't really understand, I'm looking at these websites and thinking, "how would they not store binary data in DBs"
 
Converting between bytes and ndarray should be pretty efficient I expect
 
1:19 PM
PaulMcG, a regular here who knows what's what, explained the difference between flask and django like this: flask is a pirate ship and django is a navy aircraft carrier. This helped me decide where each is appropriate.
 
jrh
I think there probably would be a hack for Django to store all Instagram user uploaded images as static files, but it sounds like a mislabeling to me, the images aren't static
 
no they are not... but the directory is?
 
jrh
technically the database directory is "static" too
 
What definition of "static" are we using here
 
fuzzy tv screen ;)
 
jrh
1:22 PM
When I think of static I think of stuff that never changes as the program executes but that might be the wrong definition
like if I make a server that just serves up a hand typed index.html, that's a static web server
IIRC that's the definition Mozilla uses, but I don't know if it's universal
 
I think we are talking about the meadia root directory (or something similar) not the static files directory
Also, Django models, have both a FileField and an ImageField.
 
jrh
setting aside static for maybe something more useful terminology (hopefully), I kinda see several kinds of data, 1) Data that is never changed as the program runs, 2) Data that is changed, but isn't stored in the database (e.g., user file uploads that go into some directory), 3) Data that changes and should be stored in the database but doesn't fit well into Models, 4) Data that is hand editable by users like a Form and does fit in a Model
 
I ask because I feel like there's a terminlology mismatch in "a hack for Django to store all Instagram user uploaded images as static files". If you're using a subjective definition of "static", then there's no problem -- just stop mentally labeling the files as "static". If you're using an objective definition like "the file is inside a directory named 'static'", then it should be quite easy to evaluate the hack to determine whether it meets that criteria
I like your four category system. I think #3 is less of a problem if you get creative about fitting the data into a model. To use your 3D double array example, perhaps you could keep it in a table with an (x,y,z) composite key, and a column for the double values.
 
jrh
@Kevin So to clarify, intuitively I would probably guess Instagram uses option (3) for images, (4) wouldn't work because nobody would want to type in pixel data on a Form, (2) could maybe work with a "hack" (store some file named aoisfhaoighoiahrfpoahifoa.png in user_uploads and put an entry in the database to link it to a user).
@Kevin sure, clever models can be nice, but would somebody want to type in each double? What about 3d models? Would you rather type in vertex values or export it to SolidWorks and edit it with a nice UI?
I guess the point I'm getting at here is, I always kept data in an "easy to edit/view" representation. You could store a .wav file as a table with thousands of signal values but that would be hideously slow and hard to use.
 
just in case one of the geniuses here know a solution lol... anybody here know how to use re.search with searching for a characters ascii? I have to characters with which are very similar when searching by them '○ ' and '● ' but their ascii is different so curious if I could use this within the re.search
● = 9679
○ = 9675
 
jrh
1:37 PM
@Kwsswart I don't think that's actually ASCII: fileformat.info/info/unicode/char/25cf/index.htm
 
hmm I thought it was used the method here stackoverflow.com/questions/227459/… to get it
 
jrh
Python 2 or 3?
 
@Kwsswart Can you clarify what you need? Are you looking for the character-of-ordinal 9679, or for the literal 9679?
 
@MisterMiyagi searching the character-of-ordinal
 
This might also be a good moment to read up on the minimum that every programmer should know about encodings.
 
1:42 PM
Will do
 
@Kwsswart Then you can directly use the character or escape in the pattern. E.g. "●+" or "\u25cf+"
 
@jrh "easy to edit/view" is a valid goal. But I think you can have that and "is stored in a database in a not-so-easy-to-view format". For example, maybe you have a table that stores images as 1d byte arrays. You can still configure your view so it displays human-comprehensible images, and has an "upload image" button instead of a "enter byte data:" text box.
 
(25cf is hex for 9679)
 
Of course, hooking up an "upload image" button requires a bit of customization, so it's already not as easy to integrate as a type that's natively supported by Django. But hey, at least it's still easy for the user...
 
@MisterMiyagi Thanks for this mate works out I will read that article on the train later thanks for the help
 
jrh
1:47 PM
@Kevin Yes, that sounds great.
that was going to be the next thing I looked at, how to get Django to show custom data that isn't "a set of ints/strings/etc models"
I'm guessing it might involve custom Models, which I can definitely make
I just wanted to make sure I had a good reason for painting outside the lines, so to speak. It sounds like I do.
 
Ah, Django has an ImageField, so I expect it can do the upload image button thing natively. So the challenge is more on the model-to-db side, because ImageFields save their data to the file system by default. If you want to put the data right in the DB, I guess you need to subclass ImageField. (Unless there's a ready-made upload_to_db flag I overlooked, which is quite possible. I'm skimming between a lot of pages right now.)
 
jrh
To me personally it makes sense from a design standpoint, that a Model can store object fields; displaying it won't be as easy as just calling __str__() on it and shoving it into a text box but theoretically if Django can specify how an int is shown, perhaps I can specify how <arbitrary data> is shown
 
Yeah.
 
jrh
@Kevin ah yeah, one thing I've noticed is that the documentation could use a bit of work.
it isn't quite up to the standard of the official Python docs, but most stuff isn't, to be fair
OK, I don't feel bad about wanting to customize things now. Thanks!
@Kwsswart One thing I'd like to add that confused me, when Windows talks about "Unicode" they really mean UCS-2 usually and it might give you the impression that all unicode char data is 2 bytes (it's not), there's more than one kind of Unicode; Linux uses UTF-8 usually.
All Unicode uses multibyte characters, it's just a question of how the characters are stored. E.g., mothereff.in/utf-8#%ED%95%99
is \xED\x95\x99 in UTF-8, it's stored as 3 individual bytes.
 
2:04 PM
In my experience, unicode Just Works without you having to know about its internal mechanisms... If and only if you're not trying to convert it into anything else
(it was not always thus, and let us pray that the Dark Times never return)
 
jrh
The dark times never left C++, I just had to drop unicode support because a library didn't document that a char* API expected CP_ACP strings
 
"use regex to find the position of this unicode substring in this larger unicode string" should be fairly straightforward since the types match up
@jrh Ooh, char array... Yep, that's a textbook conversion headache
 
jrh
not even possible to convert
 
Even more textbook then :-P
 
jrh
internally it used the A (ANSI) version of a winapi function which mangled the UTF-8 hex values and reinterpreted them using the active code page
which is very very easy to do because in Linux it would have "just worked" but in Windows char* means ANSI
personally for my own data I store everything as UTF-8 then at the last moment shuffle it back to UCS-2 for good ol' Windows to understand
Hopefully none of these creaky old APIs are a problem in Python, I haven't ran into it, but on the other hand I don't really use Python on Windows for very heavy stuff. Linux, on the other hand...
Also for context the C++ standard library kind of just shrugs at string encoding and says "std::string can contain whatever char information I don't know", Windows took that to mean ANSI always, Linux took that to mean UTF-8 OR ASCII (doesn't matter); at some point Windows "bet the farm" on UTF-16/UCS-2 taking over the world and it didn't
So hey, Python is looking like a great language. I am looking forward to this new Python project!
 
2:11 PM
Ugh, I didn't realise how much convenience there is in Python's str until I worked with UTF-8 strings + char vecs in Rust...
 
jrh
Just be careful, make sure that Rust didn't "punt" on the legwork it will have to do to use the winapi functions
The Windows UCS-2 decision infected the standard C++ library too
 
Thankfully, I can stay clear of Windows for all critical programs.
 
In the rare instances where I needed to pass a string to/from the not-quite-official Python bindings for the win32 api, I found the experience to be pretty painless... I gave it regular strings as arguments, and got back regular strings as return values. I guess it's thanks to the legwork of the binding's developers.
Also nice that I can pass in lists of things where the native API usually expects proper arrays
Granted, pretty much all string data I provide/expect in my projects are in the ascii range. So if the approaches I tried would have returned mojibake for more complicated input, I wouldn't know about it from my cursory testing
 
jrh
I've had some funny stuff happen with unicode so I paste it into everything to exercise the system. I found out the UI framework (C++) I work with can store unicode but can't display it, which is funny
A coworker of mine who speaks Korean just gave up on using his native language on his Windows computer for naming folders, his username, etc., which is kind of sad
 
Relatable. Python stores all unicode quite effectively, but the primitive shell I use to run Python will display more question-marks-in-boxes than I would prefer
 
jrh
2:21 PM
@Kevin If it's Windows you can fix that by setting the UTF-8 codepage in the console and picking a decent font
 
Yes, I've heard of that solution. I've had partial success, depending on the font.
 
jrh
sadly as far as I know there is no console font that supports all unicode symbols, yeah
e.g., NSimSun is good for Korean
Arabic chars, I don't know
what's weird is that Consolas in my text editor can display stuff that Consolas in CMD can't
I can't explain that
 
I would be quite happy with any font that can render 'THINKING FACE' (U+1F914) as "🤔" or similar
 
jrh
Thinking intensifies...
 
Yeah I ran into an issue like that last week. Couldn't figure it out.
 
jrh
2:28 PM
I have no idea why this happens, the font clearly supports it, but maybe CMD can't render certain kinds of uh.... TTF instructions?
 
Maybe chcp 65001 covers less of unicode space than I thought
 
jrh
it gets weirder though
CMD can't display the Korean character in Courier New, but Notepad++ can
Set CMD to NSimSun, and it can
it's not a codepage thing, the code page didn't change at all
 
I'm always suspicious of the code page, even after proving I shouldn't be suspicious of the code page
 
jrh
though there's something fishy about how Notepad++ displays these chars
as I change the font the Korean character and emoji don't change at all
which makes me think it's not actually using the TTF font for those
Also another fishy thing, the notepad++ unicode chars are squished into the same size of chars as letters, if the "thinking face" was the same size as the letter 'a' you couldn't even tell what it was
I'm starting to think every editor just makes it up as they go along
 
2:53 PM
hey folks, quick question. Say if i have pushed a branch on github, and put a merge request, and then get comments to make changes, how do i actually make those changes?
Is it as simple as work on the same branch in local, commit, push, and github will take care of the rest?
 
3:11 PM
@ParitoshSingh yes, but reviewers can also make code suggestions you can commit via the web UI
 
Ah, thanks AD and Aran, much appreciated
 
I always didn't understand why don't few useful python packages don't support with newer python versions such as 3.9
 
It's primarily a matter of time and resources. Python 3.9 is relatively new in the grand scheme of things, and library maintainers need time to update their codes and test things on newer python versions
 
3:27 PM
I'd expect large libraries to support it. What doesn't?
 
Few libraries like pickle, and etc.
 
wait, pickle?
 
Yeah
Pickle5 isn't supported with python 3.8.
 
pickle is part of the standard library, if im not mistaken. I don't see how python 3.9 would ship without it
whats pickle5
 
3:29 PM
> "This package backports all features and APIs added in the pickle module in Python 3.8.3, including the PEP 574 additions. It should work with Python 3.5, 3.6 and 3.7."
If you read that message, you'll hopefully realize that pickle5 shouldnt exist for python 3.9, because it's a backport of the pickle functionality to older python versions
 
I do, its just a part of what I'm trying to say.
 
So, if you're on python 3.9, use pickle. because pickle5 was a band-aid for older python versions.
So, it makes zero sense for it to exist for 3.9. Less than zero sense even.
 
It's just that there are few packages out there when I starting python struggled to install it, a few of those might be few scraping ones, and etc.
 
My only advice there would be, read the pypi pages, or the github pages, to make sure what python versions are supported. And for this package in particular, Let's not skip past the main issue here. Be aware of which packages are backports as opposed to generic external libraries, because backports will (and should) stop working after a certain version.
 
@SiddharthPilli perhaps you have an XY problem, trying to install other packages that are in the standard library?
trying to install random packages opens you to the risk of malicious namespace squatters
 
3:39 PM
@ParitoshSingh Thank you that really work out!
 
 
3 hours later…
6:33 PM
Hi
 
AAB
cbg all,
Say I have database table that contains a mapping like manufacture, category
In django how do I model this table ?
django models cant be without primary key
manufacture,category together are unique is a composite primary key
any way to create a model for this without creating another AutoID column for the table?
 
> Each model requires exactly one field to have primary_key=True (either explicitly declared or automatically added).
Sounds like a "not possible" to me
 
What is "manufacture"? Is that used as a mapping of company --> manufacture --> category?
If I'm right, then you'd have an Association Object in SQLAlchemy and I imagine Django would have something similar in its ORM
I don't understand why you'd have a composite key. "Manufacture" (and at this point I'm wondering whether you really mean manufacturer) and "category" are separate entities, so that would be a many-to-many relationship at the least
In which case, the manufacturer would have a unique id, which can just be an auto-incrementing integer, as would the category, and everything is happy once more
 
7:00 PM
Does anyone know a module that provides a dict with a key function? I want to be able to do something like
id_dict = DictWithKeyFunc(keyfunc=id)

id_dict[some_unhashable_object] = 5
 
Sounds oddly specific. Can there be pitfalls by subclassing dict and wrapping get/setitem?
 
inb4 you're going to end up rolling your own :P
 
Yeah, that's looking like a safe bet
@AndrasDeak Don't think so. It should easy to implement, just a pain in the butt
 
yeah
considering the usual things you end up reimplementing I'd say this is perfectly normal :P
 
Google results are all "can you store a function in a dict?" :|
 
7:06 PM
yep. after dialing up the google fu i found some dict related packages, but they're mostly for dot access and trying to manage deep nesting, not really for your use case
 
cbg
 
Alright then, next question: How do I name my library and the class? DictWithKeyFunc is a mouthful and not even particularly self-documenting
 
MappingMapping :P
 
laurel
 
Do library names need to be self-documenting now? :(
 
7:08 PM
doubly funny because it's true
 
@roganjosh They don't, and to be honest I have half a mind to be lazy and name it Lexicon
 
FuncyDict :P
FuncKeyDict
 
lexicon is already taken, dang
 
FuncKeyDict gets my vote
 
(you can probably sound out where im going with this) :P
 
7:11 PM
FunkyDict it is and I'm not even joking
 
yeah, i dont mind it at all honestly haha
 
we've all seen worse
 
and it might hopefully even prime someone who's reading the code to know that something is amiss
 
7:47 PM
@Aran-Fey I'd made a class for endlessly nested dictionaries. I named it Fict for fictionary. You could use that idea somehow if you like it.
class Fict(dict):
    def __getitem__(self, key):
        return self.setdefault(key, type(self)())

d = Fict()
d[3][4][5] = 5
d
# {3: {4: {5: 5}}}
 
I don't get it, why fictionary? How does that relate to nesting?
 
There isn't a strong tie-in. It's a dictionary sorta, like a fictional dictionary... <waves_hands_convincingly>
For your class though. Maybe functionary.
 
Ah!
Hmm, functionary's not bad. I'm gonna stick with FunkyDict though, since I already created the package and everything
So collections.abc.MappingView is a thing, and unlike the name of the module would suggest, it's not even an abstract class. But there's no documentation how to instantiate it or access the underlying mapping, so... it really might as well be abstract and do nothing :|
Actually it would be better if it did nothing. Ugh, I hate the stdlib
 
8:14 PM
sounds like it's mutual
 
@Aran-Fey Check out this table in the docs docs.python.org/3.10/library/… These show which methods you must implement in order to satisfy isInstance with these types.
>>> from collections.abc import Hashable
>>> class Z:
...     def __hash__(self):
...         return 42
...
>>> z = Z()
>>> isinstance(z, Hashable)
True
 
Thanks, but I'm planning to implement all the dict methods anyway
 
So if you implement all the methods for Mapping, then your class will be a Mapping.
You don't really instantiate a Mapping, this table just shows you how to quack like a Mapping.
 
That's why I said it'd be better if MappingViews were abstract. If they were, I'd just implement all the methods myself, no problem. But since they aren't, I either have to override all the methods they already implement, or figure out their internal API
 
8:52 PM
@Aran-Fey why?
I was being literal when I asked about subclassing with custom getitem/setitem
OK, probably keys too
and maybe __contains__...
 
Ok, I just worked through this, and you do subclass from Mapping. First I wrote with subclassing, but with no class body. When I instantiate I get this error:
TypeError: Can't instantiate abstract class FauxMapping with abstract methods __getitem__, __iter__, __len__
class FauxMapping(Mapping):
    def __getitem__(self, key):
        return None
    def __iter__(self):
        return iter([])
    def __len__(self):
        return 0

z = FauxMapping()
print(isinstance(z, Mapping))
This evals to True.
To be a MutableMapping, add __setitem__ and __delitem__.
But that table didn't really steer me along well, like I had hoped. The exception messages were much clearer. (This is with Py3.8)
And I had to come up with the method signatures myself.
 
@AndrasDeak What I meant to say is, my class will support all dict methods. Whether I'll have to implement them myself is to be seen
 
Ah, OK
I wanted to see if __contains__ can be inferred from __getitem__.
>>> class Foo:
...     def __getitem__(self, other):
...         print(other)
...         if other == 42:
...             return -42
...         raise KeyError('not 42')
... 42 in Foo()
0
Traceback (most recent call last):
  File "<ipython-input-1-745b5b472239>", line 7, in <module>
    42 in Foo()
  File "<ipython-input-1-745b5b472239>", line 6, in __getitem__
    raise KeyError('not 42')
KeyError: 'not 42'
What am I missing? Why is other == 0?
 
9:09 PM
Ironically, __iter__ does get inferred from __getitem__
 
My money's on "ipython weirdness"
 
hmm, good point
 
Wait, you didn't inherit from Mapping
 
>>> 42 in Foo()
0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in __getitem__
KeyError: 'not 42'
Yeah, I didn't do anything, only define __getitem__
Ah, so it defaults to sequences I guess
I didn't realize that val in lst doesn't use the index
thanks
 
A legacy from Python 1.x? maybe. If you implement __getitem__, and then do list(instance_var), Python will call __getitem__ successively with 0, 1, 2, ... until it gets a KeyError.
class Z:
    def __getitem__(self, key, default=None):
        print(key)
        if key < 3:
            return key*2
        else:
            raise KeyError(key)

z = Z()
print(list(z))
 
9:14 PM
it might be somewhere in the data model
 
@PaulMcG You mean IndexError
 
^^ yes
Adding __iter__ = None will suppress this.
 
Huh, now that is news to me
I thought __hash__ was the only one that could be set to None
Now I've gotta fix a bug in my other lib...
 
I just learned that this week. Previously I had to implement __iter__ and raise NotImplementedError. But I read this while compiling a summary of changes by Python 3.x versions.
 
Is "noneable" a word I can use without having to feel guilty?
 
9:22 PM
You could call it a "None-der" method.
 
jrh
@Kevin thinking about this a bit more, the 3-d data as an (x,y,z) composite key method, it's a smart-feeling trick but relations are sparse by their very nature and a 3-d array may not be. I just ran into this very thing yesterday, relations (in relational databases) take extra work to simulate contiguous memory, the (x,y,z) keys are redundant from the perspective of the capabilities of different data structures
what you'd essentially be doing is taking a dict and adding (x,y,z) tuple keys to it, if I saw that in Python code intended to fill a contiguous space the first thing I would think is "wow why didn't they just use a 3-d numpy array"
The other thing is, even if hypothetically you did want a sparse array, a table or dict wouldn't be particularly helpful, because from my experience sparse matrices need information to quickly traverse to the nearest neighbor to be practical (at least in the sparse applications I've worked with), and there's no such connections graph in a dict/relation
 
So why don't you store your 3D data in a non-image format and store a link to that data in your database, the same way that you would reference a link to an image?
There's no reason you can't store the data in HDF5 format and save a link to that file in your DB
 
jrh
9:45 PM
@roganjosh I could store a path like adsfhaioghoairhoraeworih.stl in the database which references some folder like 3dModels/ , sure, and if the files were large enough I'd say that makes sense
though that's a separate folder to back up, and it'd be filled with thousands (millions?) of unlabeled files that are completely impossible to link back to the relation without the help of the DB
 
Come, now. All I can find on the discussion is some dispute over the semantics of "static" and then some other discussion about composite keys and sparse arrays. This problem is ill-defined
@jrh that's why there are backup mechanisms, no?
Flip it around; what is it that you want?
 
jrh
@roganjosh The original question in my mind was, what to do with data types that don't really fit neatly into tables, I can think of a bunch of small objects that I wouldn't really want to make tables for, nor would I want to involve the filesystem with and save a lot of tiny pickles for
 
Is there commonality between the small objects, though?
If there isn't then maybe you go with the document model of MongoDB or something. But I'm not sure I follow the requirement properly in my head
 
jrh
The easy one that I kinda started with was just an array of strings as an attribute of a class; I could use a ManyToManyField and have the field reference another table with a StringSetID and have the class pick which "string set" it wants, or I could just serialize the array right in as JSON; the former makes sense if there's thousands of strings probably, the latter seems right considering I'll have less than 40
especially considering that each array of strings will have no real meaning to each other, and aren't unique
the "stringset" table would be about as meaningless as dumping every string from an executable, you'd get a mix of application text, hard coded string literals, etc
 
Disagree. ManyToMany makes sense to me as long as there are indexes set (side note; I think it is indexes and not indices in the case of setting distinct indexed fields?)
@jrh Which is why you wouldn't have the "stringset" table, but instead have single strings with no structure wrapping them
 
jrh
9:58 PM
so you'd prefer having a these tables: stringSets=[id, stringIndex, stringSetIndex], stringValues=[id, stringValue], thingThatUsesStrings=[id, stringSetID] over just thingThatUsesStrings=[id, stringArray]?
 
Yes
 
jrh
huh, interesting. Why?
 
As long as you put in appropriate indexes then you'll be fine
 
jrh
but somebody looking at stringValues would just see a blob of random string data
 
03:00 - 22:0022:00 - 00:00

« first day (3854 days earlier)      last day (1079 days later) »