« first day (4300 days earlier)      last day (652 days later) » 

6:20 AM
@Aran-Fey I wrote a post about that a while ago stackoverflow.com/questions/57628064/…
 
6:30 AM
It is possible to update versions in the CI/CD pipeline, but it requires a bit of work because you have to still somehow tell your bumping tool if it's patch/minor/major (e.g. by using something like conventional commits) and you need a specific version-bump-bot with push rights plus hacks on how to get around infinite loops.
it feels like 50% of the CI code I maintain is just for version bumping
 
6:41 AM
ugh, now that I look at that post again it needs some rework, lots of stuff has changed since then. that should teach me posting about toolchains and not languages, so much work keeping them up to date..
 
 
2 hours later…
8:28 AM
Cbg pythonistas, I'm back from holidays :)
 
8:40 AM
Holiday? I've heard of that... grumbles jealously... :p
 
8:53 AM
That's the time of the year when you can finally work without interruptions, isn't it?
 
"without interruptions" - I've also heard of that once upon a time? :p
 
 
3 hours later…
11:48 AM
While inspecting a strange directory on my personal computer, I get an unusual message. Paraphrased: "you do not have permissions to read the read/write permissions for this object". I'm the administrator. If I don't have permission, who does? The Lord Of All Cosmos?
 
Cthulhu.exe
Actually, Cthulhu.bat sounds a bit better
 
There is a certain Non-Euclidean feel to this scenario
 
Kevin accidentally peeks behind the curtain. There's Kevin peeking back at him. Better leave the curtain alone.
 
^^^ one of my favourite cards... :p
Eldrazi decks are great fun... as long as the ramp works :p
 
Elvish piper about to drop the year's hottest mix tape
 
11:57 AM
:)
 
Does anybody know anything about Christoph Gohlke's Windows wheels site (lfd.uci.edu/~gohlke/pythonlibs) being archived? I just noticed it said "Archived" at the top, and the last update was at the end of June...
 
My mysterious directory is almost certainly shovelware installed by the computer's manufacturer. But even with hardware-level control, I'm surprised it's possible to flummox Window's permission system like this.
@MattDMo It's news to me. I see the page is also listed under "Archived" at cgohlke.com
 
Yeah, I saw that as well.
 
Does anyone here have experience with Airflow? Is there a case in which a "landing time" of several hours is expected or acceptable, or is this always a bad sign?
 
@MattDMo That's era-ending news :( That said, though, I'm not sure how much it's needed these days once pip sorted itself out with compiling numpy/scipy etc. and there's also conda. Were you still using it for a particular library?
 
12:09 PM
haven't used airflow in a while... how complex is the DAG?
 
After ten minutes of googling, my opinion is, it's expected/acceptable if the task you're automating takes a lot of computing time even when Airflow isn't involved
If it takes three hours to bake a turkey in a conventional oven, don't fret too much if you buy a whizbang new oven and it still takes three hours to bake a turkey
 
Not really - I liked the idea of numpy+mkl, but I've never actually benchmarked its performance vs. the pip-downloadable one. And you're right - now that more and more packages have Windows-specific wheels, it's less of a necessity. I can't imagine how many SO comments and answers point to it though. There must be dozens just by me.
 
I last used gohlke's site about a month ago, when I needed PyWin32.
 
@JonClements a single sensor operator and 8 normal downstream operators, but they are all pretty simple and lightweight. The DAG is running in a big cluster with hundreds of other tasks though, so I suspect resource contention and scheduling problems, since I don't see any deliberately inserted delays of any kind anywhere in the code
 
@vaultah several hours seems off though... however, are those just outliers or consistently so?
 
12:15 PM
pypi.org/project/pywin32 would have me believe that pywin32 is pip-installable these days. But then again, I'm pretty sure it said that the last time I tried to pip install it, and it failed spectacularly
Surely Lucy won't pull the football away this time!
 
@JonClements 1-3 hour landing times seem to be very common. The longest landing time is 11 hours, the median is about 20 minutes
 
Obligatory suggestion: check the logs
 
@vaultah weird... never had that myself... I'll bow out as haven't touched it in a few years and what experience I did have with it wasn't exactly comprehensive anyway
 
Google tells me that workflow management software will sing praises of the quality of their own monitoring and logging
 
@MattDMo Yeah, the MKL bit was a nice addition because the size was stripped back. But back when I was working on Windows (now several years ago) it would still link MKL automatically if you had it installed, and it was, at least then, free to install MKL and have it across multiple different things, not just Python
I seem to remember getting everything I could along with MKL, including some go-faster stripes
 
12:25 PM
Hmmm, Airflow's log page returns 502 after 60 seconds. I wonder what that could mean :D
 
Then again, my Desktop - The Frobnicator - did have blue lights on its fans so it was already pretty souped up
 
Darn, I was hoping github.com/cgohlke/cgohlke.github.io would have a useful commit message for the change that added the wheels page under the "Archived" header. But it's been that way since the initial commit on June 28
stackoverflow.com/questions/72581592/… claims there was an explanatory message on the page. I don't see it on the actual site though.
 
12:45 PM
According to whois, cgohlke.com (the site lfd.uci.edu/~gohlke redirects to now) was only created on 2022-06-18, while the Github repo was created on 2022-06-28, so it's all quite new. I really can't imagine that the whole LFD has been shut down just like that, what's more likely is they lost some of their funding and Gohlke's position was cut.
Or he left/was forced out for other reasons. Hopefully he'll find a host large enough to handle all the traffic of the archive and be able to keep maintaining it.
 
I think you're in the right ballpark. For his virtuous deeds, I hope Gohlke finds a cushy position. Ideally with a sturdy file host that he can plop the wheel repository into. But that part is more for my sake than his :-P
 
Googling laboratory for fluorescence dynamics shutdown gives results for the SO question above and a Twitter post mentioning it, but nothing else relevant. However, lfd.uci.edu/people used to have a whole bunch of people listed (as of June 2021), but now only has 2. Maybe their funding really has been curtailed...
 
Concept: instead of traditional file hosting, we put the wheels on the blockchain. You get 1 download for every 100 GohlKoins you mine.
Christoph will be our elusive figurehead a la Bitcoin's Satoshi
 
You've stopped taking your medication again haven't you Kevin? :p
 
Hmm. It looks like they lost their NIH grant a year ago. Maybe UC Irvine gave them survival funding for a year for a graceful shutdown. Still can't find any news articles on it, though.
 
12:59 PM
The medication only changes the shape of the madness, from circles to triangles. Allows for neater organization.
 
As long as you don't start trying to sell us NFTs :p
 
btw has anybody figured out what relative brightness is?
and it's relation to apparent brightness? It still eludes me
 
NFTs don't interest me. Cryptocurrency might interest me if it starts behaving more like an actual currency. (No, I can't describe what that would entail.)
 
@Hakaishin context?
 
@MattDMo James Webb Space Telescope, obviously
 
1:05 PM
Obviously.
 
I recall discussing some strange astronomical data in here the other week...
 
@Kevin I think cryptos won't take off, because people want some cushion between markets and their livelihood and central banks offer that. Sure there are also drawbacks that inflations hits lower economic ranks harder but it's better than to have no form of influence on monetary policy.
@AndrasDeak--СлаваУкраїні haha sorry, yeah still hung up on webb. I was actually meditating with the swiss scientist who worked on it and he gave a cool lecture about it. But he didn't go into too much detail
 
at least be hung up on webb to make it less confusing
@Hakaishin that's unsurprising. As far as I know talking is a lot easier way to convey information than meditation.
 
Depends on the information :P
 
Just based on the words, relative brightness probably is relative to a known standard, such as a particular star or galaxy or whatnot. The standard might be set to a value of 1, or 0, or 100%, or whatever you want. Apparent brightness may refer to an actual unit, such as lumens or whatever they use in astronomy. I'm a scientist, but a biological one, not an astronomical one, so I really have no idea.
 
1:18 PM
Words? Where we're going we don't need words.
 
MattDMo records his observations through the microscope. "specimen is eukaryotic, with lophotrichous flagella. Relative brightness is...". He turns off the lights and looks through the aperture. "... Zero, as usual"
 
Speaking of non-bioluminescent eukaryiotes, my new rear bike lamp leaks light even when turned off.
It's very faint, but also very annoying. How hard is it to turn a yamming lamp off?
 
The default state of a light is to be off, so it's usually not hard at all
Lights positively crave offness
 
haha unfortunate, does it actually draw power or is it some luminescent part?
 
Are you moving the circuit very fast through a magnetic field? Try not doing that.
 
1:26 PM
@Hakaishin it's an LED
 
oh yamming heck... I'm having a blonde moment - I blame covid
LINE 1: select SUM(transactionTotal) from transaction_headers;
                   ^
HINT:  Perhaps you meant to reference the column "transaction_headers.transactionTotal".
 
I suppose that is supposed to tell me something :D
 
nothing wrong with blondes though
 
what am I missing here?
 
1:28 PM
@Hakaishin yes, LEDs work by exciting electrons in a solid state device. Light <-> current <-> power.
 
If that's SQL, it looks syntactically correct to me.
If your real query selects from more than one table, it might be fussy about column names that appear multiple times
 
@AndrasDeak--СлаваУкраїні I mean that yes, but this still leaves the two options it continuously draws power for some weird reason or it has something which "soaks" up light and then releases it slowly, like the stars we had as kids, open
 
nope... that's the entire query
 
I often run into problems doing select grommet_id from grommet left join spindle on grommet.grommet_id = spindle.grommet_id
 
@Hakaishin no, like I said, LEDs don't do phosphorescence.
 
1:30 PM
that's the word :D Ok, then it's a very funny LED
 
I don't suppose you've tried select SUM(transaction_headers.transactionTotal) from transaction_headers;
 
It's probably a very traditional LED, with a very crappy transistor circuit that does the "turning off".
 
"I have, and it works, but I want to know why the shorter way doesn't work" is a valid reply btw
 
orchard=> select SUM(transaction_headers.transactionTotal) from transaction_headers;
ERROR:  column transaction_headers.transactiontotal does not exist
LINE 1: select SUM(transaction_headers.transactionTotal) from transa...
                   ^
HINT:  Perhaps you meant to reference the column "transaction_headers.transactionTotal".
 
hmm based on a quick search, soaks up the light seems to be a very good description, fascinating
 
1:32 PM
Yeah, fluorescence and phosphorescence are similar effects, but the latter has a much longer timescale to emit over.
@Hakaishin if you want trippy, read up on triboluminescence, and then buy a roll of scotch tape and pull it off in a very dark room after 15 minutes of letting your eyes get accustomed to the dark.
@JonClements silly linter?
 
@AndrasDeak--СлаваУкраїні this is using psql directly
 
@JonClements that's "it's an LED" to me
You did what the linter tells you, but 1. you get an error, 2. you still get the same linter hint even though that's exactly what you did. So the linter must be wrong.
 
I was reading about triboluminescence the other day, when I asked the best way to remove static electricity from a cat. It's one of those classic phenomenons that any ordinary person can observe in their home, but the best scientific minds don't know quite how it works
 
okay... needs "''s apparently - else it considers lowercase...
orchard=> select SUM("transactionTotal") from transaction_headers;
       sum
------------------
 3315193.67999998
 
@AndrasDeak--СлаваУкраїні really cool stuff
 
1:37 PM
@JonClements silly
PSQL: 'well I did say "transaction_headers.transactionTotal"'
 
Oops, more specifically I was reading about TriboElectrification, which doesn't necessarily emit any light. Similar principle, I'm guessing.
Both of them have "tribo-" in the name. Therefore I posit the existence of a fundamental particle, the tribon, which is responsible for both.
 
@AndrasDeak--СлаваУкраїні works fine else where - guessing it's just something that the psql command client does? shrugs
 
I sometimes have to put quotes around column names, but only when they contain "weird" characters, or overlap with a reserved keyword
 
yup... that was my understanding and always how it has worked - not quite sure where the casing stuff comes in for the actual postgres client to do that (or not do that)
 
For once, it's not Oracle that has the silliest behavior! Hooray!
 
1:42 PM
I'd take Oracle over SQL Sever any day though :p
 
Snowflake is really pedantic about this stuff and every field needs quoting. Guessing this is the same
Got a customer complaining that I didn't push an entirely new version of their webapp live at close-of-play on Friday. Honestly can't win :/
 
That reminds me of a sql-ish problem I had this weekend. I was looking at some psycopg code the other day, and the query had a part that was like where title &@~ %s. The %s is evidently the parameter placeholder. I couldn't figure out what "&@~" was doing. From context, it could be an operator for "is similar to"
 
is it a FTI field or something?
 
I know that ~ by itself is a valid operator, but I couldn't find anything about at symbols.
 
or an extension type or something... brain's dead at the moment
 
1:46 PM
People that want releases at the last point of the working day on a Friday probably need to recalibrate a couple of things
 
orchard=> select count(*) from transaction_headers;
 count
-------
 15526
(1 row)

orchard=> select count(distinct id) from transaction_headers;
 count
-------
 15526
(1 row)
so now need to get the lines for those 15k
 
I don't know what an FTI field is, so I'm going to guess "no"
 
@JonClements Not sure what I should be taking from this?
 
Have you tried making the problem more triangular by taking my pills?
 
Given that you're having a 'mare I'll just check you're not misreading the output. (1 row) is in relation to the fact that the query returned a single row representing the count. It says nothing about how many rows you can actually pull out of the db
Otherwise I don't see anything surprising in the result - every record has a unique ID
 
1:52 PM
@roganjosh heh... I just don't trust the api I'm sourcing this data from :p
 
Oh, neat, postgressql lets you define your own infix operators. Maybe &@~ is defined somewhere else in the code base.
 
but yeah... thanks... select count(distinct "transactionId") from transaction_headers; is more asccurate
 
Not using snake case in SQL table names is a reportable offence btw. I hope you've done that
But the SQL is not SHOUTY, so I'm afraid I will have to report that myself.
 
1) nope - they're just naff, 2) can't be yammed to keep putting caps lock/shifting for a simple adhoc query
anyway... just doing some quick and dirty stuff to be revised later...
 
All I hear are excuses :P Save them for the LAW
 
2:03 PM
The good, the bad, the ugly, the quick, and the dirty
 
luv ya too Josh :p
 
 
3 hours later…
4:44 PM
@KarlKnechtel You changed the function name to example, but the text still talked about Dict, which is rather confusing. I've just fixed it.
 
5:06 PM
This may help: https://en.wikipedia.org/wiki/Magnitude_(astronomy)
The absolute magnitude (M) describes the intrinsic luminosity emitted by an object and is defined to be equal to the apparent magnitude that the object would have if it were placed at a certain distance from Earth, 10 parsecs for stars.
 
5:28 PM
ugh... now I'm having to remember what a parsec is
 
A long way, but not too long
 
3-ish light years seems to ring a bell...
trying not to cheat and look it up
 
6:05 PM
Well dang... I had a problem with an Oracle query, and I converted it to a sqlite-powered MCVE... And it ran perfectly. I guess Oracle is pickier about group by clauses.
Well, for the record, dpaste.org/a0gEw
Help, my code works too well!
 
I can't even figure out how to create and populate a table in oracle...
 
If I remember how to, I'll make an MCVE :-)
 
What is the syntax to run a .exe with an input, I think that's the issue with my script so knowing what it should look like in command prompt would help me fix my script. I've been googling for a good 2 hours
 
Turns out 'foo' is a string while "foo" is a column name. Who knew?
 
"C:/Program Files/Maptek/Vulcan 2022.1/bin/exe//bhead.exe" gns_bhm_model_eng_20220622.bmf
 
6:18 PM
@Aran-Fey I know this pain well
 
this way tells me I have an unexpected token in expression or statement.
 
@CelesteWilson That looks like a normal way to run an exe with input, IMO
For certain definitions of "input". If the program expects data to be typed into stdin, things are trickier
 
if I put this
bhead.exe 'gns_bhm_model_eng_20220622.bmf'
it works
 
What shell are you using?
 
but I want to put the path, since I have multiple versions on my laptop
powershell?
 
6:20 PM
Oof, then I can't help
 
I can bumble around in powershell, but it won't be as productive as my usual bumbling
 
Are the forward slashes always OK?
 
trying it in command prompt
 
@Kevin Try select age, min(name) from user group by age
 
ok it works in command prompt, now to get the script to run it the same syntax
lol
 
6:25 PM
@Aran-Fey Right, I would expect that to work. But suppose the table had an additional column, height, and I want to get the name and age and height of each representative. I'm worried that if I do select age, min(name), max(height) from user group by age, I might get Barney's name and Fred's height in one row.
 
cmdstr = '"' + ' '.join(\
         [vulcanbinpath + '/bhead.exe' + '"',
          '{}'.format(bhmbmffl)])
will this work? if vulcanbinpath = path, bhmbmffl = .bmf filename
 
Why not just subprocess.run(["C:/Program Files/Maptek/Vulcan 2022.1/bin/exe//bhead.exe", "gns_bhm_model_eng_20220622.bmf"])?
 
It outputs this when I test it. Are the outer quotes just the output? '"C:/Program Files/Maptek/Vulcan 2022.1/bin/exe//bhead.exe" gns_bhm_model_eng_20220622.bmf'
 
I suspect it will not work. The methods of subprocess expect a list, not a single string.
Incidentally it's not recommended to create paths by calling str.join, and there is almost never a reason to write "{}".format(x) when you could just do str(x) instead. Or just x, if x is already a string.
Oh, and I don't think you need to put the exe in quotes. subprocess is usually smart enough to add them on its own, if it thinks they're necessary. (this is an oversimplification, but never mind that)
 
what about subprocess.Popen
 
6:32 PM
I expect Popen to behave the same as run and check_output and so forth
import os
import subprocess
vulcanbinpath = "C:/Program Files/Maptek/Vulcan 2022.1/bin/exe"
args = [
    os.path.join(vulcanbinpath, "bhead.exe"),
    str(bhmbmffl)
]
subprocess.run(args)
I think this is pretty close
 
k let me try that
 
@Kevin I've got a bad solution:
 select age, name from (
   select age, name,
     row_number() over (partition by age order by name) row_nr
   from user
 ) pointless_alias where row_nr = 1
 
Hey, I was just writing up an explanation of my own inadvisable(?) rowid based design :-)
 
For some reason you can't replace the inner select age, name with a select * ¯\_(ツ)_/¯
 
Ah, the function row_number() and the pseudocolumn rowid do different things. I think yours is more stable.
 
6:37 PM
By the way, you might want to use sqlfiddle.com for your future MREs
 
with rowids_of_representatives as (
    select min(rowid) the_rowid from user group by age
)
select user.* from rowids_of_representatives left join user on rowids_of_representatives.the_rowid = user.rowid
Basically I'm collecting the memory address of each representative row, and then recombobulating the pointers back into regular row objects
I'm 80% sure this won't go catastrophically wrong if Oracle decides to defragment the table after I've created the temporary table but before I've created the final results
@Aran-Fey Ooh, I like it.
 
Why would you go for rowing?
That's not to be trusted at all. What if I vacuum the db in the meantime? I need to read up on what you're trying to do
Rowid* stupid auto-correct
@Kevin this sounds like such a bad idea :/
The whole language itself is declarative. Why should a memory address of all things be consistent? I don't know conclusively either way but this seems risky business and... for what, exactly?
 
Still blows my mind that this is the query language that was adopted all over the world
 
The only reason I think rowid might maybe possibly be safe, is because here it never outlives the session that executes the query. I use it for intermediary calculations and then discard it.
 
6:52 PM
There's no possibility that I'll fetch the rowids, save them somewhere, commit, leave for eight hours while roganjosh vacuums, come back, and try to fetch the rows that used to have those rowids.
 
I'm sure you're doing something wrong, I just won't be able to look at it for over an hour sorry
I'm not running back home, even for a Kevin emergency :P
 
Valid
 
@AndrasDeak--СлаваУкраїні Not too sure how that's relevant here, but I certainly agree that we're bad at what we do
 
"rowids_of_representatives.the_rowid = user.rowid" did you set it as a foreign key? Please say you did
 
I did not use the words "foreign" or "key", as you can see. I apologize.
 
6:58 PM
Not even a "primary key"?
 
My real users table doesn't have a primary key. I'm happy to use any key you like in rowids_of_representatives.
 
OK,, just don't move. And definitely don't give those creatures water after sundown
 
users may or may not have a "fake composite primary key" in the sense that str(name) + str(age) + str(height) is guaranteed to be unique. If another 6 foot tall 30 year old man named Fred tries to register, we'll simply terminate the registration process with a 404 page.
 
Imagine trying to sign up to Kevin's website and getting hit with a "this age is already in use, please try another one"
 
Why is it "guaranteed"?. I'm trying to wipe out a script that relies on MD5 hashes for deduplicatio n and I want rid of that mess
 
7:03 PM
@Aran-Fey that mediocrity is what we do, so no wonder the crap spread wide
 
"Your name, 'Fred', is too weak. Try a longer name, like 'Frederick'"
 
"For security reasons, your height must contain at least 1 upper case number"
 
Assuming it's crap. I don't know any query languages.
 
Throw in some special characters to really make it strong. But not too special, we can only handle ASCII. "☃️ Frederick ☃️" will probably bluescreen the server.
 
For the UK audience; my best friend got told she needed "a strong name" for her daughter, like "Jordan".
About a decade ago, that is
 
Oh, nice. I checked the real users table's constraints, and it actually does define a composite primary key using the columns that I thought looked primary key-ish.
 
Huh. Is Jordan a... what's the word again? Bilingual? Bisexual? Amphibious? ...you-know-what-I-mean name?
 
You don't need a composite key. That'll help with deduplication but you can still join on it with part of the key
 
Gender neutral? Androgynous?
 
Gender neutral! ... yeah, I was pretty far off there
 
7:09 PM
@Aran-Fey Jordan was a complete mess that used to be in our news all the time. I need to be careful about why. She had adventures
 
I'd say Jordan is more common for girls in the culture of my tri-state area. But I wouldn't raise much of an eyebrow over a boy Jordan.
 
Huh. Up until 3 minutes ago I thought it's a male name
 
I primarily associate it with a big important river, and the surname of a powerful basketball player.
 
Now we'll never know if that river is male or female
 
It might also mean "flow down", which is certainly a thing rivers do
@roganjosh But if I only use part of the key, don't I lose my guarantee of uniqueness? I will return to my fiddle.
sqlfiddle.com/#!4/cfb405/1 Here I declare a composite key on (name, height). Then I collect part of the key for each representative. But now I have four results, when I expected three.
... But if I include both parts of the key in my temporary table, then Fred and Barney cease to exist. Oh dear. sqlfiddle.com/#!4/b1acc/1
sqlfiddle.com/#!4/b1acc/14 Temporarily creating a single primary key from the composite key parts, works in the ordinary case... But It would be bad if user Fred had height 72, and user Fred7 had height 2. Then their fake primary keys would conflict.
I know, I'll convert the two values to xml!
This is an excellent plan and I assume this relatively common problem is always solved in this exact way.
 
7:42 PM
@Kevin everyone knows the only "Word of Power" is "cabbage"...
 
There are lesser words, shadowy reflections of the true cabbage, which nonetheless can topple armies
I'm reading an article about practical uses of composite keys. They give a potential use case: a student's name is not guaranteed to be unique. A student's parent contact number is not guaranteed to be unique. But surely name + contact number is unique.
 
Yeah, no
 
Then, one day, George Foreman signs up his six sons for school. Their real actual names are: George, George, George, George, George, and George.
 
There are definitely lots of composite keys that are unique, but that is not one of them
I actually know a family who had twins and gave them both the same name
 
Ok, they're George I through George VI, so in principle you can tell them apart. But, oops, you already designed your table under the assumption that a name never contains a space. Sad trombone.
@Aran-Fey I am delighted by this. Unless the twins hate it. Then I disapprove.
 
7:52 PM
Not sure tbh, I don't know them that well
 
I will place my emotional state in a lambda expression, and defer its execution until more information is available
 
@Aran-Fey twins are the one exception when you can't give the same name, in Hungary, unless one of them was born before midnight and the other after. As long as they have different birthdates, it doesn't matter that they have the same full names.
 
Found the poorly designed database.
 
That sounds like a weird restriction. There could be a completely unrelated person with the same name and birthday. What happens then?
 
mother's maiden name less likely to coincide than for siblings I assume
 
7:58 PM
Perhaps that's fine if they have different parents. But god help you if their different parents happen to have the same names.
 
8:18 PM
@Kevin Well, what is it? Here you're looking for uniqueness but before you were effectively trying to drop dupes (how did they get in there?)
select min(rowid) the_rowid is an aggregation. Ergo, you have dupes
 
I have duplicate ages, but I don't have duplicate composite keys.
I don't know if that answers the question.
 
let me re-read with that context
 
The objective has moved around a bit since the beginning of the conversation, due to scope creep and my own confusion. Maybe I'll write up a cleaner problem statement when I find a minute.
 
There's a couple of things confusing me here. Firstly, I think you need UNIQUE specified on a compound key if you want uniqueness, and secondly I'm relatively sure that fields can appear in multiple keys
Throw in things like redshift that don't give a yam about any of this so it'll let you break your own "constraints". SQLite3 won't, though
Although SQLite3 absolutely will let you put a string in a float field. For God's sake, I'm starting to see Aran's continuous complaints with SQL :P
Just to make it extra fun; I'm pretty sure that NAN can't be used in a compound key for uniqueness in postgres. So even if you set a unique compound key, if there's a NAN in it, you're stuffed. Welcome to SQL
Sorry, it can be used. It'll just get ignored. NULL != NULL
 
9:26 PM
Cabbage! I want to plot more than 100,000 points on a wide single line chart (fig.set_size_inches(35, 10)) via Matplotlib, it's a time series, but in many "windows" the points overwrite each other (naturally), even using alpha attribute it's not possible to visualize them properly. I thought about using some sort of shift to space out the points in these windows more, any ideas what I can do?
 
Why do you need 100,000 points?
 
It's a large time series
 
Why do you need 100,000 points?
 
I'm actually plotting 2 time series, each with a color, the 2 have the same number of points,
 
Still not answering the same question, though?
 
9:33 PM
@roganjosh Because I need to plot all the time series
It only took me a few seconds to answer, I was writing something before
 
Why do you? If you only had 50,000 would it materially change the plot?
 
Of course, in that case it would not be plotting the entire series
 
So?
 
In fact, there are 165,312 points in each of the 2 series
@roganjosh Sorry?
 
@Marco I'm still looking for a compelling reason for why every single point needs to be plotted
<hint; it almost certainly doesn't need to be>
 
9:37 PM
Well, because I was asked to do so
 
That's unfortunate :(
 
But if it's really not possible to do that, no problem, I'll just justify it and a better way will be thought of.
I'm checking if it would be possible or not
 
It's totally possible to do that. You just have to wait while it gets rendered
 
I managed to plot but there are the issues I mentioned
 
the word you need is "jittering" but no matter what you do you'll lose fidelity
 
9:41 PM
Interesting
I've never heard of that :P
 
I guess fidelity is not the core concept. It's that visualization is about conveying information about your data, but there's no way to convey information with 100k points.
 
"it's not possible to visualize them properly. I thought about using some sort of shift to space out the points in these windows more, any ideas what I can do?" for 100,000 points?
 
but I'll leave that problem to whomever assigned this to you
 
@AndrasDeak--СлаваУкраїні Umm...
@roganjosh I've plotted, but with the issues I've mentioned
So even the jittering doesn't solve it, right?
 
And both of us are basically saying a) these "issues" are intrinsic to plotting so many points and b) the resulting graph is probably meaningless, regardless of what you do to it
 
9:46 PM
Yeah
I agree
Is it worth it to test jittering?
 
10:23 PM
Apparently the jittering didn't work with the few tests I did. I thought about the following: would it be possible to make a line or scatter plot in which the denser that region (the more overlapping points between the points of the first and second time series), there is a progressive highlight in that region?
denser a region*
 
10:36 PM
You could try some kind of binning to end up with a 2d histogram instead of a scatter plot, which you can visualize with a heatmap for instance. This only makes sense for certain kinds of data though.
@Marco I think you have to find someone more senior who understands your task better than we do, and ask them for help.
 
@AndrasDeak--СлаваУкраїні Interesting, I will try to do this
@AndrasDeak--СлаваУкраїні Umm, but you are helping me perfectly, you made me understand that it is not possible to do what I want to do
Thank you both (Andras and Roganjosh)
but you and Roganjosh*
 

« first day (4300 days earlier)      last day (652 days later) »