« first day (3839 days earlier)      last day (1092 days later) » 

1:11 AM
from a keras callback, is there a way to gain access to the model that uses the callback. More specifically, if that model is an attribute on some class, can one call other class methods from the callback?
```
class Autoencoder():
def __init__(self):
self.model = 'this is the keras model'

class PlotSampleOutput(Callback):
def __init__(self):
pass

def on_epoch_end(self, epoch, logs={}):
print('HERE I WANT TO CALL auto.yeet()')

auto = Autoencoder()
auto.fit(X, X, callbacks=[PlotSampleOutput()])
```
why on why can't the chat use the same markdown as the site
 
 
3 hours later…
4:41 AM
looks like github changed their badges, there is a new badge pic for Artic Vault and any other badges
 
 
2 hours later…
6:34 AM
um... i want to try making a chatroom in python, but i want to understand what i'm doing. there are so many terms i do not understand, like socket, ip, dns, ... you know, internet related stuff. i was wondering if, there was like, a keyword, which i can look for in the internet to get a briefing on all of them...
 
@python_user Yup, and some got a badge related to the Ingenuity helicopter flight on Mars. Pretty chuffed about that :)
 
7:09 AM
@Maggot Just open up wikipedia of each term. There you will find then other links to unexplained terms. It's a good starter for a high level overview
I want to use multiple inheritance, change my mind :D
 
@Hakaishin Use Java, then there is no need to change your mind :p
 
7:57 AM
@Hakaishin Go for it. smiles
 
@MisterMiyagi yes, I'm at the smiles state. A bit more and I'm dropping it :P
 
Always remember Rule One.
Seriously though, multiple inheritance was the appropriate tool for me several times. Don't see anything wrongiger about it than regular OOPity.
 
hmmm, seems as it's as simple as
def __init__(self):
	A.__init__(name)
	B.__init__(name)
If I have non cooperative classes
 
Now would be the right time to make them cooperative.
 
I don't control 1 and I don't want to really write a wrapper
anything wrong with the above manual approach?
 
8:11 AM
It makes multiple inheritance super fragile. Works if you don't do anything fancy, but can fail in weird ways the moment you do.
 
8:22 AM
In specific, since multiple inheritance uses a dynamic, non-linear MRO, you can easily get cases where the static super dispatch is wrong.
 
8:42 AM
I really don't want to do anything fancy with it. It seems too complicated and I need to write more code
 
9:40 AM
stackoverflow.com/q/67175720/6251742 This question obviously already have an answer on SO, but the answers linked to the close doesn't seems to fit the issue
am I wrong ?
like Augurar said : "I don't think that will help the OP, as in that case it's a subtlety of how the * operation works for lists, whereas in this case it has to do with confusion about variables/references vs values"
Is it possible to edit the close link to a better already answered question ?
 
@DorianTurba someone with a gold badge on the Q's tags can edit the duplicate list...
 
(by obviously I mean that a question about binding the same list to other variable, not * operator on list)
@JonClements hum ok, so nothing can be done by me
thanks
This is weird to close a question about bind and copy with a list * operator question...
For newbie, it might be hard to understand the issue
 
yeah, if the dupe is correct but needs a logical step it's good form to leave a comment explaining that
I've added a few dupe targets and made the best one first
 
I like this one
 
10:15 AM
the more the merrier
@DorianTurba heads-up, if "obviously" the question has an answer on SO then consider not answering it
 
10:38 AM
Hmm, I haven't experienced this variant of the DenverCoder phenomenon before. I googled the header of an unknown file format and the one result is a github project saying "This software has been created purely for the purposes of academic research and for the development of effective defensive techniques, and is not intended to be used to attack systems except where explicitly authorized."
It's nice that DenverCoder is evidently an experienced security dev, but he might also be an international crime-doer
 
@AndrasDeak The issue isn't there, I don't care about my answers being useless, I car about the op redirected to another question where the link isn't obvious for beginners
 
by day and by night jobs? :p
 
For me, if it should be closed for duplicate, at least link to good one
 
I assume all security devs go home at night and put on their balaclava so they can do some hacking with their red glowing keyboard
 
@DorianTurba yes, we agree. "Closed as duplicate". Not "leave another answer". I don't think there's any contradiction here.
 
10:46 AM
FWIW I used to leave an answer and close as duplicate, if I felt like there were enough fiddly details that the OP couldn't easily figure out the answer just from the dupe target. I never got in trouble for it, but eventually I switched to comment+close
 
just you wait until the reckoning
 
More to avoid the appearance of impropriety, than to avoid actual impropriety, IMO. In other words, I wasn't trying to game the system by making sure nobody could answer the question after me, but I didn't want anyone to think I was being unfair
 
Um... since we know there can only be one Kevin... shouldn't that be the "quickening"?
 
Quickening is on Thursday, then Reckoning on alternating Tuesdays after that
DenverCoder's wisdom gives me the power to pwn the application that saves/loads this file, but I would really just like to read its contents please
 
Anyone seen a selfie screenshot before? What would be the explanation? Windows has the interpreter associated to the file extension and executes the script. Why wouldn't the output get directed to the terminal? Couldn't find a duplicate for this one.
 
10:58 AM
it's deleted anyway
 
Just noticed.
 
Love the tag. Reminds me of .
 
This is why libraries should only choose meaningless words for their project name
Style points to Django for just being a dude's name
Gonna name my xml deserializer "Anthony"
Ok, I'm pretty sure this save file is basically json with plaintext key/values interspersed with a lot of human-unreadable .NET gunk. So a simple ctrl-F proves that the key I'm looking for isn't present. Thus ends my hacking-my-own-personal-user-data project.
I might have soldiered on for the fun of it, except DenverCoder mentioned the "weird" variable length ints the format uses, which are something like Unicode code points. It's too early and too Tuesday to do anything remotely like Unicode.
 
11:32 AM
Actual on-topic question: while approximately following this OpenCV tutorial, if I do import cv2 as cv; print(cv.samples.findFile("starry_night.jpg")), I get OpenCV samples: Can't find required data file: starry_night.jpg in function 'cv::samples::findFile'.
Does this imply that I haven't configured OpenCV properly, and it can't find its samples directory? Or is this tutorial outdated to the point that starry_night.jpg has since been removed as a sample? Or something else?
 
well there is warning in the tutorial
 
I'm currently browsing through the locations listed at docs.opencv.org/master/d6/dba/…, no luck yet
Perhaps opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/… is less outdated. It doesn't use findFile at all, instead depending on the user to provide their own image.
I could follow this instead, but I don't know if I trust "opencv python tutroals"
 
I like how opencv people don't want to waste the reader's time by putting up visible "how to install" links on the README/documentation/etc.
 
pypi.org/project/opencv-python talks about installation options. TLDR: pip install opencv-contrib-python should give you a functional version with some extra bells and whistles
But apparently not a samples directory???
 
11:44 AM
Ok, good sign
That "less outdated" tutorial says "This tutorial is meant for OpenCV 3x version. Not OpenCV 2x" on its github page, so maybe I shouldn't use it with opencv-contrib-python-4.5.1.48.
I'm one entire X ahead of them
 
I'd try building from source first
 
Sounds hard, maybe I'll just write a monkeypatch that downloads sample images on-demand from their github
 
12:02 PM
Monkeypatch successful*. And I was even nice enough to make it only download the image once.
(*Except I have to put import samples; cv.samples = samples in all my scripts now. I don't know to achieve perfectly transparent monkeypatching.)
I guess I could create a local cv2.py file that imports all of the real cv2 into its namespace, along with my findFile patch
 
[i-dont-know-why-you-didnt-just-download-them-manually,-and-at-this-point-im-afraid-to-ask.png]
 
12:24 PM
I certainly could have downloaded starry_night.jpg, that's easy. I don't know if I can download the entire directory from the github page. Using git directly is obviously the "right" way to pull stuff, but I can't be bothered to remember the syntax
 
12:35 PM
I'd just clone the whole repo and then copy the images where I need them ¯\_(ツ)_/¯
 
1:01 PM
Next time for sure.
[arrested development narrator: "He was lying."]
> Suppose you are searching for an object which has multiple occurrences, cv.minMaxLoc() won't give you all the locations. In that case, we will use thresholding. So in this example, we will use a screenshot of the famous game Mario and we will find the coins in it.
Ah, yes. The classic game, "Mario".
I wonder if openCV's template matching is overkill when I only need to find pixel-perfect matches of my template image within the larger image
 
1:16 PM
@Kevin what task do you search for pixel perfect matches? Never heard of
 
GUI automation. for example "search for a button that looks exactly like res/the_button.png, and click it"
 
hmm, doesn't this sound like an xy problem? Like if the gui can do things, can't you just call the things behind the gui to automate the task?
 
hey guys say you have a pd series of dtype fload64, and you want to remove the decimal and convert to string how would you do this? i have tried the likes of df.astype(int).astype(str) and recieve ValueError: Cannot convert non-finite values (NA or inf) to integer
 
Not if it's a closed-source executable written by someone that has no incentive to provide a programmatic interface
 
@Kevin ah politics (:
 
1:20 PM
um... didn't there use to be one a while back that I thought was fairly cool - began with S maybe?
 
@Kevin just a bit
 
It's not even the dev being mean or wanting a bribe or whatever, he just doesn't know I exist and there's no reason to think my support ticket will get priority over the five dozen already there
 
@Kwsswart Well, what result do you expect for inf/nan?
 
i'd let open cv deal with it anyways, it's all ready to be used
 
@AndrasDeak Ah, I see you received my telepathic plea sent to all numpy knowers.
 
1:22 PM
also, cbg folks
 
(decently sure you can also just apply a format pattern)
 
do the buttons appear in fixed locations by any chance? wondering if you could just use something coordinate based instead
if we're talking simple, that's probably the simplest...*if* the assumption holds
 
Ahh... think I'm thinking of Sikuli...
 
The coordinate based approach would have to work given that any deviation from a fixed rendering of the interface would break any template matching
 
I believe I tried Sikuli a while back, but it didn't quite support some things I wanted out of the box. I forget the details, though. I'll check it out again if I get annoyed with reinventing the wheel.
 
1:25 PM
template matching has been surprisingly robust in my sample size of 1 prior task, so i've got some faith in it.
huh, never heard of Sikuli
 
I've had moderate success with fixed coordinates in the past, but with a few notable catastrophic instances where the UI would lag/hang (perhaps because of a "do you want to update?" pop up that refuses to yield focus) and my script would keep clicking where it thinks the buttons currently are, but aren't
Confirming that the button is actually on the page before clicking it prevents haywire outcomes
 
@Kevin yeah... never had a use for it myself so couldn't offer any real opinion as to it's suitability... just remember coming across it and thinking... ooo.... interesting...
 
yeah that makes sense
 
Additional small nice-to-haves: 1) my program would continue to work if the window's resolution changes and the elements get nudged over; 2) During initial development I wouldn't have to transcribe the coordinates for each element I want to interact with.
 
@Kevin you could max the attempted clicks at 3, not sure how you are verifying that any click is successful, however.
 
1:33 PM
template matching should get you both of those i believe
 
@Dodge I have considered executing a smallish batch of clicks, and then waiting for the user to press a key to confirm that everything is still on track. My current use case needs to make about a hundred clicks, so batches of ten (for example) would be reasonable for the user to supervise.
 
Automating mouse clicks, that's interesting. How do you do that?
 
@MisterMiyagi Thank you so much for that comment got the answer
 
just my suggestion but pyautogui has some template matching by default maybe pyautogui.readthedocs.io/en/latest/…
 
didnt realise what was happening until that comment ^^
 
1:48 PM
@Dodge Basically win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN,x,y,0,0)
 
@Kevin yes it works reasonably good
 
Which requires the third party library interface to the Windows API, and that library is named...... Uh, I don't know.
It's under "win32" in site-packages, so I guess that.
Or perhaps "pywin32" because there's no "win32" on pypi
 
@python_user pyautogui is pretty cool, hadn't seen that before. Looking at the source, I realize the whole concept is far less mysterious than it seems.
 
Love it when the pypi name is different from the package name, which is different from the module name
 
@Dodge I know that it exists because I saw some youtuber making an autoclicker (using pyautogui) to win a flash game
 
2:00 PM
en.wikipedia.org/wiki/String-searching_algorithm talks about how to make small_str in big_str run faster than O(len(big_str)). Exact template matching is essentially a 2d string search. I wonder if I could use anything mentioned there.
I could take a 1 pixel tall slice of the template, and do a 1d string search against each row in the image. If I choose a slice based on the highest entropy*, then that maximizes fail-fast and minimizes false positives.
(*in other words, the line that is least likely to coincidentally appear more than once in the image. The topmost row of the button template is likely a solid line, so it would be a poor choice. A slice going right through the center of the button's text would probably be best)
 
2:14 PM
Me: "Hmm, maybe numpy has special magic to make 2d sliding window iteration faster than O(N*W)"
Numpy docs: "a sliding window approach with an input size of N and a window size of W will scale as O(N*W) where frequently a special algorithm can achieve O(N). That means that the sliding window variant for a window size of 100 can be a 100 times slower than a more specialized version."
 
morning cabbages folks
 
2:36 PM
@Kevin that's a new helper, meant for prototyping
The specialized solution is convolution which is pretty much template matching here
That being said I'd use the sliding window anyway
 
Hello
Anyone who has experience with MSYS?
 
ParseError: Unterminated clause. Expected predicate, found ? instead
A more helpful response might be "welcome to room6. Ask your question. If anyone knows, they'll probably speak up"
 
 
1 hour later…
4:02 PM
How hard would it be to rig the type annotation system so that I can tag numeric measurements with arbitrary combinations of SI units? e.g. accel: Meter/Second**2 = 10
 
I know there are some libraries that implement units, but I'm not sure if they can be used as annotations
 
Maybe I could also benefit from non-SI units, so I can have widgets per second and splines per sprocket
I guess it doesn't have to be annotations necessarily. Just something that makes it apparent that I have a logical error in a code path by the time it has finished executing
My sample input data unfortunately has a couple ones in it, so it's not always obvious that I multiplied by a unit when I should have divided
 
@Kevin Your main problem will be to teach some tool what Meter/Second**2 actually means. I don't think the tooling has support for custom type operators.
As for annotating, PEP 593 is likely adequate.
If you don't need the fancy unit math, you can just define a NewType for Acceleration or MpS2 or such. That should work out of the box.
 
Yeah, my actual combinations of units are quite finite so it wouldn't be hard to define them by hand
 
anyone know how to force pycharm to do python syntax highlighting on a non-py file? In this case it's a cgi file.
 
4:16 PM
@piRSquared Most editors allow to select the highlighting template for each file.
@Kevin Go for NewType then, unless you find it is lacking. And be wary that natural units are not a safe type system.
 
I assumed as much. I'm fumbling around atm. My google skills are also lacking
 
Oh my, need more coffee... :/
 
no, I forgot to add it and had to edit it in. You read it before I added "pycharm" so the fault is not yours
 
IIRC if your file already has highlighting, do right-click and "Mark as Plaintext". Then on the next right-click you should be able to select a file type for highlighting.
 
Is this an appropriate place to ask whether a Django question is appropriate for SO or not? If not, where might I go to figure out whether it's topical? My question is a flavor of "What should I do to improve performance of a write-heavy django app?" (where I would enter details into the question explaining our current architecture, and what the performance data shows)
 
4:23 PM
Dang, the siunits module looked like it would be suitable for lightweight runtime type validation, but its type comparisons aren't as robust as I hoped
import siunits as u
def test1():
    distance = 23 * u.m; time = 23 * u.s; velocity = distance / time
    assert velocity.coef == 1
    assert velocity.unit == u.m / u.s

def test2():                                              # wrong v
    velocity = 1 * u.m / u.s; time = 1 * u.s; distance = velocity / time
    assert distance.coef == 1
    assert distance.unit == u.m

test1() #expected outcome: no error. Actual outcome: no error.
test2() #expected outcome: AssertionError. Actual outcome: AttributeError: 'BaseUnit' object has no attribute 'base_units'
@jchung Yeah, we often talk about the art of improving Python questions in here
 
@MisterMiyagi Oi! I found it.
 
@Kevin That's... not the kind of error I expected.
 
@Kevin Thanks! In your opinion, is my question ^ topical for SO? I'm used to asking simple questions with objective answers like "Why did I get this error message on this code?" or "Why is this function returning this thing?" I think that the current question I'm looking for advice on ("What should I do to improve performance of a write-heavy django app?") might be a bit more subjective. Is that ok for SO?
 
"How do I improve the performance of my program?" can get a negative response on SO, since it rather intrudes on Code Review's turf. But sometimes you can get good answers from people that don't know there are other Stack Exchange site. It's a gamble.
 
Generally it's not, unless you can clearly define what "improve" means. Questions that allow many wildly different answers are hard to answer properly and thus not well received.
FWIW, a "write-heavy django app" sounds more like the problem is with "write-heavy", not django. If you can nail it down to some specific bottleneck and some specific optimisation goal, that should work better.
 
4:31 PM
Along similar lines, going to Code Review and posting "how do I improve a write-heavy django app that also has X Y and Z in its tech stack?" without providing a complete MCVE, will probably get a negative reaction, because there's nothing concrete for them to review.
 
@jchung There is a blurry line between code review and stackoverflow. I've often posted improvement type questions on stackoverflow but I think yours is more along the lines of "This works but I need advice" That, imo, goes on code review
 
Last time I went to CR, their its "post entire, real programs" rule proved... impractical.
 
Hi I'm wondering what the proper way to set kwarg params in a class is?

```
class A:
def __init__(self, **kwargs):
self.__dict__.update(kwargs)
self.kwargs = kwargs

def __getitem__(self, idx):
idx_kwargs = {}
for attr, value in self.kwargs.items():
idx_kwargs[attr] = value[idx]
return idx_kwargs

temp = A(x=[1,2,3], y=[-1, -2, -3])
```


```
(Pdb) temp.x
[1, 2, 3]
(Pdb) temp.y
[-1, -2, -3]
(Pdb) temp[0]
{'x': 1, 'y': -1}
(Pdb) temp.x = [4,5,6]
(Pdb) temp[0]
{'x': 1, 'y': -1}
(Pdb)
```
 
@genescuba You might want to start by learning the proper way to format code in char
 
As a distant runner up, Computer Science is often quite comfortable discussing optimization problems that have no code snippets. But they're more interested in, like, rotating red-black trees in O(W * log(omicron)!) time, than figuring out how to reduce I/O overhead in a web framework.
 
4:35 PM
Can you clarify why you want to set the kwargs as attributes on the class itself? Setting them as a single (private) attribute like kwargs should be sufficient.
 
If anything, just mentioning a specific architecture is too concrete for them
 
log(omicron) /wistful_nastalgia... good times.
 
Hmm, maybe I'll ask them about my image matching two dimensional string search problem...
 
@MisterMiyagi ah it wont let me edit no more :( . I'm not sure what you mean by your second message. Are you asking why I'm doing self.kargs = kwargs?
 
@genescuba No, I'm asking why you are doing self.__dict__.update(kwargs)
 
4:38 PM
"Welcome to Computer Science! You don't have to preface your question with 'I already have an O(N^2) solution that works fine for all my practical inputs, but out of curiosity Im interested in something better', because that is assumed for all posts on here"
That's like saying "hello, nice weather" to them
 
Ah I see, I'm doing this simply because I want to access the attribute like A.x rather than A.kwargs["x"]
 
@piRSquared If you were working on problems that required third string Latin letters like omicron, I can only assume you're wistful for the time before that when you still had your sanity
 
@genescuba Consider whether you want to use/inherit from types.SimpleNamespace then. It does that out of the box.
 
s/Latin/Greek, it's all Greek to me
 
"Third string Latin letters like omicron"?
 
4:42 PM
I'll fetch the pitchtorches...
 
Thanks everyone. I reviewed the other sites and saw ServerFault, Software Engineering, and DevOps.
I may end up rolling the dice on code review first and see how far I get.
 
@Kevin also not type annotations but astropy has unit dtypes or something like that
 
How can you not tell the difference between Latin oh: O and the Greek omicron: Ο
 
Just listen to the screams of people raging against LaTeX.
That's a sure sign they are dealing with Greek letters.
 
Greek is great. IPA? Oof...
 
4:50 PM
ɪnˈdiːd
 
Official Greek letter power ranking
S tier: Alpha, Beta, Pi. The superstars. Even normies know about these.

A tier: Epsilon, Sigma, Phi, Psi, Delta. Useful and common in at least one of calculus, trig, algebra, or floating point math.

B tier: Iota, Lambda, Tau, Omega. Lambda and tau get bumped out of C tier for being the namesake of lambda calculus or a humorous alternative to pi, respectively. Iota and Omega are occasionally used by normies in non-math contexts.

C tier: Gamma, Zeta, Eta, Kappa, Mu, Nu, Xi, Omicron, Upsilon, Chi. Literally who?
 
Edit "theta" into A tier quick
 
Excuse me? Gamma ray burst super soldier serum side-effects anyone?
 
Ok, Gamma can get a bump if we get a quorum of astronomers and marvel movie fans to ratify it
If you're saying Theta deserves S tier because it's the name of a blockchain company, I don't want to reward them and risk incentivizing more blockchain companies to come into existence
Oh, you're pointing out that it's not in my list at all. My bad. It's the bread and butter of trig, so it gets A tier easy.
If Phi is A tier, Theta is A-plus tier
 
@Kevin frat boys?
 
5:01 PM
Mm, I feel that their popular culture influence has faded since the college movie golden age, so I'm not willing to give much additional weight to letters they like to use for their houses.
 
@AndrasDeak For C Tier?
I agree that greek letters used by frats/sororities belong in C Tier anyway.
 
yes, answering the only question in that message
 
Possibly Kappa can get a bump from a quorum of frat boys and Japanese mythology buffs, if they can stay in the same room together for more than ten minutes
 
I was thinking the same about Iota and Omega. re: frats
 
It doesn't count if the mythology buffs can't leave the room because they've been wedgied and hung upon the coat rack
 
5:06 PM
Rush week was a boon of free pizza.
 
raf
5:23 PM
Hi, I like to use a specific font (chewy) in my streamlit web app (https://github.com/rafisics/suvat_calculator/blob/main/suvat_calculator_en.py#L24). I have done the following:
```
st.markdown("<div><h1 style='text-align: center; color: #ff7903; font-family: Chewy'> SUVAT Calculator </h1></div>", unsafe_allow_html=True)
```
I can see the font because it's installed on my computer. But I want to make it visible from every browser without having the font installed. How can I do that?
 
5:36 PM
btw guys, does pandas by default not come with any excel writing library. had module errors, no openpyxl when I swapped another system and was running some basic test scripts. Are license issues that bad with anything excel related?
 
you'll need xlrd or openpyxl. pip install pandas usually installs some of these deps, but I've seen it happen where you have to install one or more yourself
 
ended up having to install both, still figuring out issues xlrd saying XLRDErro: Excel xlsx file; not supported
 
> excelData = pd.read_excel('students.xlsx', engine='openpyxl')
 
@Skyler Chris (the current maintainer) of the xlrd library deliberately introduced that change as there was a potential security threat when using it to read .xlsx files... depending what version of pandas you're using, it should automatically try openpyxl by default now for those, or you can specify the engine='openpyxl' option
 
yea, I think since I was working with an iostream an not an associated filename it defaulted to trying to look at an xls file file when xlsx was what had been written
 
 
1 hour later…
7:09 PM
thank you, SO...
 
aah, perfection
hmm, they did revamp the interface apparently
 
7:29 PM
hi! 
I’ve been trying to see if I could generate a random matrix, such as:

```
import numpy as np
data = np.random.randint(242, 255, size=(1200, 1200, 3), dtype=np.uint8)
```

but instead of having a normal distribution of integers, I'd like to code it in a way that most of the matrix values are 242 and the rest numbers (243 to 255) appear but in a much lower proportion but haven’t been able to address this. Do you guys have any idea about how this could be possible??? 

as the size of this matrix could be really big, I’m trying to avoid using a couple of for loops to prevent a very long r
 
I mean, I'm sure it would be hard for the engine to check that I have done 1100 reviews already.
@lorelayb take np.full((1200, 1200, 3), fill_value=242) and then generate a handful of random indices that you'll replace with randints
No loops necessary
Also chat markdown doesn't support many things, please see our code formatting guide to chat
 
7:48 PM
@AndrasDeak I see! I believe I would need at least 1 loop so I can replace the random indices with randints, right?
@AndrasDeak thank you!
 
@lorelayb no, you'd want to use advanced indexing
import numpy as np

arr_shape = 2, 3, 4
n_replace = 4

rng = np.random.default_rng()
arr =  np.full(arr_shape, fill_value=242)
indices = np.random.choice(arr.size, n_replace, replace=False)
values = rng.integers(242, 255, size=n_replace)
arr.flat[indices] = values
>>> arr
array([[[247, 242, 242, 242],
        [242, 242, 242, 242],
        [242, 242, 242, 248]],

       [[242, 242, 249, 242],
        [242, 242, 242, 242],
        [242, 242, 242, 242]]])
you can only see 3 replaced values because one of them ended up being 242
>>> values
array([248, 247, 249, 242])
 
8:03 PM
@AndrasDeak awesome, did not know about advanced indexing. thank you so much, taking a look at this approach.
 
@lorelayb then I recommend giving numpy.org/doc/stable/reference/arrays.indexing.html a read
 
@AndrasDeak thanks! will give the documentation a read
 
8:47 PM
@AndrasDeak whoops, that middle line should read rng.choice(...), sorry
so much for best practices
 
Does anyone have much experience with Markov chains?
I have a feeling that my problem is pre-solved and is an extension of the principle. Just curious if anyone will stick their neck out and boot me in the right direction :P
 
it's been a while, and I never was much of an expert, but shoot
 
I'm dividing a delivery driver's "patch" into a 2D grid (let's assume that lat/long is Cartesian - min/max of lat and long, and I create a grid). Every day, they'll get a selection of sectors they'll have to deliver to, so I can map that out fine and see how they prefer to move from one sector to another
 
ooh! this sounds interesting
 
9:02 PM
So, I can see if they really like going from (22, 34) to (22, 35) in my grid system. But; they'll only do that on the day they actually had to go to (22, 35) in the first place
So I can see the probability of moving from one sector to another, but I'm failing to account for the case where they had a choice. I could account for the number of times they went from (22, 34) to (22, 25) but I'm totally losing the data on whether they had a choice to move to a different grid square. Maybe they'll always choose to make that move when the opportunity exists. They're always going to make a move, but I'm unsure on how to weight it against the choice they were given each day
 
I would look to Bayes' Theorem for this, myself. But that might just be due to a lack of experience with Markov chains
 
Well, I'm not wedded to Markov chains
 
@roganjosh well, that sounds a bit pickle-y. The point of Markovian processes is that they have no memory, so what matters is only what you have "here". You need a static graph that encodes the transitions between grid points. Although you could derive a model that depends on the given trip itself, but that seems to defeat the purpose of a Markovian model? What exactly do you want to get from this model and what is its input?
 
formulate the problem like this:
how much does driver like driving from A -> B?
find days in which A,B,X exist in the points to drive to, where X is any other point
Now determine how often A->B happens and A->notB happens
you should now have all the pieces for Bayes' Theorem
 
My point is that in the most naive approach "driver goes from A to B with probability P(A, B)" is the best you can do for a Markovian process that belongs to a given driver (rather than driver + additional context).
 
9:14 PM
Is it possible to make a subset of your plot, only showing a route if there was a choice available against that route*?
I suppose what im trying to figure out is, what's your big picture goal here.
 
I have to predict a route with nothing other than lat/longs (don't ask)
 
gdpr strikes again!
 
Ah! Then you don't need a Markov chain. You need a crystal ball.
 
laurel. yeah, i think a crystal ball will help
 
"These are the lat/longs they've got to visit. How will they do it?". But I have a lot of historical data and a completely arbitrary view of a random driver shows something that my "human" brain sees as a pattern
 
9:18 PM
I'd go with 1/4 transition probability, modulated by rivers and canyons :P
 
maybe do a cop-out
 
There's probably a lot more nuance there, due to local road structure. But I suspect any real insight can come from averaging the hell out of your data.
 
basically, give them top n paths with some confidence
as for the paths themselves, perhaps cluster your drivers, see if your data sorts itself out somewhat nicely
 
If point A leads to point B with more than 1/4 chance there has to be a reason. Many one-way roads, large avenue that's hard to cross, perpetual traffic jam on that road, things like that.
 
then, for each bucket of drivers, see what is the trend for their section choices
just dump all of that in decreasing order of bucket size as a proxy for confidence into the final result perhaps
 
9:20 PM
@ParitoshSingh this has been a nightmare in itself because I have to be able to compare predicted and actual sequences without the metric blowing up
I think the basis of gap penalty might give me a metric to compare the sequences where, for example, a customer wasn't available and the driver re-visited at the end of the day
@AndrasDeak I can actually, very easily, draw on road structure if needed. But I think a greedy route will fail. There seems to be a pattern in the drivers' own mental greedy model
So I think it's easier to draw on their sector visits vs. greedy road network
 
@roganjosh no no no, I didn't mean factoring in the road structure yourself. I meant as an explanation for any underlying pattern that might be there.
> But I suspect any real insight can come from averaging the hell out of your data.
 
that's interesting. I would have expected a BFS spiral, tbh. But perhaps local road structures are not conducive to that
 
are your routes matching historical data? Like say, if i ask to predict sectors ABCD, am i going to find historical data that has atleast ABCD in the driver paths?
 
@ParitoshSingh No, there's no guarantee. I need to figure out what I do in that case (I'll probably take a greedy approach on road network driving time). I still think that dividing the map into sectors is the best way, I just can't account for the fact (properly) that different combinations of sectors need to be visited each day, and the relative probability of choosing one over the other when a particular choice arises
FWIW this is not a fun project, but hey, it's landed on my desk
 
Im thinking that might be a harder problem to solve, and one you may be able to bypass.
 
9:27 PM
Is there some python style guide that will disuad my colleagues from using kwargs and varargs everywhere (possibly both in the same signature...)?
 
but i might be oversimplifying the issue in my head
 
@Mikhail import this
 
So how about this. assuming your sector method as the groundwork, you take the largest chunks of sectors that are present in historical data at least n times. or some other way to threshold it. So, suppose the goal is ABCDEFGH, and you find two subsequence of sectors ABEH and CDFG. Then, suppose your historical driver data had BAEXYH, you simplify it to say BAE-H was one observed pathway. just get the frequency of all pathways possible for your sector subsequence
pick the top pathways as hopefully ideal ,and just go with that
no models as such, purely picking from historical data
 
"pick a known path" is technically a model :P
 
grumble grumble touche :P
 
9:36 PM
one with limited predictive power, mind you
 
That could work if I fill in the blanks with a greedy route from the maps
I have an acceptable margin of error on my side
 
I hope this model will only be used to choose colours for lunchboxes for the drivers or something equally important.
 
In any case, no standardised model name has popped up from this, and I don't want to push my stress on the room too much. I'll figure something out. Some adaptation of a Markov chain with re-normalised probability based on the sectors that needed to be visited was my best first guess
@AndrasDeak Yeah, no :/
 
laurel. now im trying to imagine what colour lunchboxes i'd refuse to use
purple and brown seem outlandish enough that i would probably not want those
 
Purple is fine. Brown is weird.
 
10:12 PM
This is simple, no? You can have a lunchbox in any colour you want as long as it's blue?
 

« first day (3839 days earlier)      last day (1092 days later) »