I'm trying to figure out how to pre-process MNIST images so that the output of the function would be the same in case of translations of an image: f(an_image) = f(same_image_but_slightly_translated)
Motivation behind this is that I'm trying to design a custom distance-function that would use the output of that function (or maybe I don't even need such a function) that would ideally realize that an image translated by 1 pixel is the same as the one not translated (applying L2-distance-function to those two images would return a big distance, which is counter-intuitive).
I have MNIST, and can do whatever I want with it before trying to apply my similarity-metric to it. Your diagram, although not exactly clear, seems to indicate that you understood. :)
sounds like a straight forward feature to implement. There should be plenty ways to do it, a simple one would be to split the image into pixel chunks and putting them in buckets. If two images have very similar buckets, they are probably shifted.
You know what I mean?
def chunkify_image(image: List[List[float]]) -> List[List[List[float]]]
# n*m pixel image produces some < n*m chunks
def hash_chunk(chunk: List[List[float]]) -> String
def image_similarity(image: List[List[float]], other: List[List[float]]) -> float
# chunkify both images, hash their chunks, compare how many they share, return share-ratio [0..1]. Done.
If they are shifted, they should hold equal values
anyways, the problem doesn't become much harder if you want it to work with a close-enough-compare as opposed to an equal-compare
The example i used in that repl was shifted one pixel to the right, and with 7*7 pixels and a chunksize of three it gives 0.84 similarity, which is a good start imo
What's the consensus here on opinion-based style / design questions on Python?
they're gathering close votes pretty quickly these days. but I always felt Python was designed to be read and so it's a core part of learning the language.
Hey, sorry for interrupting previous discussion:^ does anyone have experience with GitPython?
I'm trying to get some commit data (I want to access code files that had been committed in a particular commit), but all blobs and trees inside it have paths pointing toward .gitignore.
This is probably a rather silly question, but I'm new at this and I've been trying to figure it out for such a long time. I would really really appreciate any help
@coldspeed, There's a reason why people put headings like "Explanation", "Example", etc. The majority of readers (we hope) aren't OP but have come from google. Humans aren't good screen readers, they have low attention spans online, and (mostly) a left-aligned bias, the headings help align people's eyes to what they want at any specific time.
Ha, yes. I do like headings generally (using ### for mark-up), esp if there are alternatives. For experienced users, putting the method name in the title helps give the gist at a glance.
I have a silly question. I'm looking into feed parsing for the first time (arxiv.org specifically, which is in Atom 1.0) and I'm trying to decide what libraries to use for this. The feedparser and beautifulsoup libraries both sound appropriate for this; would anyone have a recommendation for one or the other?
Hi, could someone point me in the right direction for the following?
I'd like to populate a dictionary with all of the functions that have been defined within a script. With the following code, is it possible to populate the data within the `functionDict` dictionary automatically/at runtime?
And, further, how could I manipulate the keys of the dictionary via `input()` (e.g. choosing `f1` (or any another string) as the key for the `function1` function (which would be the value of the aforementioned key)?
def showCmds():
print(functionDict.keys())
functionDict = {'f1': function1, 'f2': function2, 'help': showCmds}
def main():
while True:
selection = input('Selection > ')
if not selection:
break
elif selection in functionDict.keys():
functionDict[selection]()
else:
print('...')
Essentially, I'd like for functionDict to be filled with all of the defined functions within a script. From there- perhaps the script could prompt whether I'd like to define the names of the keys for each function or to allow the script itself to assign keys automatically for each value (which would be the name of a function that has been defined in the script)
I guess you can always parse globals() and pick out functions from there, but I don't know if it's possible to distinguish between functions defined in the given module and the rest (my guess would be "no")
Anyway, I don't know enough about the subject to be able to help. I could guess but this is messy business and some ways are surely more robust/safe than others
>>> def myfunc():pass
...
>>> {func.__name__: func for func in globals().values() if callable(func) and not isinstance(func, type)}
{'myfunc': <function myfunc at 0x000000000040C1E0>}
There isn't really a goal for this script. I'm simply learning-- and after reading various questions/answers concerning this on SO I've reached the code that I sent above.
Oh! Could you perhaps elaborate a bit more on that please? @marxin ? Could you throw together a bit of code to illustrate how to do this?
I'm still quite a beginner with Python; however, I have read about both decorators and dictionary comprehensions (which is what that last piece of code looks like).
Could you perhaps show me what it would look like -- essentially where to place that in my original code?
Would it be useful or redundant to use both a decorator, such as the one @marxin mentioned- and what @Aran-Fey wrote? What are some pros/cons to each of the two?
Awesome! Thank you so much @marxin -- the visual makes it perfectly clear.
And thanks Andras and @Aran-Fey
If I were to use the Cmd module instead of the vanilla input code within main(), would I need to modify anything for the decorators to behave in a similar fashion?
@MooingRawr: I'll still pop in from time to time, and I'll still answer questions, but I'm reorienting my hobbies to be less SO-focused. I used to make a lot more PRs to OSS packages than I do now, for example, and I'd like to get back into that.
I see, as long as I can reach you when we win the cup, that's all that matters. We or at least I, support you and feel free to reach out if you need anything
/saystoself enough rocking back and forth in a corner... @DSM will be back now and again, won't he?! snap out of pir, wipe those tears away... you have a meeting to go to.
I have a strong opinion that my method is better than the one offered up by the competing answer. However, I have a difficult time arguing the point as I see that it may come off as pandering for the accepted answer. I get frustrated when other frequent contributors don't come forth to make their opinions known. I can't get too upset though as I'm guilty of the same thing. On occasion, I find it my obligation to scroll through q&a just to vote without stopping to answer myself.
Thank you for listening to me on my soapbox... stepping off now.
@Rick we are walking in circles. Your questions and confusion don't seem to make sense. Please try to formulate a coherent question, otherwise we're wasting each other's time
Note that we started from "does a return end a while loop" then code that doesn't return in a while loop, then "when does the except return false", now "this code passes tests but how does it work?". All this within 10 minutes.
Semi-interesting puzzle that came up while I tried to solve a recent question: given a list of integers and a number x, group the list into sub-lists where each sub-list adds up to a multiple of x. ex. [2, 5, 4, 3, 7] and 7 becomes [[2, 5], [4, 3], [7]]
Pretty straightforward to do by accumulating values in a while loop, but I wonder if there's an efficient one-liner solution
@AndrasDeak in itself, my first guess was to just take the image (1), compute its distance to the other image (2). Then take back image (1) and shift it by 1 pixel in all directions and compute the distance of those shited images with (2). The returned distance would be the minimum of all the computed distances.
Those two pass solve([1,6,1,1,5], 7) == [[1,6],[1,1,5]] (modulo type), but they don't pass solve([8, 6, 7], 7) == [[8,6],[7]]
def chunk(seq, x):
result = [[]]
for item in seq:
result[-1].append(item)
if sum(result[-1]) % x == 0:
result.append([])
if not result[-1]:
del result[-1]
return result
This is the long-winded implementation I had in mind
>>> def solve(lst, x):
... out = []
... [out.append([item]) if not out or sum(out[-1]) % x == 0 else out[-1].append(item) for item in lst]
... return out
...
>>> solve([8,6,7], 7)
[[8, 6], [7]]
>>> solve([1,6,1,1,5], 7)
[[1, 6], [1, 1, 5]]
almost oneliner...
if it's a single-shot function I could turn it into a lambda with out=[] default kwarg
You're right, I should write more idiomatic code. Here it is:
def solve(lst,x): return (lambda out: [out.append([item]) if not out or sum(out[-1]) % x == 0 else out[-1].append(item) for item in lst]+[out])([])[-1]
Maybe something like [[v[0] for v in vv] for _, vv in groupby(zip(seq, chain.from_iterable([[0], accumulate(x % n == 0 for x in accumulate(seq))])), key=lambda x: x[1])]? Corner cases not tested, but something like that should work.
Hmm, looking at it, I think that's just trying to be a more itertools-y version of Andras'.
@DSM yeah... I was thinking along the lines of accumulate and mod 7 and at those points increment a group id... something along the lines of:
from itertools import groupby, accumulate
x = [2, 5, 4, 3, 7]
def f(iterable, n):
group = 0
total = 0
for i in iterable:
if total and total % n == 0:
group += 1
yield i, group
total += i
res = [[el[0] for el in g] for k, g in groupby(f(x, 7), lambda L: L[1])]
But making f actually somehow in the lambda to groupby :)
So what I'm hearing is, for most collection types, for x in y: print(x in y) will never print False, and it would be weird if only dictionaries broke that trend. OK.
I expect built-in containers to behave nicely (except when they contain objects that don't equal themselves), but of course you can write your own classes that violate whatever design principles you choose
>>> class DumbList(list):
... def __contains__(self, x):
... return False
...
>>> y = DumbList([1,2,3])
>>> for x in y:
... print(x in y)
...
False
False
False
Hmm today I wrote an answer to a post asking how to find all the paths in a tree that satisfy a particular requirement, and it occurs to me that the OP is going to shoot themselves in the foot if they incorrectly assume it only finds paths that start at the root and end at a leaf, and yet I can't be bothered to edit my post a fourth time to add a warning
Yes because then they'd say "ok, so where's the code that only finds root-to-leaf paths? I'm waiting" and I'd have to redouble my efforts to get the single upvote that's been dangled before me like a carrot on a string all day
Wow... I realize that the use of "unartributively" is a mere typo but if you Google that word you get absolutely nada. Which is odd considering that "asdfsadf" will at least return a few hits.
If by 'all other languages" he means "the boring mainstream languages" then it's relatively answerable: no, in those languages you can't access a child class' member from a parent class' function.
Python ignores this rule because it's a loose cannon that doesn't play by the rules
Hey guys, I got a little off-topic question if you don't mind. Does anyone know the name of the paradox that goes like this?
You wouldn't call of bundle of two hay straws for a bundle, but you would if it eg consisted of 50 hay straws. So the question is, if you start with one hay straw and keep adding one at a time, when is it considered a bundle?
The sorites paradox (; sometimes known as the paradox of the heap) is a paradox that arises from vague predicates. A typical formulation involves a heap of sand, from which grains are individually removed. Under the assumption that removing a single grain does not turn a heap into a non-heap, the paradox is to consider what happens when the process is repeated enough times: is a single remaining grain still a heap? If not, when did it change from a heap to a non-heap?
== The original formulation and variations ==
=== Paradox of the heap ===
The word "sorites" derives from the Greek word for heap...
Except there's one paradox I can never find the article for. It's the one about the two generals in the army that wish to collaborate to attack their enemy at dawn, but neither wants to attack alone because they'll get destroyed. So general A sends a messenger to general B saying "we attack at dawn, please reply to confirm. If you do not reply I will assume this messenger was captured before he arrived, and I will not attack".
General B receives the message and sends a messenger back, saying "yes, we will attack at dawn. Please reply to confirm. If you do not reply I will assume this messenger was captured before he arrived, and will not attack."
General A receives the message and sends a messenger back, saying "I acknowledge that you have acknowledged my request. Now please acknowledge this acknowledgement. If you do not reply I will assume this messenger was captured before he arrived, and will not attack."
I just get a bunch of results about the civil war when I google two generals in the army that wish to collaborate to attack their enemy at dawn, but neither wants to attack alone because they'll get destroyed
Question: I have Jupyter Notebook installed from Anaconda. However, one of the libraries I'm trying to use is not available in Anaconda. What happens if I install Jupyter Notebook via pip3 now?
TCP/IP is the communication protocol that computers sometimes use to talk to one another. Maybe it's how the Internet works. This has been "half-remembered Wikipedia Articles" with your host Kevin
What about the paradox about the guy who googles about the paradox of googling paradoxes and in the process could not find the correct paradox because the paradox he was googling was his own recursive never-ending paradox... (eyes cross and drooling ensues)
@payne you can always fire up a native python virtualenv (venv) and install everything there. Then all the libraries are isolated from the main and other installations
My favorite paradox is the Abilene paradox because I can use it to justify not leaving my house. My least favorite paradox is Newcomb's paradox because it gives me a headache if I think about it for more than fifteen seconds.
If I have 3 numpy arrays, j, k, and i. and I run shape on them I get back 1045 for each one. How can I create a dataframe from them where j,k, i are columns?
Little math exercise for today: my wife's tutoring student (4th grade) was given this assignment: How many combinations of quarters, dimes, nickels, and pennies can be made that add up to 30 cents?