last day (14 days later) » 

4:16 PM
0
A: The need of semaphore in python threading module

werewindleBecause you can't be sure, when scheduler will switch threads. Consider SMP machine. It will always process one thread at a time. But nevertheless multithreaded code should use semaphores, mutexes, critical sections and other synchronization primitives. Although CPython have GIL and can't execut...

 
Python cannot execute multithread, but have the Semaphore Class.
A CPU execute threads by switching. In your example, the threads are assigned to each CPUs. I think that there should not be need of Semaphore for thread and that there is only need of Semaphore for processes
Here link, there are that Semaphore is for sending signals.
I think only mutex is enough for SMP because one thread is executed in a CPU.
 
@q2ven Every thread in a Python program is a real OS thread, and the OS has complete control over when each thread executes. The GIL prevents threads from executing Python bytecode in parallel across CPU cores, but the OS can still switch between threads whenever it wants, and different threads can take the GIL at any time. If you need a critical section protected, you still need synchronization primitives (like semaphores) in order to control which thread(s) can enter the given section. Otherwise you could still end up with two threads inside the critical section concurrently.
 
@q2ven Python (at least CPython) will create OS thread for every python thread. And task scheduler will switch them as he wants. Yes, all those threads can't run at the same time. But you can't guarantee which thread will be run right now, if you does not use synchronization primitives. Semaphore is only one of such primitives.
 
Is it really that different threads can take the GIL at any time? more than two threads could be executed simultaneously?
Why does programmer have to guarantee which thread will be run now?
 
@q2ven Only one thread can hold the GIL at a time, but you don't have much control over which thread holds the GIL, and for how long. The Python interpreter has an internal mechanism that makes the GIL holder release the GIL so that other threads can take it. As a Python programmer you have very limited control over how that mechanism works - see sys.setcheckinterval
 
4:16 PM
@q2ven GIL can be taken by another thread every say 100 instructions. So from your POV threads will be executed simultaneously. Yes, technically (on CPU) they will work one after another. But within thread it will look like they are running simultaneously.
 
I have read this and I knew what you said. You two still have not answered to my question with persuasive reasons
 
@q2ven what about my example with 5 printers (okay, 5 DMA controllers)?
 
If three threads tell a printer to print something simultaneously, the printer execute the operations in order. So I seem that there is no need of Semaphore.
 
@q2ven Okay, you have connection pool to database. This pool consists of 5 connections. When one thread want to execute DB query, it takes connection from the pool. Semaphore will ensure, that no more than 5 connections will be taken from the pool.
 
But python does not execute five threads at the same time, doesn't it? I think it is only useful when multiprocessing.
 
4:16 PM
@q2ven. Consider this: one thread uses connection to send request and the waits for an answer. It releases GIL and another thread begins to work. Second thread does not know that connection is used (there are no Semaphore) and uses the same connection to send another request. This ruins internal state of connection object and your program crashes.
 
If the first thread takes the connection, should it be removed from the pool and the second cannot use the connection?
If the second happened to use the connection, there should be a problem even with Semaphore. As long as there are no way of checking whether the connection is used or not.
And with the mechanism, there are no use of Semaphore.
 
@q2ven. Okay, where if we have 5 connections, and all 5 was used by different threads, where should wait sixth thread that wants to utilize connection? Just think about this.
 
You mean that Semaphore is for sending signals? threading module also have Condition Class, it is enough isn't it? Threads do not have to check if the pool of five connections is empty. Once they tried and error has occurred, then, they wait for the connections to be released. Semaphore is only for the preventing the error in advance?
 

last day (14 days later) »