last day (19 days later) » 

03:11
1
Q: boost::asio with named pipes; parent process can't read from pipe via async_read_some

capsProblem Statement I have a windows server/parent process that creates two named pipes, one inbound and one outbound. Then it spins up a client/child process and gives those pipe names to that child process, which connects to them. The child process is able to successfully read and write to the pi...

Why are you combining a dozen different approaches? Speicifically, why use async and syncronous streams at the same time? Why use Boost's async_pipe as well as your own named pipes? Is there anything not done by Boost Process's pipes? Or getting from the other angle: what are you trying to achieve, and does Boost add anything?
Hey sehe, thanks for stopping by. All the code in this question is using boost::asio::windows::stream_handle and calling the async_*_some APIs. I understand the confusion, since I included the async_pipe.hpp header (will edit that out) but you will note that I am not actually using them in the code (I was using them in place of stream_handle in my first try, but they didn't work at all). What I am trying to achieve is use named pipes asynchronously on Windows (and eventually Unix) without writing the low-level code myself. boost::asio is the first thing I am trying.
I suspect that async_read_some is swallowing errors. When I try to peek at the pipe from another thread while waiting for the read to time out, I get "pipe closed" errors. But async_read_some claims that it will surface any errors it gets from the APIs. Why isn't it telling me that the pipe is closed? Under the hypothesis that maybe the synchronous API would give me the errors, I wrote the code in my "answer" and then it turned out that code worked, which didn't get me any closer to finding out why the async API isn't working.
I think async_pipe should just work (it uses stream_handle internally anyways, and saves you all the trouble of setting up the pipes). If it doesn't, that's annoying - it could be something straightforward. I don't have a winodws environment so I don't think I'll be able to look at it in reasonable time frame
Also, async_read_some is not swallowing errors. If it does, that's a bug (or the handle assigned to stream_handle doesn't meet some documented criterium?)
@sehe async_pipe ... saves you all the trouble of setting up the pipes This was part of the problem, I think. It sets the pipes up to be used a certain way, which was not the way I wanted. I looked at the code, and it does not match any of the examples of how to set up named pipes in Windows that I could find. There is also no way in the async_pipe API to distinguish between read-only access, write-only access, or duplex acces; not to mention server vs. client. I think it was designed to be used in a very specific way, and that way is not the way I wanted.
"It sets the pipes up to be used a certain way, which was not the way I wanted" - perhaps you should elaborate on exactly what way you want to use them. I imagine Boost Project does want to use them for stdin/stdout/stderr redirection, mainly.
03:11
As my example code implies, I want to use named pipes for IPC. Typical IPC with named pipes (on Windows, at least) involves a "server" and a "client" which use different APIs; the "server" creates the named pipe and then waits for a connection from a client. The "client" connects to an already extant named pipe. This is the kind of behavior I'm looking for (a parent process spawning a child process) and is essentially what my code in this question does using stream_handles. And it mostly works! Except for the read on the server side.
Also, async_read_some is not swallowing errors. Perhaps not, but when I try to Peek at the contents of the pipe while waiting for my async read operation to complete, I get a "pipe closed" error and my async read operation times out before completing or giving me any error.
@caps Can you make it work / simplify the SSCCE to not use Boost Process? Processes (not) inheriting descriptors could be wreaking havoc?
@caps I'm afraid you already said this. Like I said, if that observation is reliable then it could be either a bug or a violation of preconditions for stream_handle. There might be different modes for IOCP - not an expert here.
@sehe I'm not sure descriptors are at play; this is Windows.
It's very similar on UNIX - though they call them UNIX domain sockets. There's also FIFOs. I think from your description named pipes could actually be more like FIFOs?
Also, because fork/exec child processes is pretty different, I think the dominant way for parent/child processes to "talk" is just stdin/stdout redirection to pipes.
38
Q: Are FIFO, pipe & Unix domain socket the same thing in Linux kernel?

JustinI heard that FIFOs are named pipes. And they have exactly the same semantics. On the other hand, I think Unix domain socket is quite similar to pipe (although I've never made use of it). So I wonder if they all refer to the same implementation in Linux kernel. Any idea?

For reference, just so we can check assumptions/understanding
@sehe I will look into the preconditions for stream_handle when I get a chance.
@sehe Ah, I didn't realize that the Unix name for domain sockets was "named pipes." Yes, I am talking about the FIFO equivalent on Windows.
@caps just m.m. of course. I know Win32 calls everything handles, but I don't think that really adds new insight :)
03:19
Named Pipes have been around in Windows a long time; since way before Windows brought Unix Domain Sockets over to their OS, which I think was 2017.
@caps Well, I think FIFO might be more like Named Pipes, but you could check by reading that answer
@caps I'm well aware. I've used them. It's just been very very long
@caps Unix Services for Windows have existed since way before that (I think I remember 2008 at least)
@caps Ifff they're documented. Annoyingly, the "tests" for stream_handle exclusively test compilation interface, not any runtime behaviour. o.O
@sehe Naturally. That's what tests are for, right :troll:
Well, it's also pretty important, but not what we needed
@caps en.wikipedia.org/wiki/Windows_Services_for_UNIX I can't find evidence that the POSIX layer included Unix Domain sockets, but indeed it's 22 years old. Anyhoops, I think by stating AF_UNIX, they more or less imply that feature was added to WinSock32 wholesale, not as part of a POSIX layer
Yup. Winsock API indeed.
@sehe Right, it's more "native" now. Unix Domain Sockets are my next point of investigation, but I'm more wary of them for backwards compatibility purposes. Unfortunately I think some of my ultimate end users are still on older versions of Windows.
So to level set, what I want is:
* FIFO on linux
* Named Pipes on Windows
Wrapped up in a cross-platform way, preferably using native async APIs (e.g. overlapped I/O APIs on Windows)
I thought boost::process::async_pipe was what I wanted, but it didn't work at all the way I wanted on Windows.
Some kind of conditional compilation + typedef magic, where I use async_pipe on linux and stream_handle on Windows seems like my next-likeliest option, but the stream_handle is giving me the issue I described in my question.
03:49
@caps Is there a problem with actually using them in the "unix sensible" way of redirecting stdout/stdin to them? I mean, it should make the whole ordeal a no-brainer because that is built into Boost Process, which intends to abstract away your platform differences.
@caps Wait what. Does it do what you want on linux?!
Because if it does that, and not on Windows, then you should take it up with the Boost Process library devs?
@sehe I spent what coding time I had since posting the question adjusting my code with the conditional compilation stuff I mentioned above. It is compiling now but giving me runtime errors I haven't been able to diagnose yet. So I don't know for sure yet, but I expect it will work based on what I know of FIFO (and async_pipe wraps FIFO, does it not?)
@caps Nah. It just wraps pipes.
@sehe Doing IPC through stdout/stdin just seems so weird to me. And prone to issues; any code anywhere in the program can write to stdout or read from stdin at any time (although the latter is admittedly unlikely).
So, you didn't try it on linux yet? You remark made it seems like it worked on linux, but not on windows.
@caps You know, that might not be the best argument to opt out of the most conventional method for child/parent IPC on linux
@caps The good news is you can easily dup the pipes on different filedescriptors on linux (probably not on windows?)
@sehe No, I tried it on windows first.
03:56
Ah. Capiche
@sehe I like doing things in "standard" ways but at the same time some conventions are bad.
Perhaps you're right, though.
(I'm heading to bed. Just completed this ordeal - I think asio docs are relatively okay compared to that)
@sehe Now that you mention it, someone else said something similar. I don't know if Windows supports similar behavior. I will look into that.
For the entry point into that puzzle piece, see eg
1
A: Similar functionality to dup2 in boost

seheYou can use Boost Process: https://www.boost.org/doc/libs/1_74_0/doc/html/boost/process/posix/fd.html This property lets you modify file-descriptors other than the standard ones (0,1,2). It provides the functions bind, which implements dup2 and close. So, example: #include <boost/process.hpp> ...

@sehe Thank you!
@sehe This is where the terminology differences hurt. I see "pipes" and I think "the equivalent of both named and anonymous pipes on windows"
er, I should say "support for both named and anonymous pipes on windows + the equivalent on linux"
04:10
Yeah. Worst thing is that "pipes" are also the popular name for the shell construct with |, which on MSDOS has been implemented with temporary files, for the loooongest time. So,

  last day (19 days later) »