Jump to content
Search In
  • More options...
Find results that contain...
Find results in...

Welcome to our site

Take a moment to join our board

Sign in to follow this  
Spirited

System.IO.Pipelines Review & Discussion

Recommended Posts

First Glance

At first glace, System.IO.Pipelines appears to be an interesting solution for reducing the complexity of server sockets and efficient memory management. Without going into any details, the blog post suggests that Pipelines could provide an application with higher performance and a better / less complex server architecture. In this topic, I explore that idea using a Conquer Online server as a demo.

Architecture

What isn't mentioned in the blog post is the general architectures supported by Pipelines. The only workflow promoted by Pipelines via the blog post is an asynchronous one, where memory is written to asynchronously and read from asynchronously using two separate tasks. This scales nicely for a single data source, such as a single TCP socket accepting RPC; however, multiple tasks processing multiple data sources from multiple clients does not scale. A queue needs to be inserted for managing requests appropriately; therefore, two tasks per client connection just for receiving data in sequence doesn't seem appropriate. Rather, two tasks working asynchronously using a pipeline as a channel is overkill. That's the job of worker roles that dequeue work.

A second workflow is promoted through the API for synchronous reads and writes. An asynchronous socket system could synchronously manage memory for a single asynchronous callback operation. This seems like the best approach for managing data request from multiple data sources. That would be one task per client managing receives. 

Performance

For this test, Pipelines was implemented for synchronous memory management, which would be the most likely use case for a high performing game server that manages data using a worker role and queue. When analyzing performance, I wanted to see if it were any better than just keeping a fixed length buffer, such as in the case for SocketAsyncEventArgs. Since we're using Pipelines for synchronous memory management, the framework becomes simply weighed down by thread safety. Therefore, the obvious answer is that it shouldn't perform better than a fixed length array, and it doesn't. Pipelines takes ~41000 ticks while a fixed length buffer takes ~1800 ticks (tick count includes ReceiveAsync, which accepts and uses the buffer from both tests). 

Offerings

So, besides performance, what else might Pipelines offer programs that wish to use the framework for synchronous memory management? The first answer that comes to mind is inherent in the question: memory management. Pipeline memory blocks grow as needed using an initial size hint. Received data is copied from a 2048 byte buffer to the dynamic memory buffer, and the application only processes bytes that it received. A maximum buffer size can also be specified by overriding the Pipe's MemoryPool option. All of this can be accomplished using fixed length Array Segments, and splitting on the received byte count. If you don't expect to be processing lots of packets per client, such as the case for Conquer Online's packets which are always smaller than 1024 bytes, then there's not a lot that Pipelines has to offer.

Conclusion

Although I feel that the Pipelines framework isn't helpful for the specific design of a game server, it does greatly simplify general worker role implementations for WCF. Rather than dealing with the complexity of Azure Queues, Microsoft Message Queuing, or implementing your own queue system using blob storage or memory for producer/consumer tasks, Pipelines offers a very clean and elegant solution for managing work from a single data source. Pipelines may also be helpful for multiple sources and a single, generic consumer, such as in the case of reading and processing map files for Conquer Online. 

Stepping back, it seems like Microsoft is learning from other languages such as Go, which offers the same functionality as a language primitive (channels) while also attempting to wrap and simplify its functionality. Hopefully we see more efforts like this in the future as these languages continue to compete against C#.

With all of that said, what are your thoughts on my findings? Do you feel that my research was a good field test for Pipelines that went wrong, or did I take a wrong turn somewhere in the implementation? How have you worked with Pipelines so far, and has it helped simplify complex server code? I'd be curious to know. Thanks for reading.

Edit: Corrections

Quote

So, I spoke to Smaehtin about it, and I think my analysis of Pipelines needs to be adjusted. In terms of using SocketTaskExtensions, Smaehtin said it uses IOCP in the same way something like SocketAsyncEventArgs would, which uses a thread pool for operations similarly. The callbacks themselves would be different in the way they wrap around operations, but there's no difference in how that thread pool is being used. So perhaps using Pipelines with SocketTaskExtensions would yield similar performance metrics. It would be interesting to find out. 😄

 

  • Like 2

Share this post


Link to post
Share on other sites

With the introduction Memory<T> and Span<T> how would you say those play into the architecture one might implement. Do you think an agent/actor model with a message based queue help with some of the multi client problem? 

Share this post


Link to post
Share on other sites
1 minute ago, Smallxmac said:

With the introduction Memory<T> and Span<T> how would you say those play into the architecture one might implement. Do you think an agent/actor model with a message based queue help with some of the multi client problem? 

I see Memory<T> and Span<T> as language improvements that were becoming necessary for C# to remain competitive with other languages. Python, Go, etc. all use sliceable arrays that can be implicitly or explicitly casted into multiple other types. To me, it feels useful everywhere, not just one architecture. For a Conquer Online server I'm currently working on, I use Memory for the buffer itself, and Span for slicing into the buffer for packet splitting.

Regarding your second question, I'm really not sure. The agent/actor model is vague. The real power behind a good implementation of the agent/actor model is the communication between actors, but Conquer wouldn't benefit from that. Actors only communicate with server collections. If we're just talking about single actors working on a message queue, then yes. It depends on how the message queue is implemented and fed messages, though. The Conquer Online client requires that messages be processed sequentially. If you process two packets from the same client at the same time, or less likely one packet before another, you could run into race conditions or undesirable behaviors. I'm currently investigating into a good approach for handling that scenario, and I may implement messages as tasks that encapsulate a list of packets processed from a single receive. That way, the receive task can wait on that processing task to complete before accepting more data.

Share this post


Link to post
Share on other sites

In terms of Conquer Online, which was not my general idea, you would have to have a Commander agent on top or right under the server socket system which would communicate with client agents(client sockets) for cross client communication/interactions. It would be a rather confusing design at first. I would have to spend some more time thinking about how it would be composed. Though if done correctly client agents would be just recursive loops that accept a sort of state. At that point you would not have to worry about concurrency. At least that is in F#'s implementation. 

I'm not sure if you have seen this before but I saw this a while back and thought it was pretty neat.

 https://github.com/mgravell/Pipelines.Sockets.Unofficial/blob/master/src/Pipelines.Sockets.Unofficial/StreamConnection.cs

Though if you are going down the stream route just wrapping you stream in NetworkStream would be suffice. 

Share this post


Link to post
Share on other sites

That example you posted is much better than the one Microsoft posted. I didn't think about treating pipelines's stream architecture like a proper wrapper around a socket stream. It's complex, which is the complete opposite to what Microsoft was advertising, because writing that wrapper is still our job and not theirs, but it's an option. I'm not happy solving a complexity problem with much more complexity, but I might give it a try using your NetworkStream suggestion. 

Share this post


Link to post
Share on other sites

So, in general, pipes are in-memory buffers that are shared between threads or separate processes. They can be directional or bi-directional. I/O operations on the pipe are usually one-to-one, in that writes on the pipe are matched with reads from the other end one-to-one, or vise versa. When taking a second look at Microsoft's Pipelines project, I get why they opted for asynchronous access, since that matches with their async-await architecture. It makes sense for worker role / cloud applications, which I don't think I hit on enough. Azure is their primary money maker these days. It makes me wonder if they're developing this for Azure Functions in a way that can act as an awaitable pipe reader for their IoT edge cloud services. 

In terms of writing my own, I wrote a pipe reader around the receive buffer for TCP connections. I think it's a good addition, as it allows me to say how many bytes I've consumed in packet splitting, and then whatever's remaining is appended to the next receive just by the nature of the pipe reader. Makes it more intuitive, I think, or at the least separates logic into invidicual components that can be reused / overridden. I might post something later once I have more to show.

Share this post


Link to post
Share on other sites

So, I spoke to Smaehtin about it, and I think my analysis of Pipelines needs to be adjusted. In terms of using SocketTaskExtensions, Smaehtin said it uses IOCP in the same way something like SocketAsyncEventArgs would, which uses a thread pool for operations similarly. The callbacks themselves would be different in the way they wrap around operations, but there's no difference in how that thread pool is being used. So perhaps using Pipelines with SocketTaskExtensions would yield similar performance metrics. It would be interesting to find out. 😄

Share this post


Link to post
Share on other sites

I still have mixed feeling in the way Microsoft introduced this new feature for the framework. Microsoft tends to have this idea of magic that just seems to work. It would make sense that they would not make a abstraction on top of SAEA act much different in terms of threading and task pools. 

Share this post


Link to post
Share on other sites
3 hours ago, Smallxmac said:

I still have mixed feeling in the way Microsoft introduced this new feature for the framework. Microsoft tends to have this idea of magic that just seems to work. It would make sense that they would not make a abstraction on top of SAEA act much different in terms of threading and task pools. 

Yeah. I definitely feel that way at times with their various async options for sockets. It's hard to get a gauge on which is best for each application without knowing exactly how they implemented it. If you implement Pipelines with SocketTaskExtensions, then you're basically using APM socket methods like BeginReceive and EndReceive. It's just a wrapper around those methods that convert IAsyncResults to awaitable tasks. So, SocketTaskExtensions doesn't seem to provide any optimization other than the ability to architect your socket system around the async-await design. You can also use Pipelines with SAEA. SAEA really just gives you a away to preallocate buffers. As far as I'm aware, preallocation and how it handles buffers are the only benefits.

So what happens if you throw Pipelines into that? That doubles the task footprint of the entire socket system if you implement Pipelines asynchronously. I see the benefit of doing something complex with files, such as reading from one file as read from the pipeline, and writing to another file as output from processing and writing to the pipeline. I don't see a benefit from implementing pipelines for sockets. You would want something more synchronous that doesn't impact the task pool, such as having a simple in-memory pipe reader around the preallocated SAEA buffer. It would be interesting to do some tests around it, but I don't have time for that. I don't think I'll be using Pipelines for the time being.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×

Important Information

By using this site, you agree to our Terms of Use.