handling http pipelined requests

I'm working on a project developed in an SOA, loosely coupled fashion.
My ColdBox app is handling incoming a bunch of inbound requests,
performing atomic actions, then returning a simple JSON success/fail
object. Right now the front end app is on the same machine as my CB
app but it is designed in such a way that that can easily change and
will be easy to separate and build out.

The performance issue that I know is going to become more of an issue
is that traffic to this app is very bursted. An action will happen out
there in the world and then my app will get hit with 20,000 http
requests, each with a JSON payload of 15 or 20K, then a long period of
quiet.

One thing I was considering as a way of making the application more
performant is to try and implement http pipelining. Since each request
is fairly small and the actions are atomic (the order they are
processed is irrelevant), eliminating the http open/close overhead
could potentially increase application performance significantly. I
know, theoretically, that pipeline support would have to be enabled on
the client (sending) side, in the web server and, I believe, on the
request handling side because you don't have the same page request
life cycle with the http request/response.

I've never heard about implementing http pipelining on the CF side. Do
any changes need to be made at all? What are the gotchas? Are there
any resources out there to help me investigate?

Thanks,
Judah

Just food for thought, but since the order of your individual executions is irrelevant, have you considered leveraging CF’s asynchronous gateways in this scenario (keyword: sendGatewayMessage)? I’ve used this approach for performing multiple (thousands) of individual db inserts simultaneously and the results were outstanding regarding app response and total time to execute (2000% increase when performing such tasks asynchronously)

Doug B

I thought, briefly, of using event gateways. A couple problems there
though, because the (for lack of a better word) client application
that is sending me data is a .Net app and we really have decided on a
RESTful type interface using HTTP. I haven't written any event
gateways, but it is my understanding that they really aren't meant for
handling http requests. I could be wrong though of course. I'd
consider switching to another protocol perhaps but I'd have to justify
the change on both the CF and the .Net side.

My other problem is that the app is currently on CF 8.0.1 Standard and
I am in the process of switching to Railo. Neither currently support
event gateways, so I'm screwed there anyway :slight_smile:

But I love the suggestion Doug. Perhaps I am simply uneducated about
event gateways though and maybe it is something I should look into
further. How did your setup pass the incoming requests to your CF
asynchronous gateway?

Judah

Hi Judah. Here’s the snippet of code that makes the call to the gateway:

<!— dsn that will be used in this template and by the gateway —>
<!— name of the gateway —>

variables.stData.theData=variables.thisdata[j]; sendGatewayMessage(variables.gateway,variables.stData);

Here’s the actual gateway cfc:

insert into TestTable (txt_groupid,txt_code,txt_description,inserttime,testtype) VALUES ( , , , , ) #cfcatch.error#
#cfcatch.message#
#cfcatch.detail#

I may not have a clear enough picture of your scenario, but in my mind I don’t see why you would have to alter anything about the way your .NET client is interacting with your backend app. Your consumer is passing you a JSON string; your app receives the string, could then simply package it up into an appropriate structure for your gateway cfc, and then execute the sendGatewayMessage method. Right? Your consumer wouldn’t have to know anything about how you’re handling the submitted data nor what you’re doing with it once it’s been received.

As far as event gateways being available in 8 standard, I believe they are. As of version 7 you needed the enterprise version, but they started making it available in Standard starting with 8.

Ah, I see what you are trying to say. The question is basically one of
multithreading and where it happens. You could have, I believe,
accomplished the same thing in your app by using cfthread to spawn a
new thread to perform the insert.

My issue is that each incoming http request is already one action.
There isn't anything to loop over, so asynchronous nature of the
gateway (or cfthread) doesn't actually help anything.

The specific scenario in my case is actually parsing a big file of
account statements. The parsing of the file into individual statements
is handled in the .Net app. And that part is already multithreaded. So
the call to my app consists not of a bunch of statements together to
loop over, but rather a bunch of calls each with one statement that I
have to persist and do a file manipulation on, etc.

The bottleneck in my case seems to be different than yours. Your
bottleneck is how to get the data out of the loop and into the
persistence quickly. My bottleneck is how to handle the incoming http
request faster.

Thanks though. And if I am still not understanding correctly (wouldn't
be the first time) then please tell me so :slight_smile:

Judah

and i see better what you’re saying now, too. Your assessment is right: my proposal won’t help you. Sounds like what you need are additional clustered web servers that can be spawned on an as needed basis, then destroyed as the load lightens again. Enter cloud computing. :slight_smile: I’m told that Amazon’s EC2 service is an infrastructure cloud that allows apps to exist in a world where horizontal scaling can expand and contract as needed to accommodate just such a scenario as yours. I know it doesn’t immediately address the issue, but maybe it wouldn’t be all that difficult to migrate toward.

Sorry I couldn’t help more.

Doug Boude :0)

I appreciate the help Doug, really. Amazon EC2 is on my radar and I'll
be doing some tests there but that isn't something that really fits
within the corporate environment at my job quite yet. As the sole CF
developer in an otherwise all .Net shop, there are some practical
limitations on what I can introduce :slight_smile:

Judah