Use CommandBox's rate limiter on only certain requests

Communities CommandBox CLI
1

This came up in conversation internally at Ortus today and I wanted to write it down somewhere public. There has already been a setting in CommandBox for some time called web.maxRequests in the server.json to limit the total number of concurrently-running requests for the entire server. Once this pool is full, any additional incoming requests will be queued until there are free threads available.

The problem scenario is what if you allow 25 concurrent requests (max threads) and someone whacks the refresh button 30 times on a report that takes 5 minutes to run? Well, all your available threads will be busy for the next 5 minutes and your site won’t respond to other requests in the mean time, even if they are quick ones.

The same potential scenario goes for CommandBox 6’s new multi-site mode. The worker thread pool in Undertow is global and applies to all sites. That means if you have 2 sites and allow 100 concurrent requests and site 1 is using all of them, there will be no threads left to process any requests for site 2.

The solution to both of these scenarios is a server rule that is built into Undertow called request-limit(). You can specify a request limit for an entire site or even for a specific type of request.

Here’s what scenario 1 would look like to prevent more than 5 reports from running at a time by pairing it with a path prefix predicate.

{
    "web":{
        "maxRequests":"30",
        "rules":[
           "path-prefix-nocase( '/reports/' ) -> request-limit( 5 )"
         ]
    }

}

Or, you could limit how may CF requests can run at a time, ensuring there are always 5 threads available to serve static assets like images or javascript.

{
    "web":{
        "maxRequests":"30",
        "rules":[
           "regex-nocase( '.*\.cfm$' ) -> request-limit( 25 )"
         ]
    }

}

And finally here is a simplified multi-site example which allows 100 total requests, which each of 2 sites, having a limit of 50 apiece.

{
    "web":{
        "maxRequests":"100"
    },
    "sites":{
        "site1":{
            "rules":[
                "request-limit( 50 )"
            ]
        },
        "site2":{
            "rules":[
                "request-limit( 50 )"
            ]
        }
    }
}

If you’d like to learn more about multi-site including some more complete examples, check out this repo of demos I’ve recently created:

3 Likes