[coldbox-3.8.1] Load-balanced application CB app, SES and Base URL

Communities ColdBox HMVC
#1

Hi guys, I’m in a bit of a pickle as I learned our production Adobe ColdFusion Servers are load-balanced kind of last minute during deployment. We’re using the SES Interceptor in our DB app and when I reinit the CB application I need I need to type the specific URL of each server in the browser URL not have the load-balancer send me where it wants. However, doing this means the application gets init’ed with the specific server URL and not the “load-balanced URL”

So I need each the app instance to respond to:

  • http://intranet.local.domain/path/to/my/app

Even though in reality they are all living at these addresses:

  • http://server1.local.domain/path/to/my/app
  • http://server2.local.domain/path/to/my/app
  • http://server3.local.domain/path/to/my/app

Eeep!

Wes

#2

Yep, this is a common issue. For an immediate stop-gap, create host entries on each server to point to itself and reinit on each server via remote desktop.

Now, first ensure that all the appropriate production URLs are in your environment configuration so you get production settings when you reinit.

Now, I’m a little unclear on how your network is set up and if you have a way to target a specific server yet. The way I have done this is by creating multiple policies on the load balancer. One general policy that balances between each IP and another for each server that always hits just the IP for one server. Then map your DNS entries to point to the appropriate LB IP. If you don’t want the public doing this, don’t NAT the per-site IPs externally and hit them over the VPN using local DNS.

Some of this depends on how your LB is set up. If your servers are not on their own subnet (“behind” the LB), then you can skip the extra policies and just route DNS to the correct IPs. Also, if you’re using host headers to bind your web server, make sure all the possible host names are configured. If you’re binding just to IP, you should be fine.

DNS resolution:
www.mysite.com → points to public load balanced IP
www1.mysite.com → points to the IP for web 1
www2.mysite.com → points to the IP for web 2
www3.mysite.com → points to the IP for web 3, etc…

Now if you do want people to be able to legitimately hit www1, www2, etc and use it then install this or something similar:
http://www.coldbox.org/forgebox/view/Multi-Domain-SES

It overwrites the SES base URL in the request context on every request so any links built for that person reflect their current URL (whatever that may be)

However, if you want to always and forever force your public to one “sanctioned” domain, then hard code that one in your routes.cfm and all links will always use it.

One final note, if you use the session scope to save state, check and see what the load balancing algorithm is. Round robin will move people between servers and they’ll use their sessions, so sticky sessions (server affinity) may need to be enabled unless you’ve written your application to handle it.

Thanks!

~Brad

ColdBox Platform Evangelist
Ortus Solutions, Corp

E-mail: brad@coldbox.org
ColdBox Platform: http://www.coldbox.org
Blog: http://www.codersrevolution.com

#3

Apologies if this comes through twice, it didn’t show up the first time:

Yep, this is a common issue. For an immediate stop-gap, create host entries on each server to point to itself and reinit on each server via remote desktop.

Now, first ensure that all the appropriate production URLs are in your environment configuration so you get production settings when you reinit.

Now, I’m a little unclear on how your network is set up and if you have a way to target a specific server yet. The way I have done this is by creating multiple policies on the load balancer. One general policy that balances between each IP and another for each server that always hits just the IP for one server. Then map your DNS entries to point to the appropriate LB IP. If you don’t want the public doing this, don’t NAT the per-site IPs externally and hit them over the VPN using local DNS.

Some of this depends on how your LB is set up. If your servers are not on their own subnet (“behind” the LB), then you can skip the extra policies and just route DNS to the correct IPs. Also, if you’re using host headers to bind your web server, make sure all the possible host names are configured. If you’re binding just to IP, you should be fine.

DNS resolution:
www.mysite.com → points to public load balanced IP
www1.mysite.com → points to the IP for web 1
www2.mysite.com → points to the IP for web 2
www3.mysite.com → points to the IP for web 3, etc…

Now if you do want people to be able to legitimately hit www1, www2, etc and use it then install this or something similar:
http://www.coldbox.org/forgebox/view/Multi-Domain-SES

It overwrites the SES base URL in the request context on every request so any links built for that person reflect their current URL (whatever that may be)

However, if you want to always and forever force your public to one “sanctioned” domain, then hard code that one in your routes.cfm and all links will always use it.

One final note, if you use the session scope to save state, check and see what the load balancing algorithm is. Round robin will move people between servers and they’ll use their sessions, so sticky sessions (server affinity) may need to be enabled unless you’ve written your application to handle it.

Thanks!

~Brad

ColdBox Platform Evangelist
Ortus Solutions, Corp

E-mail: brad@coldbox.org
ColdBox Platform: http://www.coldbox.org
Blog: http://www.codersrevolution.com

#4

Brad, you are the man – thank you for some crucial, clutch knowledge here. I’m working with our network and server teams to get some of these answers now.

To clarify, in my specific case we won’t have users accessing the app via the individual DNS host names of the servers, but rather everyone will access the app through the common hostname that hits the load balancer and then the rest is invisible to the user.

I was hitting the individual server hostnames in order to make sure the CB app on each server had re-initialized properly, but ultimately the app on every server should respond on just one hostname (though sometimes http and https).

Is there any best practices to be aware of if I hard code the hostname in the Routes.cfm file? Right now I’ve got this:

// Base URL if (len(getSetting('AppMapping')) <= 1) setBaseURL("http://#cgi.HTTP_HOST#/"); else setBaseURL("http://#cgi.HTTP_HOST#/#getSetting('AppMapping')#/");

But I should change this to something like the following it sounds like:

`
// Base URL
var prodURL = “TheLoadBalancedHostname”;
if (len(getSetting(‘AppMapping’)) <= 1) setBaseURL(“http://#prodURL#/”);
else setBaseURL(“http://#prodURL#/#getSetting(‘AppMapping’)#/”);

`

Also, on a related note I was planning on using Query Caching and Secondary Level Caching but I should probably abandon that until I have a shared caching option like CouchBase, right?

Big, BIG thanks!

Wes

#5

Yes that change to routes.cfm should make all links that are built use the URL of your choosing. Since you need to target individual servers for reinit, (and probably testing/troubleshooting) I would still recommend using the multi-domain SES interceptor which resets this on every request. That way you can actually navigate around and test any server on the special url.

Concerning caching, there’s no reason to abandon that just because you have multiple servers. Just keep in mind that in-process caches mean that the same data might be cached multiple times so each server has to “warm up” its own cache. Couchbase would be an excellent option for the future since all web servers can share the same cache, plus it is non-volatile meaning it persists even when you reboot a server.

If you were on Railo, you could actually store your session scope in Couchbase for a truly scalable server cluster using round-robin balancing (using the commercial Ortus extension). You could get close on Adobe ColdFusion as long as you used the SessionStorage plugin for all you rsession access since you’d have one encapsulated API for session storage.

Thanks!

~Brad

ColdBox Platform Evangelist
Ortus Solutions, Corp

E-mail: brad@coldbox.org
ColdBox Platform: http://www.coldbox.org
Blog: http://www.codersrevolution.com

#6

As a follow-up I went the route that Brad suggested and used a modified version of the multi-domain SES interceptor. It makes sure that the SESBaseURL has the currently accessed URL hostname in it, for the situation where a server can be known by different names because of load balancing. Also, this makes sure SESBaseURL has HTTP or HTTPS based on how the site is currently being accessed. in Coldbox.cfc my interceptors fire in the following order – and though I’m not sure it’s “correct” it does seem to work:

  1. Autowire
  2. SES
  3. SSL
  4. MultiDomainSES
  5. Security

MultiDomainSES.cfc

`
component name=“baseURL” output=“false” extends=“coldbox.system.interceptor” hint=“Fixes up the SESBaseURL after the SES and SSL interceptors run” {

void function configure(){
}

void function preProcess(event, interceptData){
// Change the domain to the domain currently being accessed via URL
var curSESBaseURL = ReReplaceNoCase(event.getSESBaseURL(), “//(.+?)/”, “//” & cgi.http_host & “/”);

// adjust the SESBaseURL based on whether SSL or not
if (event.isSSL()) event.setSESBaseURL(ReplaceNoCase(curSESBaseURL, ‘http:’, ‘https:’));
else event.setSESBaseURL(ReplaceNoCase(curSESBaseURL, ‘https:’, ‘http:’));
}

}
`

Brad, thank you for your incredible help on this and feel free to steer me in a different direction if it still looks weird.

Wes

#7

That looks like you’re on the right track. Just a couple notes: I don’t think the autowire interceptor is needed any longer. I’m pretty sure it get phased out shortly after 3.0.
Also, if the SSL interceptor does any redirect (because you’re on HTTP, but should be on HTTPS) you might want the mutli-domain SES to run prior so you have the correct URL to redirect to.

Thanks!

~Brad

ColdBox Platform Evangelist
Ortus Solutions, Corp

E-mail: brad@coldbox.org
ColdBox Platform: http://www.coldbox.org
Blog: http://www.codersrevolution.com

#8

Ah, very helpful – thank you! I had initially spawned this application long ago from the Simple Blog 5 sample application which still has Autowire in the config. I should probably compare my bloated config to the new base template that gets created from CommandBox with coldbox create app

I placed the MultiDomainSES interceptor before SSL as you recommended, thank you.

New order:

  1. SES
  2. MultiDomainSES
  3. SSL
  4. Security

Wes