URL Rewrites / Redircts

Probably a basic question for someone, but since I’ve done away with IIS and developing with Commandbox locally - My IIS redirects/rewrites aren’t going to work.

I’ve read through the tips on converting them and taken a stab and reviewing Tuckey’s documentation, but its not clear to me exactly what I need to do.

Does anyone know if there is a good source or online converter available for converting the following IIS rewrites to work with Commandbox

Did you see this blog?

https://www.ortussolutions.com/blog/the-12-tips-of-commandbox-christmas-day-10-converting-apache-and-iis-url-rewrites

It has an example IIS rule being converted to Tuckey that looks familiar to what you have.

You’ll want to follow these instructions to enable your rewrite logs to help you debug them

https://commandbox.ortusbooks.com/embedded-server/server-logs#rewrite-log

Yes I followed the blog as best I could but keep getting a Not found error because something isn’t quite right. I not sure how to debug it … I’ll just keep trying different stuff.

Did you follow the instructions in the docs I sent you to enable and tail the rewrite logs with a debug/trace start? Also, I can’t help much without seeing the rules you’ve created that you’re trying.

The rewrite logs when trace is enabled literally give you a blow by blow of every single condition in your rewrite rules that’s evaluated and the result. That’s how you debug it. If I have some free time, I can try to help you convert them, but I’ve got work piled up ATM so all I can do is help you through it :slight_smile:

Reviving an old thread, but I think I might be having the same issue, and I have a bit more specifics.
I’m trying to use a proxy rule in the Tuckey URLRewrite and either using the xml format with

<rule>
        <from>^/test/(.*)</from>
        <to last="true" type="proxy">http://test.com/$1</to>
    </rule>

Or a similar set up with a .htaccess format file but using the [P] flag.
I get the same error

[ERROR] io.undertow.request: UT005023: Exception handling request to /apply-gateway-v1/owners/127667077/documents
java.lang.NoClassDefFoundError: org/apache/commons/httpclient/methods/RequestEntity

On Tuckey’s documentation it says that commons-codec and commoncf-http are required on the classpath to support proxy rules, so I guess this is a limitation of the CommandBox redirect? I did try to add those libraries to the libDirs folder but that didn’t help, I guess they need to go on the runwar classpath?

The Tucky proxy functionality does require the httpclient lib to exist in Runwar’s classpath. We added these libs as part of this ticket
https://ortussolutions.atlassian.net/browse/COMMANDBOX-1194
but it broke stuff so we reverted it again here:
https://ortussolutions.atlassian.net/browse/COMMANDBOX-1352

My recommendation is to not use the Tuckey-style rewrites or proxy at all, but to instead use the newer and more powerful Server Rules which are based on Undertow’s native “Predicate Language”.

These are capable of performing proxies as well:

regex('^/test/(.*)') -> reverse-proxy({'http://localhost:5678/'})

That said, after doing a quick test, it appears Undertow’s proxies don’t allow you to have dynamic bits in the URLs being proxied to. I’ll have to do some more testing on that and enter a ticket with Undertow.

Your best bet may be just dropping the missing jars in your classpath and being wary of the bug I linked to above.

I did try to add those libraries to the libDirs folder but that didn’t help

I’m pretty sure that should work. Are you sure you weren’t just still missing some jars?

FYI, here’s the thread I started on the Undertow list to discuss how to provide the same sort of proxy functionality you were using in Tuckey.

https://groups.google.com/g/undertow-dev/c/wSzMPTlHy9Q/m/acPCmUovAgAJ

Thanks for the quick response,
I think this might work for us but I’m having a hard time getting reverse-proxy to work with https.
So my rule would be something like:

"path-prefix('/apply-gateway-v1') -> reverse-proxy({'https://www.google.com/v1'})"

Then when I make a request to /apply-gateway-v1/test I get a 503 with body of:

<html>
<head>
	<title>Error</title>
</head>
<body>Service Unavailable</body>
</html>

I did validate that we have access to that URL from the server and http seems to work fine.
I did get this trace from the request:

[DEBUG] requested: '/apply-gateway-v1/test'
[TRACE] Server Rules: Path prefix(s) [/apply-gateway-v1] MATCH input [/apply-gateway-v1/test] for HttpServerExchange{ GET /apply-gateway-v1/test}.
[TRACE] Server Rules: Storing "remaining" string of [/test] for HttpServerExchange{ GET /apply-gateway-v1/test}.
[TRACE] Server Rules: Predicate [path-prefix( '/apply-gateway-v1' )] resolved to true. Next handler is [reverse-proxy( 'https://www.google.com/v1' )] for HttpServerExchange{ GET /apply-gateway-v1/test}.
[WARN ] responded: Status Code 503 (/apply-gateway-v1/test)

Which indicates that the extra path at the end of the request is being stored so I guess it would pass it downstream.

I guess my question here would be if this is a known limitation or am I missing some config?

Thanks,
Fernando.

In my experience, Google (and other major websites) have some sort of configuration in place that prevents them from being proxied to, so I wouldn’t use them in your tests. Point to another local CommandBox server started with --debug so you can server log --follow and “see” the requests come in.

Right, if you read the post I linked to above on the Undertow mailing list, that’s part of the current behavior of the reverse proxy feature-- it appends the original RequestURI onto the URL of the target server and I couldn’t find a way to change that. That was the whole point of my E-mail to their list, to ask how to change that behavior since in many cases, it’s desired to just completely rewrite the requestURI to a totally new one as part of the proxy.

Now that does bring up an interesting idea-- I wonder if I could pair a rewrite() handler prior to the reverse proxy handler to rewrite the URI locally before proxying it :thinking: I’ll have to do some testing with that.

Sorry, I put google as an example there but seems like any https URL is returning 503, and actually http://google.com does work. ( I get a 404 because of the extra path).

I’ll try out the rewrite followed by reverse-proxy too, that might work.

thanks!

I’d check and make sure your HTTPS endpoint is using a valid cert. You can trick your browser into using a self-signed cert, but java will curl up in the corner and cry if you ask it to connect to an invalid cert. I’ve honestly never tested the proxy over HTTPS before.
Also, check the logs on the remote server I’m not sure if the 503 is coming from your actual remote server that you’re proxying to or coming from the Undertow proxy code itself. It has several code paths that return a 503 so it could just be a generic response to something going wrong. I see a PROXY_REQUEST_LOGGER in Undertow’s code that I never noticed before. I think I’ll wire up Commandbox/Runwar to capture whatever it logs to see if it contains other clues.

I tested the rewrite coupled with the proxy and it actually works great. This was the rule I created:

{
    "web":{
        "rules":[
            "regex('^/test/(.*)') -> { rewrite('/$1'); reverse-proxy({'http://localhost:5678'}) }"
        ]
    }
}

Note Undertow is sort of picky about the curlies and semi-colons, so make sure you copy exactly what I have there. But basically i use the rewrite handler to rewrite the request in Undertow’s exchange first and then fire the proxy, which re-uses the newly rewritten request URI.

I don’t know if it will reveal anything, but can you replace your runwar jar in your commandbox lib with this version
https://s3.us-east-1.amazonaws.com/downloads.ortussolutions.com/cfmlprojects/runwar/4.5.3-SNAPSHOT/runwar-4.5.3-SNAPSHOT.jar

And do a server start --trace --console while you test. That version of Runwar will capture anything Undertow logs to its io.undertow.proxy logger and present it to the console. There isn’t a ton of logging in place, but I’m curious if it reveals anything about why its failing.

The certs are valid, I’m working to replace a Lucee+Nginx deployment with CommandBox, so the certs are actively being used by our current version of the application.

But still, to double check I created some tests using https://httpbin.org/.
So I created two rules:

"regex('^/bin/(.*)') -> { rewrite('/$1'); reverse-proxy({'http://httpbin.org'}) }",
"regex('^/bins/(.*)') -> { rewrite('/$1'); reverse-proxy({'https://httpbin.org'}) }"

When I hit http://localhost:8001/bin/get I do get the expected response and see in the logs:

[DEBUG] requested: '/bin/get'
[TRACE] Server Rules: Regex pattern [^/bin/(.*)] MATCHES input [/bin/get] for HttpServerExchange{ GET /bin/get}.
[TRACE] Server Rules: Storing regex match group [0] as [/bin/get] for HttpServerExchange{ GET /bin/get}.
[TRACE] Server Rules: Storing regex match group [1] as [get] for HttpServerExchange{ GET /bin/get}.
[TRACE] Server Rules: Predicate [regex( pattern='^/bin/(.*)', value='%{RELATIVE_PATH}', full-match='false', case-sensitive='false' )] resolved to true. Next handler is [PredicatesHandler with 2 predicates] for HttpServerExchange{ GET /bin/get}.
[TRACE] Server Rules: Executing handler [rewrite( '/${1}' )] for HttpServerExchange{ GET /bin/get}.
[DEBUG] Server Rules: Request rewritten to [/get] for HttpServerExchange{ GET /bin/get}.
[TRACE] Server Rules: Executing handler [reverse-proxy( 'http://httpbin.org' )] for HttpServerExchange{ GET /get}.

When I hit the https URL: http://localhost:8001/bins/get I still get the 503 with a very similar log followed by the 503 warning:

[DEBUG] requested: '/bins/get'
[TRACE] Server Rules: Regex pattern [^/bins/(.*)] MATCHES input [/bins/get] for HttpServerExchange{ GET /bins/get}.
[TRACE] Server Rules: Storing regex match group [0] as [/bins/get] for HttpServerExchange{ GET /bins/get}.
[TRACE] Server Rules: Storing regex match group [1] as [get] for HttpServerExchange{ GET /bins/get}.
[TRACE] Server Rules: Predicate [regex( pattern='^/bins/(.*)', value='%{RELATIVE_PATH}', full-match='false', case-sensitive='false' )] resolved to true. Next handler is [PredicatesHandler with 2 predicates] for HttpServerExchange{ GET /bins/get}.
[TRACE] Server Rules: Executing handler [rewrite( '/${1}' )] for HttpServerExchange{ GET /bins/get}.
[DEBUG] Server Rules: Request rewritten to [/get] for HttpServerExchange{ GET /bins/get}.
[TRACE] Server Rules: Executing handler [reverse-proxy( 'https://httpbin.org' )] for HttpServerExchange{ GET /get}.
[WARN ] responded: Status Code 503 (/get)

I do see some extra log that keeps repeating itself after that:

[DEBUG] io.undertow.proxy: Attempting to reconnect to failed host https://httpbin.org
[DEBUG] io.undertow.proxy: Failed to reconnect to failed host https://httpbin.org

Actually this seems to be coming from PROXY_REQUEST_LOGGER so I guess its already turned on:

I did find a similar error reported but it looks like it was resolved some years ago, so not sure if it’s the same thing:

I also tried the newer version of runwar but no luck there either.

Thanks for all the help and let me know if you have other suggestions.

Ahh, I found it-- I was looking in the wrong logger. The logging output in the ticket you linked to above gave me the clue that the error I was looking for wasn’t coming form the proxy logger.

java.io.IOException: UT000065: SSL must be specified to connect to a https URL
        at io.undertow.client.http.HttpClientProvider.connect(HttpClientProvider.java:94)
        at io.undertow.client.UndertowClient.connect(UndertowClient.java:161)
        at io.undertow.server.handlers.proxy.ProxyConnectionPool.openConnection(ProxyConnectionPool.java:274)
        at io.undertow.server.handlers.proxy.ProxyConnectionPool.connect(ProxyConnectionPool.java:550)
        at io.undertow.server.handlers.proxy.LoadBalancingProxyClient.getConnection(LoadBalancingProxyClient.java:340)
        at io.undertow.server.handlers.proxy.ProxyHandler$ProxyClientHandler.run(ProxyHandler.java:329)

So the error is that the internal proxy client inside undertow is angry that you’ve pointed it to an HTTPS URL but the SSL flag wasn’t passed internally when adding the host to the client (not something you or I can control as this is deep inside of Undertows proxy handler builder). But Undertow logs this as a DEBUG level message, not a warning or an error :angry: It comes as a DEBUG level message in the io.undertow.request logger. Undertow has a lot of random logging and CommandBox cherry picks the loggers and log levels that seem the most useful. Any WARN or ERROR level log message for any logger for example will get picked up, but this one wasn’t.

So as far as the fix-- this appears to totally be a bug (and I’ve only ever tested the proxy on HTTP URLs!) If you look at the ProxyHandlerBuilder class, you’ll see it calls

loadBalancingProxyClient.addHost(url);

for each host you provide. The overloaded addHost( URI ) method which only accepts a URI sets the SSL flag to null and nowhere does the proxy client seem smart enough to look at the URL and default that setting accordingly. I’ll report this to their bug tracker and hopefully there will be a fix in the next version of Undertow.

Here is my report to the Undertow dev mailing list

https://groups.google.com/g/undertow-dev/c/8WsqPTdfiDE/m/0p4vBnqBAQAJ

Got it, thanks for all the details, this was an interesting learning experience :smiley:
I’ll follow that ticket, hopefully we get some feedback quickly.

Thanks,
Fernando.