Setting up Ghost behind a different reverse proxy (træfik)
Whilst installing Ghost on Ubuntu using the recommended setup I encountered a step that required installing nginx.
I like nginx, I've used it a bunch in the past but recently whilst experimenting with Kubernetes I've moved my reverse proxying needs to træfik. So I didn't want to install it if it wasn't needed.
I continued the setup without it but quickly discovered that ghost install
is not a fan of that and will not start without it. So I gave in, installed it, and continued with the ghost installation; setting the website url to be https://blog.wastedcake.com
and ignoring everything about SSL because I've set that up with træfik.
Setup completed, with træfik pointing to my server on port 80, I tried to access my new blog only to be met with a too many redirects
error.
I didn't setup any redirects so that was a surprise. I checked what the issue was with curl -svILk https://blog.wastedcake.com
and found that it was basically just stuck in a loop; redirecting to itself.
The Adventure
Disable nginx? nope.
Ok let's go back to my original plan of just not involving nginx. sudo service nginx stop
.
I found the Ghost configuration file /var/www/ghost/config.production.json
which showed me that Ghost was running on port 2368 and hosted on 127.0.0.1 so was not accessible externally (hence nginx as the reverse proxy) so I just changed with ghost config server.host 0.0.0.0
and tried to access it locally with a simple curl curl localhost:2368
and got Moved Permanently. Redirecting to https://localhost/
. Huh? how? where? Nginx isn't running (and it's config had no redirects anyway).
I tried again with 127.0.0.1:2368
and got redirected to 127.0.0.1
. So something is just stripping out the port...
I found this test which makes sure if the blogs host is different to the requested url it just redirects the user to the correct url but that would mean I should be getting redirected to https://blog.wastedcake.com.
So I set the host headers to make sure nothing like that was triggered;
curl --verbose --header 'Host: blog.wastedcake.com' 'localhost:2368'
Moved Permanently. Redirecting to https://blog.wastedcake.com/
This looks better but it's still redirecting me around in circles.
Http instead of Https? nope.
I decided to just change the url that I configured the site with to just be http
and see what happened;
ghost config url http://blog.wastedcake.com
ghost restart
And it worked!
I could access my site and it seemed fine. Until I noticed now all the links on my page were to absolute pages `http://blog.wastedcake.com/whatever` and not relative links. I don't really want that... http->https redirection will fix it but it's not nice. There has to be a better way. So I changed the url back to https.
Reconfigure nginx? yes!
Here's the default Ghost nginx configuration found here /var/www/ghost/system/files/blog.wastedcake.com.conf
.
server {
listen 80;
listen [::]:80;
server_name blog.wastedcake.com;
root /var/www/ghost/system/nginx-root; # Used for acme.sh SSL verification (https://acme.sh)
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host https;
proxy_pass http://127.0.0.1:2368;
}
location ~ /.well-known {
allow all;
}
client_max_body_size 50m;
}
Something has to be causing this redirect loop. Http works so it's related to https and we know Ghost has its own redirect rules. It's doing odd stuff there so it's not irrational to assume it's also doing a http->https redirect internally.
- Go to https://blog.wastedcake.com
- Traefik forwards request to http://x.x.x.x
- nginx picks that up and proxies the request to Ghost using the same protocol
-proxy_set_header X-Forwarded-Proto $scheme;
- Ghost isn't happy with the non-https request
- Ghost then redirects the user to https://blog.wastedcake.com
- Bam, back at #2
Taking a look at Ghosts source code I found a test which ensures this behavior
it('blog is https, request is http', function (done) {
urlRedirects.__set__('urlUtils', urlUtils.getInstance({
url: 'https://default.com:2368/'
}));
host = 'default.com:2368';
req.originalUrl = '/';
redirect(req, res, next, getBlogRedirectUrl);
next.called.should.be.false();
res.redirect.called.should.be.true();
res.redirect.calledWith(301, 'https://default.com:2368/').should.be.true();
res.set.called.should.be.true();
done();
});
So we have to make sure the final proxy to Ghost is https or else it will just redirect us.
The Solution
The problem is ultimately that Ghost redirects http
to https
if you've setup your blog to work on https
. I'm not a fan of this, it's better to leave this stuff up to the reverse proxy.
With nginx
So to fix this issue and keep nginx running it's as simple as saying the protocol used is always https by changing configuration found here; /var/www/ghost/system/files/blog.wastedcake.com.conf
By replacing
proxy_set_header X-Forwarded-Proto $scheme;
with
proxy_set_header X-Forwarded-Proto https;
You can then restarted ghost and nginx;
ghost restart
sudo service nginx restart
And everything should work as expected. Calls to http://x.x.x.x should now be treated by ghost as https and it wont try to redirect you.
Without nginx
Now we know what the problem is we can go back and fix the redirect loop we got back when we turned off nginx.
Summary of previous steps;
- Stop/remove nginx
- Set Ghost to expose itself externally
ghost config server.host 0.0.0.0
and restart itghost restart
- In your chosen reverse proxy set it send traffic to http://x.x.x.x:2368 and set the header
X-Forwarded-Proto https
Example in træfik;
[http.services]
[http.services.blogwastedcake.loadBalancer]
[[http.services.blogwastedcake.loadBalancer.servers]]
url = "http://x.x.x.x:2368"
[http.routers]
[http.routers.blogwastedcake]
entryPoints = ["websecure"]
rule = "Host(`blog.wastedcake.com`)"
service = "blogwastedcake"
middlewares = ["[email protected]"]
[http.routers.blogwastedcake.tls]
certResolver = "default"
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: https-protocol-header
spec:
headers:
customRequestHeaders:
x-forwarded-proto: "https"