Problems When Setting Up WriteFreely Part 2: Firewall
This is a two-part series. Read part 1 here.
In my previous post, I ran into an HSTS issue. I bought a domain name that's apparently HSTS preloaded. That means whenever I use a modern browser to visit my blog, it will be strictly loaded over HTTPS. My plan to simply load my blog over HTTP was foiled. To work around that, I ran WriteFreely in standalone mode, which supports SSL out-of-the-box. With that, I solved the problem. I can now visit https://teddyh.dev!
Or so I thought.
At least, until I found out that I still couldn’t load my blog. I was still seeing the same ERR_CONNECTION_TIMED_OUT in my browser. How come?
╭─teddy@teddy-ubuntu ~
╰─$ curl -vvv https://teddyh.dev
* Trying 54.151.219.0:443...
* TCP_NODELAY set
* connect to 54.151.219.0 port 443 failed: Connection timed out
* Failed to connect to teddyh.dev port 443: Connection timed out
* Closing connection 0
curl: (28) Failed to connect to teddyh.dev port 443: Connection timed out
╭─teddy@teddy-ubuntu ~
╰─$ curl -vvv http://teddyh.dev
* Trying 54.151.219.0:80...
* TCP_NODELAY set
* Connected to teddyh.dev (54.151.219.0) port 80 (#0)
> GET / HTTP/1.1
> Host: teddyh.dev
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 302 Found
< Content-Type: text/html; charset=utf-8
< Location: https://teddyh.dev/
< Date: Tue, 14 Jun 2022 08:05:09 GMT
< Content-Length: 42
<
<a href="https://teddyh.dev/">Found</a>.
* Connection #0 to host teddyh.dev left intact
The fact that I got a proper response over http should indicate that my standalone WriteFreely setup was working as intended. Indeed, the documentation said that WriteFreely should redirect http connections to https. However, I couldn't see any log of the http connections I made. Either WriteFreely doesn't log the http redirects, or something else was handling the redirects.
I was pretty sure that WriteFreely was the one handling the redirects. However, I had to verify. I tried checking for open ports using netstat:
[email protected]:~$ netstat -t4lpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:33060 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:3306 0.0.0.0:* LISTEN -
I saw nothing was listening to port 80 & 443. I scratched my head. How come? Earlier, I was able to get a response from port 80! (That shall be the topic of my next post)
Anyways, I tried another method to verify — I stopped WriteFreely and tried curling again:
╭─teddy@teddy-ubuntu ~
╰─$ curl -vvv http://teddyh.dev
* Trying 54.151.219.0:80...
* TCP_NODELAY set
* connect to 54.151.219.0 port 80 failed: Connection refused
* Failed to connect to teddyh.dev port 80: Connection refused
* Closing connection 0
curl: (7) Failed to connect to teddyh.dev port 80: Connection refused
╭─teddy@teddy-ubuntu ~
╰─$ curl -vvv https://teddyh.dev
* Trying 54.151.219.0:443...
* TCP_NODELAY set
* connect to 54.151.219.0 port 443 failed: Connection timed out
* Failed to connect to teddyh.dev port 443: Connection timed out
* Closing connection 0
curl: (28) Failed to connect to teddyh.dev port 443: Connection timed out
Seeing how my http connection was refused, I concluded that indeed WriteFreely was the one redirecting http->https. Still, it puzzles me why I couldn't connect to the https port.
I hadn't notice this at the time, but there was a subtle difference in the errors reported above: Connection refused vs Connection timed out. That should've been a clue.
It took me some time and some visits to Google & StackOverflow before it dawned on me: maybe a firewall was blocking https requests!
I checked my iptables and saw the word ufw scattered around. Aha! I remembered that I had once played with ufw on this server — as part of Linux Upskill Challenge. I've completely forgotten it because that was ages ago.
Sure enough, I saw https traffic was treated with the ufw default incoming DENY policy (since only ssh and http traffic is allowed):
[email protected]:~$ sudo ufw status
Status: active
To Action From
-- ------ ----
22/tcp ALLOW Anywhere
80/tcp ALLOW Anywhere
22/tcp (v6) ALLOW Anywhere (v6)
80/tcp (v6) ALLOW Anywhere (v6)
Once I configured ufw to allow https traffic, I was able to visit my blog on my browser! :)
This ends the two-part series on problems I faced when setting up my WriteFreely instance. Writing these posts have been truly rewarding! I learned a new command (tcpdump), new tricks with old commands, and I gained a better understanding of Linux & networking. It made me better at troubleshooting too.
Running my own blog and writing down my learnings have paid huge dividends! :)
Next, I think I'll cover some things I learned in the process of troubleshooting:
- Why didn't
netstatshow me processes listening to port80&443, yet I was able to connect to port80? CONN_REFUSEDvsCONN_TIMEOUT
Stay tuned!
PS:
It didn't cross my mind at the time, but I could've ssh into my server and curl my local port 443:
[email protected]:~$ curl -I https://teddyh.dev --resolve 'teddyh.dev:443:127.0.0.1'
HTTP/2 200
content-type: text/html; charset=utf-8
date: Tue, 14 Jun 2022 08:52:53 GMT
Had I done so, I would have been assured that my WriteFreely setup was working correctly. It would have also pointed me to look at my firewall rules sooner.
On a side note, troubleshooting a confluence of issues can be challenging. It's easy to assume that A is caused by B and B only. But, it could've been caused by B, C, and D. I guess multi-factor problems like these reflect life more accurately, wherein issues are often caused by multiple colluding variables.