Please enable JavaScript to view the comments powered by Disqus.

Teddy Hartanto

thoughts and experiments

I haven't talked much about this, but I've been building Hozuko, Singapore's direct-landlord property rental platform. I'll share more about that in future posts.

For now, I need to vent:

Mailgun is crap.

TL;DR: I had an email deliverability problem. Their customer service took 2-3 weeks to reply and claimed that they'd fixed my problem when they didn't. They even asked me to upgrade my plan. I upgraded, and that did nothing. The problem is still there. I filed another ticket. It's been 2 weeks, and there's no news. Screw Mailgun. I'll switch to Postmark.

It's been over a year since I wrote the first line of code for Hozuko. If there's one thing I've learned, it's this:

Building a great product is extremely hard.

Every part of the system has to fit together perfectly—like a well-oiled machine—for the whole thing to work. It's even harder when you're going solo. But that's the challenge I chose. I want to prove to myself (and the world) that this can be done alone.

Part of what makes product-building so hard is choosing the right vendors—auth providers, email services, analytics platforms, etc. There are dozens of ways to accomplish any goal, and there is no shortage of tools that claim to be the best.

Take auth providers, for example:

  • Supabase
  • Firebase
  • Next Auth
  • WorkOS
  • Kinde
  • Clerk
  • SuperTokens
  • Hanko
  • Ory
  • Lucia auth
  • Build-your-own
  • and more...

On paper, they all seem similar. Feature checklists look the same. But when you actually use them, the differences are night and day. Some are just plain bad.

Here's the analogy I now live by when evaluating vendors:

Choosing a vendor is like hiring a chef. Every chef claims to be able to make spaghetti, fried rice, and pudding on their resumes. But only when you taste their cooking do you realize some of them break spaghetti in half before boiling it or make fried rice using freshly cooked rice. It's the same with software vendors. The devil is in the details.

Mailgun: The Chef Who Can’t Cook

Let me walk you through what happened with Mailgun.

📅 6 Mar 2025

I noticed emails to @live.com and @hotmail.com were failing. I opened a support ticket:

My emails to addresses on domain @live.com or @hotmail.com are failing. Could you please advice how can we fix this? Pasting the email metadata below.

Mailgun returned, asking me to verify my business and describe my email practices. After I answer all those questions, they'll assign Hozuko a shared IP pool with a higher reputation.

📅 7 Mar 2025

I replied that the verification form was inaccessible, but I explained how we use email at Hozuko.

📅 8 Mar 2025

They said they’d escalate the ticket to another team.

📅 13 Mar 2025

I followed up. No response.

📅 17 Mar 2025

I followed up. No response.

📅 25 Mar 2025 — finally a reply:

Hello Teddy,

Thank you for contacting Sinch Mailgun Support.

After reviewing the currently assigned IP, it appears that a problematic sender led to the IP's listing on a reputable blocklist. While we work to delist this IP, we have reassigned your sending to a new IP.

You are currently on our free plan which is a very limited plan that is only for basic testing.

It took 12 business days to tell me this.

📅 27 Mar 2025

Emails to Microsoft inboxes were still failing.

I replied:

I'm still running into the same issue. What do you need from me so that I can be on the shared IP pool with a higher reputation as X previously mentioned? If I need to be on the paid pricing plan for that, that's no problem. Just let me know. I can only commit to the basic plan as of now though looking at the volume of our emails.

📅 28 Mar 2025

Their response:

As mentioned in the previous email, we will need your account to be on a paid plan in order to proceed. The current plan you are on is intended solely for testing purposes. Once you have completed the upgrade, please let us know, and we will be more than happy to assist you further!

But here’s the thing: they never clearly said I needed to upgrade. If they did, it was buried between the lines.

Anyway—I upgraded to the 10K basic plan and asked for help again.

Their reply? Exactly the same message as before:

“Your IP was blocklisted, we’ve reassigned it.”

📅 4 Apr 2025

Emails still weren’t working. I filed yet another ticket.

Their reply:

Thank you for contacting Sinch Mailgun Support.

After reviewing the ticket, it has been determined that we will need to engage another group of colleagues.

We are transferring the ticket to them, and they will be responding as soon as possible.

📅 19 Apr 2025

Still no response.

That’s it.

I’m done.

I’m moving to Postmark.

Screw Mailgun.

Nevermind ditching MySQL. Traccar recommends not using the embedded H2 database for production

It's interesting that we need to run the following to secure the MySQL server:

sudo mysql_secure_installation

What happens if we don't?

teddy@debian-2cpu-2mem-sin-1:~$ sudo mysql_secure_installation 

Securing the MySQL server deployment.

Enter password for user root: 
The 'validate_password' component is installed on the server.
The subsequent steps will run with the existing configuration
of the component.
Using existing password for root.

Estimated strength of the password: 100 
Change the password for root ? ((Press y|Y for Yes, any other key for No) : Y

New password: 

Re-enter new password: 

Estimated strength of the password: 100 
Do you wish to continue with the password provided?(Press y|Y for Yes, any other key for No) : Y
By default, a MySQL installation has an anonymous user,
allowing anyone to log into MySQL without having to have
a user account created for them. This is intended only for
testing, and to make the installation go a bit smoother.
You should remove them before moving into a production
environment.

Remove anonymous users? (Press y|Y for Yes, any other key for No) : Y
Success.


Normally, root should only be allowed to connect from
'localhost'. This ensures that someone cannot guess at
the root password from the network.

Disallow root login remotely? (Press y|Y for Yes, any other key for No) : Y
Success.

By default, MySQL comes with a database named 'test' that
anyone can access. This is also intended only for testing,
and should be removed before moving into a production
environment.


Remove test database and access to it? (Press y|Y for Yes, any other key for No) : Y
 - Dropping test database...
Success.

 - Removing privileges on test database...
Success.

Reloading the privilege tables will ensure that all changes
made so far will take effect immediately.

Reload privilege tables now? (Press y|Y for Yes, any other key for No) : Y
Success.

All done!

It seems that the default MySQL installation configuration allows anonymous users login and remote root login.

References [1] https://docs.vultr.com/how-to-install-mysql-on-debian-12

I previously chose to run MySQL because of the added challenge of managing it. I love challenges. However, it has become a burden because not only do I need to set it up when I have a new VPS, but also it consumes a lot of memory and I don't need that. I now prefer to keep things simple.

Funnily enough, by choosing to keep things simple, I'm making things a bit challenging for myself yet again. I've spun a new VPS on Hetzner and got a Writefreely instance up. Next is figuring out how to restore the data.

When I migrated from AWS->DO, I dumped the MySQL DB from AWS and restored it into DO. That's not possible now that we're changing the SQL engine. Luckily, I found mysql2sqlite. A savior!

It wasn't a smooth process because that tool is old and there were some INSERT statements that contain binary values as such:

INSERT INTO `table` VALUES (1,_binary '<BINARY>',_binary '<BINARY>');

So, I just dump the posts table:

sudo mysqldump --skip-extended-insert --no-create-info --compact writefreely posts > dump_$(date -u +"%Y-%m-%d_%H%M%S")_posts-only.sql

Convert it into sqlite format:

./mysql2sqlite dump_2025-01-01_085842_posts-only.sql | pbcopy

Then run the INSERT statements against the sqlite3 DB in my Hetzner VPS.

Voila! We're back in business.

My parents are getting old. They sometimes forget where they place their belongings: wallets, debit cards, phones. Sometimes, their phones are actually somewhere out in the world. Other times, they just misplaced their phones somewhere inside the house. In either cases, we're all panic-stricken.

This is where I come in. I'm the technophile. Let's solve this using tech.

Not long ago, a friend told me about Life 360, a location sharing app. The idea is that I can install that app on my parents' phones and keep track of their locations.

But, the issue is that it asks for phone numbers. I don't like that. I went looking for alternatives and found two names in this space: 1. Owntracks 2. Traccar

Upon reading reviews, I've decided to try out Traccar.

Next, I need to host the Traccar server somewhere. Hosting it on the same server as the blog seems like a no-brainer. After all, who reads my blog? 😂🥹

Anyways, my blog was running on a 1vCPU 1GB memory DigitalOcean droplet with 25GB SSD. It costs me $6/mo. Here's the resource utilization: – CPU is ~1-2% most of the time. Never crossed 10%. – Disk usage is 26% – Memory sits at ~75% though. A whopping 46% was used for MySQL server.

It's obvious that the MySQL server need to go. But, even so, I'm not sure if the resources are enough for Traccar.

I found a good deal on Hetzner: 2vCPU, 2GB Mem, 40GB SSD at a price tag of $9.25/mo. It's not an apple-to-apple comparison, but the cheapest 2vCPU 2GB Mem DigitalOcean droplet comes with a price tag of $18/mo. I compared the prices of both VPS providers in case I needed an upgrade. It seems to me that Hetzner is cheaper.

Price wasn't the only reason I decided to migrate to Hetzner though: 1. It was just nice that Hetzner expanded into Singapore this year 2. I want to get a taste of Hetzner. DigitalOcean's log in page takes a long time to load 3. Migrating the systems would be a nice challenge during this lull period in my life. I need a challenge.

With all that, here's the plan: 1. Spin up a Hetzner VPS 2. Migrate my blog from DO –> Hetzner 3. Install Traccar

I wanted to also try out Coolify. Check out the intro video. But, let's do things the hard way now. I want to learn more about nginx so instead of having Coolify abstract the nitty gritty details away, I'd like to dive deep into the technicalities.

I hiked Mt Batur in Bali a few days ago and I heard an eye-opening anecdote. When my hiking guide — Kadek Bagawata — showed up, I was quite surprised. He was topless, barefoot, and wore only a sarung.

Sarung

I thought he was going to change into a proper hiking outfit. He did not...

He was very wholesome by the way. Before we embarked on the hike, he expressed gratitude that we're all able to come together and celebrate this hike together. He encouraged us to sing along with him on the journey. Yeah... he carried a guitar and a drum for the hike. That is insane. I've only seen that in video games, not in real life!

Because of his bare outfit, I thought, “oh, the terrain must be easy.” Well, it wasn't. There were screes. So many loose rocks and pebbles. He walked on them like it's nothing!

At the summit, we were greeted by a blanket of dense mist instead of a rewarding sunrise. At the top, I asked him why he decided to hike barefoot. Apparently, he had a good reason. He developed Tourette's Syndrome when he was 11. By chance, he discovered that his condition was hugely alleviated when he hike up the mountain barefoot.

That's when it hit me. Well, that could make sense. Tourette's is a nervous condition. As far as I understand, reflexology works by stimulating the nerves.

Walking on stones

As always, that made me meditate on how much we're taking for granted today are actually good for us? I wore shoes from when I was a little kid and it became something normal. But, is it really normal for humans to wear heavily padded footwear? What's the history of shoes? I know there's a barefoot running movement. I haven't quite wrapped my head around that topic, but I shall explore that.

Brb, going to walk on some stones downstairs...

A snail on the sidewalk

A snail on the sidewalk

You're walking or running on the sidewalk.

You spotted a snail.

There, in the middle of the sidewalk.

Do you avoid and walk past it?

Or do you pick it up and move it aside so no one would accidentally run over it?

A plate of stir-fry long beans and a fork

A plate of stir-fry long beans and a fork

You're in a restaurant, eating with friends or family.

The waiter served a plate of long beans right in front of you.

There's a fork provided with it.

Do you use the fork to try grabbing as much long beans as you want?

Or do you ask the waiter if they have like a big spoon or something that will make it easier to scoop the long beans?

Company with broken processes

You just started a new job in a new company.

It's been 6 months.

You start noticing the ugly things — the gaps in processes, the toxic behavior of a teammate that goes unnoticed, the mess in the systems.

Do you turn a blind eye, avoid the ugly things, and focus on what you like?

Or do you advocate for fixing the broken processes, call out the toxic behavior, fix the systems?

I've noticed two types of response when dealing with an external situation:

  • influence; or
  • adapt

Sometimes, when things are out of our control, we have no choice but to adapt. We change our mindset and perspective.

Some other times, we are given a choice: we can influence, or we can adapt. Influencing is more difficult. You have to take actions. It's more active. Adapting is relatively easier. It's passive. I'm not saying one is better than the other. I'm saying, be intentional with your choice of response.

I had a fascinating experience today. I was the de-facto event photographer at Ruth's casual wedding party. For the longest time, I've been mainly taking pics of mostly still subjects (natural landscape, buildings, etc) in an outdoor setting where light is typically abundant. Today, I had to take pictures of people in a completely different setting (moving subjects, indoor, with varying degree of lights). I have to admit that I was inexperienced. But, I learnt a few things. It was a great opportunity to practice my photography.

Lesson 1: on mind game

I'm an introvert. I don't mind approaching individual strangers. But, approaching a group of strangers talking amongst themselves? That was uncomfortable for me. It was out of my comfort zone. I was a bit disappointed that I wasn't proactive enough to invite people to take group pics. Of course, it doesn't come naturally to me. But, I could've done better. In retrospect, I feel that if I had change my mindset, I would have been able to overcome that mental blocker. If only I keep in my mind: “today's an important event for Ruth, and I have one job — to capture and memorialize the space, time, and people at today's event.” If I can keep that in mind, it would've been easy to overcome my reluctance to approach groups of people. I would have been able to capture more attendees.

Lesson 2: on candid pics

Instead, I took pictures of people discreetly, thinking I was going to capture some candid pics. But, candid pics are tricky, aren't they? In this particular event, there were a few obstacles to taking good candid pics: 1. We had a buffett. People don't clear their empty plates after they eat. Empty plates don't look great on pictures. People eating also don't look great on pictures. Those are unglamorous. I would've been pissed if I saw a photographer took a picture of me eating. Yet, that's the exact mistake. 2. People sat in a circle, making it difficult to include everyone's faces in the pictures. It's not nice to take pictures of some faces and some backs. 3. I couldn't easily position myself discreetly in a good angle to take good candid pics. The space was quite packed, and if I move to a certain good angle, people will notice. Then, it's won't be candid anymore.

In hindsight, it would've been nicer to just invite people to take group pics so they can prepare themselves and appear glamorous.

Lesson 3: on lenses

I'm using Sony alpha a6000 mirrorless camera. I brought 2 lenses today. One is a 35mm f/1.8 prime lens. The other is a 16-50mm f/3.5-5.6 lens. The reason I brought prime lens is because I know it take crisp pictures. It has a shallow depth of field. I thought it might be great for an event. And, I would argue that: it is, for certain setting. The reason I brought the other lens was due to the fact that the fixed focal length of the prime lens could pose a challenge when taking group pics. The field of view is not large, so it might be difficult to fit everyone into a pic.

During the event itself, the 16-50mm lens prove to be a better choice. Prime lens are too static. To take good pics, one have to move around. And there isn't much luxury to move around. A photographer has to be quick. Moments are fleeting. A prime lens also has a shallow depth of field. That becomes a problem when people stand in front of one another. Some people will get the focus, while others will be blurred. Good composition is difficult to achieve because it highly depends on the photographer's positioning, since zooming in and out is not an option. Lastly, as mentioned, the fixed focal length makes it hard to fit many people in a picture.

Telephoto lens is probably the best because its long-ranging zoom helps with sub-ideal positioning, rather than having to move here and there.

Lesson 4: on camera settings

Shutter speed has to be fast. People moves. Slow shutter speed creates blurry pictures when there's motion. Fast shutter speed freezes actions.

One strange thing I noticed was the different aperture on the pictures I took. That's weird because I set my camera on aperture priority mode. I intentionally set my aperture low so that I can get higher shutter speed. Should I have used shutter speed priority mode though? To answer that, I need to do some experiments.

Definitely, 1/80 shutter speed isn't enough. A quick googling tells me that 1/100 to 1/200 should be fine.

I was quite disappointed with the pics I took. Nonetheless, it was a good learning experience. I've tried to learn photography again and again in the past. But, the concepts just don't stick as much as when I practise it. As always, hands-on experience beats theory classes. Event photography is a totally different ballgame than landscape photography. I now have a higher appreciation for event photographers. They have to be super attuned to the environment they operate in. They have to think about their positions, their subjects' positions, the lighting, the turn of events, etc. They have to constantly adjust to capture the best moments so that the event owners can cherish those important and memorable events forever.

A short but intense focus leads to a better and quicker outcome than long but mild focus.

It's common sense, but I had a deeper understanding and appreciation when I delibrately reflect on it and experience it. Friends in my circle who are smart and quick to get things done are also the most focused people I know. They may not necessarily know a concept or a new technology, but they learn quickly with their intense focus.

Cultivate intense focus because it allows you to get more things done in a shorter time. You'll do more and have a fuller life. You don't need to live a long life, you just need to live a focused life.

I was prompted to write this short piece when reading an excellent write-up on procrastination by Paul Graham here. Read it!

When the SL & SR buttons of my Nintendo Switch joycon broke, I looked up how much it would cost me to get them repaired. It was way too expensive. The spare part is cheap and the repair is straightforward.

I decided to order the part and do the repair myself. After a successful personal repair, I decided to turn it into a side business.

Here are the lessons I learned.

  1. Starting is difficult. I’m only doing repairs, but imagine starting up a more complex business. I had stripped screws, broken tools, broken parts, missing parts, delayed work, uncomfortable conversations with customers. But they’re all normal part of starting up. Fuck ups are normal. Fuck ups are signs of growth
  2. Fake it till you make it. I had impostor syndrome when I started. But, how do you become good at something without ever feeling like a fucking noob? You have to just do it. Make mistakes, learn from them, avoid them, plan better, do better.
  3. Cheapest repair service wins. Cheap repair makes people truly consider repairing. Why would people bother with repairing if the repair cost comes at around 50-60% of buying a new one? They'd have to bear the hidden cost of conversing with the repairman, meeting with him, waiting for the repair, etc.
  4. People don't mind going down to my home. Some were driving.
  5. Do the right thing. I realized that I can't afford to botch my job and focus purely on the money. Sometimes, people bring their joycons for repair so that their children can play. I don't want to disappoint those kids.
  6. All parts are not equal. For instance, the quality of analog joysticks vary between suppliers.
  7. Mind the hidden costs. I can profit about $11 per 10 minutes of easy repair jobs. That's a whopping $66/hr. But, that calculation is valid if I do repair work only and that the problem has been obviously identified. There were hidden costs: chatting with customers, meeting them, troubleshooting their joycons, testing them. Troubleshooting is especially time-consuming when dealing with a more complex case.

There's a parallel between a good repairman and a doctor. A good repairman doesn't need to open up the joycon and use measuring tools like a multimeter to determine where the faults are. He can narrow down or sometimes accurately diagnose the faults by asking questions and testing for functions (symptomatic troubleshooting).

For example, a customer said that their joycon's battery is spoilt. I asked them what they mean by that. They told me that they plugged their joycon into the Switch console and it wasn't detected by the console. I wouldn't be too quick to conclude that this is an issue with the battery. It could be an issue with the charging rail connector instead of a dead battery.

Likewise, a good doctor rule out diseases by testing for functions (movement, reflex, etc) instead of jumping into asking patients to go for diagnostic procedures like CT scans, MRI, etc.

If I clear my inventory, I'm supposed to make about 5x the capital I invested into spare parts. But, I couldn't clear my inventory. I had to stop my repair service due to life circumstances. I did break even though. I managed to cover the tools I bought: a microscope, an iFixIt Pro Tech Toolkit, etc.

I bought a pair of Timberland boat shoes on Amazon for SGD80. They're at least SGD200 in physical stores. (I don't exactly remember because I posted this in 2025 when I should've done it in 2022)