Because we are not perfect and we do not pretend to be.
Failures are the inevitable aspect of learning and should be celebrated. Here lies before you all our shameless shortcomings over the years, and lessons learned along the way.
2020
Slow off-hours support
- PROBLEM – we’re a small company and our core staff (usually) does not work during off-hours (USA time)…which is unfortunately when some clients have issues or downtime issues. We had night staff as well to cover USA night hours as well as Asia/Pacific day times but they don’t respond quick enough since we often go 1-2 months without any issues. Understandably, this is painful for our clients.
- SOLUTION – it’s costly but we’ve added fantastic support staff to man our support tickets 24/7. We are now able to support clients 24/7 should any issues happen in the middle of the night. They are fast and helpful.
2019
Aggressive Security
- PROBLEM – we had an over-zealous security system that falsely blocked real users as hackers, especially when going through the Cloudflare proxy (which marks all visitors under the same IP). It was blocking users on frontend and admins on backend. We lost a dear friend and big client who loved our speeds but couldn’t stand these erratic outages.
- SOLUTION – we simply adjusted the system to be more compatible with the Cloudflare proxy. We’re very embarrassed not to have caught it sooner. Luckily, we are still great friends.
Low bandwidth limits
- PROBLEM – client site went down in middle of the night for using way too much bandwidth (like 10x more). She was understandably angry and left our service immediately.
- SOLUTION – we raised our bandwidth limits, also enabled system notifications when users get close to the limit, also put a quick and easy remedy in the docs for how to quickly remedy the issue. We also have 24/7 ticket support now to prevent outages for silly reasons like this.
2018:
Email Deliverability
- PROBLEM – clients emails weren’t delivered.
- SOLUTION – there’s no easy one. Email servers everywhere are making it harder to get in. While we still allow email usage on our servers, we have marketing this feature as an unofficial one following the route of other big webhosts who don’t even offer email hosting at all. We’ll do our best to help you mitigate issues and reconfigure DNS and servers and such but we don’t officially guarantee anything. We are a webhost, not an email host.
2017:
Mixed server environments
- PROBLEM – we were managing too many servers with too many custom configurations. Fixing or diagnosing issues often required lots of studying old logs and asking other admins what they did in the past.
- SOLUTION – while we still like mixing stacks, we’ve since introduced much more uniform ways of deploying and customizing our server stacks. Problems are easier to diagnose and also deploy across all servers quickly.
2016:
Difficult newbie clients
- PROBLEM – we love helping people, almost to a fault. This made us too popular with new users that don’t know how to manage their site, clogging support channels, and raising costs for everyone.
- SOLUTION – we re-marketed our service as being only for tech-savvy customers. It decreases sales and profits but gives so much less hassle and frees up our time to help existing clients.
2010
Poor hardware
- PROBLEM – finding a reliable hardware vendor was painful. So much downtime despite their constant promises of many 9’s.
- SOLUTION – there’s no easy solution. We simply tried all of them until we found some that were reliable. We quickly learned never to buy hardware from companies that dealt directly with end users. Just different culture and different service altogether.