26/Oct 2019
1 min. read
I am dealing with golang and CI (namely CircleCI) recently. There are lots of tricks to speed up the build / test in CI and this is a short post about go fmt.
This CircleCI blog post suggests the use of this:
! go fmt ./... 2>&1 | read However, I find this unreliable as it works locally but exits with return code >128 for read on CircleCI. According to the man page, it indicates some kind of a timeout.28/Jul 2019
1 min. read
This popular book was on my list for a while until I recently have the time to read it. The book is about how the software systems are managed throughout their lifecycle in Google at its massive scale. Here I will jot down some key takeaways.
SRE enables a better balance between innovation and reliability of products. (Chapter 3)
Introducing planned outage may help identify parts that have false assumptions about reliability.2/Feb 2019
1 min. read
To work around some restrictions, I need to load balance outgoing traffic such that they appear to be coming from different IPs.
Here is how you do it with HAProxy:
backend servers server srv1 target.example.com:8080 check source 0.0.0.0 usesrc 10.10.0.4 server srv2 target.example.com:8080 check source 0.0.0.0 usesrc 10.10.0.5 server srv3 target.example.com:8080 check source 0.0.0.0 usesrc 10.10.0.6 Use mode tcp and this works for SSL connections too. The IPs like 10.10.0.4 in the example are private IPs, but they will be translated to different public IPs thanks to Azure’s SNAT.2/Feb 2019
1 min. read
Background: I have multiple 200GB+ MySQL dumps to import. It takes a lot of time.
pv shows you the progress so you have the idea how long will the import take. Pipe the dump.sql.gz file to gunzip instead of unzipping it then import. Saves much disk read and write, not to mention the time and effort. e.g. pv dump.sql.gz | gunzip -c | mysql grep is slow for a large file.10/Dec 2018
3 min. read
I have recently migrated to Amazon RDS from MySQL on Azure VM.
This percona article about migrating to RDS is actually very helpful. I have a few things to add though.
Setup My setup is a self-hosted MySQL 5.6 server in VM on Azure and a new Amazon RDS for MySQL 5.6.
AWS DMS I have tried AWS Database Migration Service (DMS). It is reasonably fast. The main problem is it does not look like mature technology at all.