Random Notes to Conclude 2019

1. OpenVPN & MFA

There are few tutorials about setting up OpenVPN and multi-factor authentication. For MFA, Google Authenticator is used here.

To setup OpenVPN server, I use OpenVPN road warrior installer. For MFA config, the script from egonbraun is forked and modified to work with road warrior setup: https://gist.github.com/carsonip/b02eecf9f7d036555a53fea6f516ced8

Also, the default OVPN server will renegotiate every 1 hour, but this will fail the MFA authentication, effectively disconnecting the OVPN connection periodically. To workaround this, use reneg-sec 0 in server config.

2. Amazon RDS EBS Lazy Load

It is observed that RDS instances created from EBS snapshots will not load the actual data entirely from S3 to EBS volume. This results in high read latency and replica lag on new replicas on new reads. See this blog post, this reddit discussion, and another blog post.

3. Python Reference Cycles, GC & Memory

To debug ref cycle and gc issues, gc module’s gc.set_debug(gc.DEBUG_SAVEALL) and the objgraph package are very useful.

Python __slots__ saves memory by allocating space for object attributes instead of dynamically doing it in the object’s __dict__. The latter will trigger dict resize which harms both memory usage and CPU usage. __slots__ may still be useful for classes that inherits a parent class that uses __dict__ instead of __slots__, but in this case the memory usage should be measured carefully, using sys.getsizeof and pympler’s asizeof. See this SO answer for good __slots__ tips.

Python 3.6’s dicts are more compact. For empty dict, it is 240B (Python3.6) vs 280B (Python2.7), and for a dict with 1000 keys, it is 4704B vs 12568B. See this mailing list message and Raymond Hettinger’s talk on compact dict.

Python 2 does not release memory for allocated integers due to implementation of its free list. See Python doc, this blog post, and this SO question.

4. Certbot in China

certbot-auto stucks in step “Installing Python packages” for cloud servers in China. It is probably due to PyPI somehow not being accessible from China. The trick is to set env var PIP_INDEX_URL to your favorite Chinese PyPI mirror, e.g. Aliyun. See this GitHub issue and this PR.