bx.ee

Jeremiah Gowdy

Asherah

Asherah is a family of open source projects that leverage the power of envelope encryption to provide a framework for key management and rotation for encrypted fields in databases or encrypted blobs in object or file storage. Asherah roots the envelope encryption in a trusted store like Amazon KMS, but avoids calling Amazon KMS for each decrypted field, as KMS is just used to bootstrap the envelope encryption hierarchy. Asherah is currently used in production at GoDaddy and is involved in serving a very large number of the websites on the internet today. I personally designed Asherah and built the prototype implementation and trialed the concept in production in C#, and I contributed to the Java implementation as well. A team at GoDaddy then wrote our current primary implementation in Go. The Go implementation can be consumed via sidecar, however, I’ve built working wrapper libraries in several languages that call the Go implementation compiled into a C shared library via FFI.

Asherah

Cobhan

Cobhan is the name of the banner of projects I’ve undertaken to explore the idea of leveraging FFI to provide unified shared library code to multiple programming languages. At GoDaddy, we have developers writing JavaScript/TypeScript/Node, Ruby, Python, Go, Rust, C, C++, Swift, Kotlin, C#, and Java. Thus, cross-cutting libraries end up being written once per platform and then supported with varying levels of effort / contribution by the consuming teams. This also allows the different implementations of shared libraries to drift over time.

Instead of continuing to build per-language implementations of the same cross-cutting libraries, we use one of the languages capable of generating native C ABI shared libraries, and then we wrap that library with a language specific FFI wrapper. The native C ABI shared libraries can be written in these candidate languages: C, C++, Go, and Rust. Using C or C++ is possible, however we minimize the amount of unmanaged development we do in order to reduce the chances of introducing memory safety bugs and having a stability impact. Instead, our primary languages for creating the “base” libraries are Go and Rust.

Go is a great programming language with a very small learning curve that allows for writing high performance native compiled code. However, the Go compiler is very limited in its implementation of thread local storage. The Go compiler only generates libraries with the init-exec thread local storage model. These libraries are not suitable for late loading with dlopen() as is done with FFI because they require a hack where the libc reserves some space for dlopen()’ed libraries. This hack is implemented by glibc, so Go compiled shared libraries can be dlopen()’ed on glibc. However, this hack is not implemented by musl libc, and therefore you cannot use dlopen() to load Go libraries on musl libc distros like Alpine Linux. Go libraries also assume they will be passed additional non-standard parameters when initialized and tends to crash when not passed these non-standard parameters. In short, the Go compiler seems to embrace non-standard behavior and glibc’isms.

One of the experimental concepts of Cobhan is the “Cobhan buffer” which is used to transfer any non-scalar data types including strings and buffers / byte arrays. The Cobhan buffer is length delimited rather than null terminated. One of the advantages of using a length delimited serialization format is reduced likelihood of heap corruption due to missing null termination or other overflows. Cobhan buffers have a length when they are allocated and from there they only shrink (when a smaller value is copied in), they never grow.

Leveraging FFIs at GoDaddy
Cobhan-Go

Asherah-Cobhan

Asherah-Cobhan is a wrapper for Asherah-Go that packages the project up for consumption via FFI

Asherah-Cobhan

Asherah-Node

Asherah-Node is a Node Addon API wrapper for Asherah-Cobhan written in C++

Asherah-Node

OpenSSL

I contributed the Windows implementation of OpenSSL’s secure memory APIs in this PR.

Asterisk

I contributed a few different patches to Asterisk over the years.

Node.js

I contributed a patch to enable pointer authentication on arm64 builds of Node.js.