bx.ee

LLM Shared Storage

In this era of downloading different LLM models, we need the vendors of different inference engines and their associated user interfaces to use a unified directory structure for downloading models. If I'm working with two different platforms that support GGUF, those two platforms should download and store models in the same default path and directory structure. I currently have terabytes of models downloaded, and the level of duplication is beyond annoying.

/tmp

The design decision to have a big shared /tmp that's world writable (and assumed by many packages like installers to not be mounted +noexec) was a huge mistake. Applications that assume /tmp rather than using TMPDIR (standards) are awful. We should migrate the shared /tmp from world writable to group writable and have a particular group membership to be allowed to write to it for daemons that don't have home directories until they can be fixed up to not need a shared /tmp.

Port scanning defense

One of the port scanning defenses that of one of our network defense appliances at work takes is the approach of defending against port scans by responding to every SYN on a closed or filtered port with a stateless SYN+ACK. This shifts the burden back to the party who sent the SYN, and doesn't consume any memory or other resources on the server side. If the party who sent the SYN is doing mass SYN scans with raw SYNs, the outcome will be inaccurate results. If the party who sent the SYN is doing anything with the actual connection, like scraping headers, OS detection, protocol analysis, etc. they will experience a tarpitting effect as they timeout trying to continue to use the connection after the SYN+ACK.

I recreated this defense using nftables and the synproxy syntax. At the end of my input chain, before it would have went to policy drop, I added:

            tcp flags syn jump SYNPROXY
        

Then I added a new SYNPROXY chain that does the following:

            chain SYNPROXY {
                synproxy mss 1460 wscale 7 sack-perm timestamp
                tcp flags syn,ack drop
            }
        

With this change, you can see that port scans of bx.ee now return every port as open.