Unless you’re a security specialist, you probably never worry about cyber security beyond following the common sense practices like not visiting sketchy websites, not clicking on suspicious links that come from unknown sources, and not typing your credit card number on a sketchy website.

Unfortunately, if you’re doing anything of economic / societal value, you must immediately start worrying about cyber security and changing habits.

The mental model you should have is that EVERYTHING on the internet can be and will be compromised. If you’re a computational scientist, that means any package you install can immediately lead to loss of all your passwords, ssh keys, and other sensitive information, and your most immediate action to mitigate this can lead to irreversible damage:

Please be careful when revoking tokens. It looks like the payload installs a dead-man’s switch at ~/.local/bin/gh-token-monitor.sh as a systemd user service (Linux) / LaunchAgent com.user.gh-token-monitor(macOS). It polls api.github.com/user with the stolen token every 60s, and if the token is revoked (HTTP 40x), it runs rm -rf ~/. (It looks like it might also have a bunch of persistence mechanisms. I haven’t studied these closely.) (from May 11 TanStack attack discussion)

In the first voice, I share my understanding of the two most recent exploits that scared me enough into actually changing my daily habits: TanStack supply chain attack and CopyFail.

In the second voice, I share the practical steps I’m taking to mitigate these risks and why most python developers immediately became radioactive liabilities for any company.

Some readers (particularly not in tech) may be tempted to dismiss these warnings because they don’t use Github tokens or ssh keys, and so do not have much to lose. So let me paint a realistic picture.

Losing ssh keys or API tokens is the default behavior of most exploits because they’re required to gain access to more critical/valuable systems, which would be the source of revenue to the exploiters. See, finding an exploit used to be very, very difficult and required a lot of effort from a very narrow set of specialists. It was so expensive, you could only justify such expenses if you could penetrate, say, a large proprietary database. It is even rumored that intelligence agencies might have, at any point in time, a set of silent exploits that they’ve never used or disclosed, saving them for a very rare special occasion which would justify their use.

With the recent progress in AI, finding exploits is becoming cheaper and faster, which inevitably will result in the lowering of the threshold of immediate economic benefit that would warrant their use. For example:

• attacking devices of a computational scientist in a pharma/biotech company and stealing the structures of drugs being developed
• attacking devices of pharma/biotech employees and silently corrupting critical research data: raw measurements, analysis, experimental inputs

Why? Out of spite, out of competition, out of desire to railroad clinical trials and earn money by shorting the stock.

Most people never had to seriously entertain such outcomes, and if I were to guess, mostly because they didn’t think there were enough people in the world who had both the capability and intent. Well, the AI is changing the capability aspect rapidly.

And if you’re dubious if there are people who would be spiteful enough to do something like this, you probably forgot the case of NeurIPS 2024 award-winning paper written by an intern who, reportedly, “deliberately disrupted experiments, causing erroneous and irreproducible results”by modifying the cluster pytorch source code, changing seeds, randomly killing multi-node processes, and directly modifying model weights and was later sued by ByteDance (where he interned) for $1 million.

So yeah, it was never about lack of people who didn’t have the intent, it was only the question of whether they had the capability. And if they don’t have it widely available now, they will in the nearest future.

A reader non-affiliated with the entities working on AI progress might be tempted to object: well, I didn’t ask for AI models capable of cyber attacks, why are we allowing their development?

In the third voice, I argue that the way we, as a society, handle this challenge will determine whether we will ever be able to have any computational tool that could cure all disease.