On the Poisoning of LLMs

Interesting essay on the poisoning of LLMs—ChatGPT in particular:

Given that we’ve known about model poisoning for years, and given the strong incentives the black-hat SEO crowd has to manipulate results, it’s entirely possible that bad actors have been poisoning ChatGPT for months. We don’t know because OpenAI doesn’t talk about their processes, how they validate the prompts they use for training, how they vet their training data set, or how they fine-tune ChatGPT. Their secrecy means we don’t know if ChatGPT has been safely managed.

They’ll also have to update their training data set at some point. They can’t leave their models stuck in 2021 forever…

May 25, 2023
Read More >>

Phishing campaign targets ChatGPT users

A clever phishing campaign aimed at stealing users’ business email account credentials by impersonating OpenAI, the company behind the ChatGPT chatbot, has been spotted by Inky researchers. The attack ChatGPT has quickly gained popularity and is …

May 25, 2023
Read More >>

How to avoid shadow AI in your SOC

Samsung’s recent discovery that employees had uploaded sensitive code to ChatGPT should serve as a reminder for security leaders to tread carefully when it comes to integrating new artificial intelligence tools throughout their organizations. Shadow AI…

May 24, 2023
Read More >>

6 ChatGPT risks for legal and compliance leaders

Legal and compliance leaders should address their organization’s exposure to six specific ChatGPT risks, and what guardrails to establish to ensure responsible enterprise use of generative AI tools, according to Gartner. “The output generated by ChatGP…

May 24, 2023
Read More >>

Credible Handwriting Machine

In case you don’t have enough to worry about, someone has built a credible handwriting machine:

This is still a work in progress, but the project seeks to solve one of the biggest problems with other homework machines, such as this one that I covered a few months ago after it blew up on social media. The problem with most homework machines is that they’re too perfect. Not only is their content output too well-written for most students, but they also have perfect grammar and punctuation ­ something even we professional writers fail to consistently achieve. Most importantly, the machine’s “handwriting” is too consistent. Humans always include small variations in their writing, no matter how honed their penmanship…

May 23, 2023
Read More >>