Research Blog

Written by: Steven van der Baan In the ever-evolving landscape of cybersecurity, the integration of artificial intelligence (AI) has revolutionized various aspects of threat detection, prevention, and mitigation. Web penetration testing, a crucial component of ensuring the security posture of digital assets, has seen significant advancements through AI-powered tools. While AI undoubtedly offers numerous benefits […]

As we explained in a previous blogpost, exploiting a prompt injection attack is conceptually easy to understand: There are previous instructions in the prompt, and we include additional instructions within the user input, which is merged together with the legitimate instructions in a way that the underlying model cannot distinguish between them. Just like what […]

This executable blog post is the fourth in a series related to machine learning and is a fascinating trifecta involving hardened cryptography software, embedded IoT-type hardware, and deep machine learning techniques. While the AES algorithm is designed such that a brute-force secret key guessing attack would likely finish ‘sometime near eternity’, the power side-channel attack […]

In the following blog post, we explore how overfitting can affect Large Language Models (LLMs) in particular, since this technology is used in the most promising AI technologies we see today (chatGPT, LLaMa, Bard, etc). Furthermore, by exploring the likelihood of inferring data from the dataset, we will determine how much we can trust these […]

This executable blog post is the third in a series related to machine learning and explores code generation from a 16 billion parameter large language model (LLM). After a brief look under the hood at the LLM structure and parameter allocation, we generate a variety of Python functions and make observations related to code quality […]

TL;DR: Don’t use ChatGPT for security code review. It’s not meant to be used that way, it doesn’t really work (although you might be fooled into thinking it does), and there are some other major problems that make it impractical. Also, both the CEO of OpenAI and ChatGPT itself say that you shouldn’t. Large Language […]

This blog post is the second in a series related to machine learning, and demonstrates exactly how a data poisoning attack might work to insert a backdoor into a facial authentication system. The simplified system has similarities to that which the TSA is running a proof of concept trial at the Detroit and Atlanta airports. As background, […]

If you frequently deliver source code review assessments of products, including machine learning components, I’m sure you are used to reviewing Jupyter Notebook files (usually python). Although I spend most of my time reviewing the source code manually, I also use static analysis tools such as semgrep, using both public and private rules. This tool […]

Written by Jose Selvi and Thomas Atkinson If you are a Machine Learning (ML) enthusiast like us, you may recall our blogpost series from 2019 regarding Project Ava, which documented our experiments in using ML techniques to automate web application security testing tasks. In February 2020 we set out to build on Project Ava with […]

Professor Ron Rivest observed the close relationship between cryptography and machine learning at the ASIACRYPT conference back in 1991. Cross-fertilization of common notions, such as integrity, privacy, confidentiality and authenticity, have only grown in the following three decades as these fields have become more central to our everyday lives. This blog post is the first in […]