Malicious model to RCE by vocab file load in TransfoXLTokenizer (as well as the code reuse to evade the pickle scanning) in huggingface/transformers
Sep 9th 2023
The huggingface/transformers implements
TransfoXLTokenizer to automatically load vocab.pkl file from the remote repo using the risky
pickle.load without any restrictions, which make the victim users vulerable to remote code execution attacks by simply running the demo code
AutoTokenizer.from_pretrained("the repo deployed with malicious vocab.pkl file"). We understand the huggingface implements a
pickle scanning to flag the
unsafe file alert if the
os.system and many other black-listed modules imported in the
vocab.pkl. But we can bypass this
unsafe alert in our attacking repo by launching the real attack in another repo. Saying we have two repos A and B. In the repo A, we just deploy the malicious vocab.pkl with the execution of
AutoTokenizer.from_pretrained("B"), and in the repo B, we deploy the malicious
vocab.pkl with the OS commands we try to execute. By this way, the repo A looks bengin since its
vocab.pkl do not contain any black-listed imports, but the victim users who fetch the pretrained
TransfoXLTokenizer models from repo A will be forced to execute the malicious command deployed in repo B. It is worth noting that, the victim users cannot find any tricks for the existent of the repo B in the repo A in huggingface websites.
Proof of Concept
We deploy a PoC repo in https://huggingface.co/zpbrent/test, by which you can run the following code to fetch the pretrained
TransfoXLTokenizer in your local machine:
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("zpbrent/test")
and then you can find a
HACKED filed has been created in your machine, which means our malicious OS command
touch HACKED has been successfully executed.
Note that, in our current PoC demo, you may see the
ValueError: Unknown format at the end of the code execution (despite this cannot prevent the execution of our malicious commands), this is because we have not embedded our malicious payloads in some normal vocab.pkl. To hide such errors, a possible way is to embed our attacking payload into some typical vocab.pkl for TransfoXL such as from https://huggingface.co/transfo-xl-wt103 and so on.
This vulnerability can be exploited to execute arbitrary codes and OS commands in the victim machine who fetch the pretrained repo remotely, and obviously leading to disatrous consequences depending on which commands the malicious model includes.