A security issue can make a DoS attack in the ASM.

We find a docker security issue to exhaust all struct files in Linux Kernel and causing a DoS attack in the Alibaba ASM environment.

Reproduction steps:

1. Follow the ASM tutorial to set up ASM clusters. We use two Virtual Machine with 16G memory, 120G ESSD Disk, CentOS 7.7 OS, Kubernetes Version V1.18.8-aliyun.1, and Docker Version 19.3.5, to set the Alibaba Kubernetes Cluster. All those settings are done through Alibaba ASM UI.
2. Deploy the docker unprivileged malicious container with UID 1000, dropping all capabilities, using limited memory 4G, running on special core, and disable privilege escalation. We run the malicious container in a separate Kubernetes Namespace.
3. Inside the malicious container, we repeatedly make the timerfd_create/open syscalls. In the malicious container, we start 2100 processes, each process with a 1024 max number of open files, which is the default ulimit value. In total, around 2097152 number struct-file are consumed and there is no available struct-file in the kernel. As a result, the victim container can't create any new files. The total consumed memory of the malicious container is less than 4G, which is small, so the memory control group cannot help.

Is there any way to defend against this attack inside the Alibaba ASM environment? Looking forward to your reply!