Software Obfuscation Technique based on Recurrent Neural Network

Abstract

Protecting intellectual property against tampering is an urgent issue to many software designers, where illegal access to sensitive data is considered as a form of copyright infringement. Software owners apply various protection techniques in order to address this issue. Many of used mechanisms are weak, since they are vulnerable to both dynamic and static analysis, where the other are very costly since they impose considerable performance penalties. In this paper, we proposed a data and control flow obfuscating technique based on integrating encryption mechanism within recurrent neural network (RNN). Neural network provides a robust security characteristic in software protection, due to its ability of representing nonlinear algorithms with a powerful computational capability. The system is designed to enable the neural network generating of different encryptions for the same protected data. This creates a many to one relationship between the keys and the encryption. Thus, we train the neural network to simulate the conditional behaviors of the program in order to complicate the reverse engineering analysis of the software and hindering the “Concolic testing” attack. Consequently, we replace the critical points of program’s data and control flows with a semantically equivalent neural network. Our method is designed to enable the neural network to execute the conditional control transfers, where the complexity of neural network ensures that the protected behavior is turned into a complicated and incomprehensible form, making it impossible to extract its rules or locating the accurate inputs that lead to the execution paths behind the network. The protection presented by our mechanism is robust against both static and dynamic analysis. Furthermore, our evaluations confirm that employing the neural networks in our system significantly increase the difficulties in revealing the obfuscated software.