Stack Smashing is the technique of inserting payloads into the stack taking advantage of a stack overflow vulnerability. A number of methods to protect against this type of attack have been currently implemented in compilers and in operating systems. This post lists the defence mechanisms employed against stack smashing. They are:
- Stack Canaries
- Data Execution Prevention (DEP)/ Non Executable (NX)
- Address Space Layout Randomization
The main aim of the above methods is to prevent control flow hijacking and to prevent the knowledge of the layout of memory, making attacks such as return-to-libc hard to achieve, since the attacker will not be aware of the address at which the code resides.
Stack Canaries are nothing but random integers or characters placed around the return addresses in the call stack. The purpose of these random strings is to provide an provide an indication about whether the address has been modified. Since these canaries are random, the buffer overflow will in all possibility modify the canaries and hence the compiler will recognise the modification to the return address and stop execution effectively preventing the hijacking of control flow.
Data Execution Prevention and NX bit
The logic behind the Non Executable bit (known as eXecutable Disable (XD) bit in Intel architecture and eXecute Never (XN) in ARM) is a method of marking certain regions of the memory as non-executable i.e containing only data and no executable instructions. This general technique is called Executable Space Protection. The implementation of the same at the level of Operating Systems (software solution), specifically in Windows OS is called Data Execution Prevention. Note that the NX bit can still be accessed by programs using the VirtualAlloc and VirtualProtect Win32 calls.
Address Space Layout Randomization
Address Space Layout Randomization involves storing different parts of the address space such as the base of the executable, stack ,heap and libraries of a process at pseudo random locations so as to prevent reliable access by an attacker using a buffer overflow vulnerability. This technique is still not foolproof as it turns moot if there exists a dangling pointer vulnerability. More on this later in a separate post.