There is a library that implements a replacement malloc library that helps developers find buffer overflows.
Sometimes buffer overflows dont SEGV. A buffer overflow only causes a segmentation violation, when it tries to access memory that it doesnt have access too. The heap occupies a large space of memory that houses any number of buffers. A buffer could overflow into another buffer on the heap. But the overflow wouldn’t go past the end of the heap, where it doesnt have access to.
Electric fence uses a stategy of placing guard pages at the end of buffers it allocates. Memory is dived into a series of pages, each page having its own acces s rights. A memory allocation in electric fence is placed so the end of the buffer, is directly at a page boundery. The next page is marked as non readable or writeable. If the buffer overflows, it hits this guard page. The result is a segmentation violation.
Electric fence can also be used to find buffer underuns, where a guard page is placed before the buffer.
Running your software in a debugger, or debugging the core dump will then establish where the buffer overflow occured in source.
The disadvantage of electric fence, is that it uses alot more memory than normal. It allocates between 1 or 2 extra pages for every malloc. And also rounds up all allocations to a page boundary.
In 2001, a co-worker was using electric fence for the first time on the software we were developing, and discovered that early Linux kernels would not at kernel compile time allocate enough vm_area_struct’s for use at runtime. The result was that electric fence wasnt able to allocate enough memory, even though it had available physical ram and a large enough swap. This bug was fixed shortly after.