Normal File deletion in Windows (Shift + Delete) deletes only its metadata. Actually, the files hides on your hard disk somewhere and Windows only removes the entry from the FAT (file allocation table) so that the system does not see the file. These files can be found and recovered again as long as they haven't been overwritten by new data yet. As such the entire file can be recovered with the use of special data recovery softwares. To avoid privacy and security issues resulting from insecure file deletion, you need a software that will delete every trace of the existing file. Kakasoft Super File Shredder is such a software. It is distributed as Free under GNU public license. The software can quickly shred files by overwriting random bits, 15 times using advanced algorithms, making recovery literally impossible. According to the website:
Kakasoft Super File Shredder is simple but powerful program that surpasses manu commercial file shredders out there. The Author believes that such a utility should be available to anyone for free and that permanent and safe removal of confidential documents is a matter of basic right to privacy.