For people who want to delete files immediately and without sending them to the Recycle Bin in Windows can find this article useful. Moreover, some additional options can be configured, such as the amount of space reserved for the Recycle Bin and whether the user wants to be prompted to confirm deletion of the files.
To constantly delete files from the PC, without moving to the Recycle Bin, perform next steps:
1. Right-click on Recycle Bin icon and choose 'Properties':

2. Check the following check-box and press O.k. to apply changes:

Uncheck 'Display delete confirmation dialog' in order not to be prompted each time when deleting files.
Note: the next key combination can also be used to delete files without moving to the recycle bin: <SHIFT>+<DELETE> simultaneously.