Hi,
When you upload an image for a product or a category, if an image with the same name is already in the upload folder, the system checks whether it's the same file or not. If it's the same file, it directly uses it. If not, it adds that number at the end of the file name of the new image file so that it can be saved in the upload folder nex to the other file.
Even if you delete the previous image afterwards, it doesn't change the fact that when you uploaded the image, there was already an image with that same name so you still get left with an image with a number at the end.