Hi,
It don't seem to be easily possible, this is the way it's working so to change that it will require core editions.
Created order don't mean that it is confirmed.
We create the order even if the payment is not done because this way it allow you to follow the orders, and we have an option to allow customers to pay their orders later via their user control panel.
If you want you can create a mass action which can delete the order with the created status if they have been created more than one hour ago or more.