Hi,
The right approach is to have two prices in the product: The price for the EU and the price for the rest of the world.
You can use the "zone" setting of the prices to restrict them to the correct zone.
That way, when the customer is in the EU, they will get one price and the tax will be removed from it based on the customer type, its country, etc. And when the customer is not in the EU, they will get the other price. And with no tax rule configured for countries outside the EU, no tax will apply.