Hi,
Yes. You can indeed do that with a product mass action. You can use a filter on the category, and an action "update the values" on the "product_tax_id" column that you can set with the "int" value "0". Once you click on "process" it will do its work and set the product_tax_id to 0 for all the products of that category, effectively removing the tax category from the product.
Then, you need to make sure that you don't have tax rules without any tax category setup as otherwise, the taxes of these tax rules will apply to products without a tax category.