Can we set a tax on some products and not others food within the same state?

Yes.
That's what the "tax category" setting is for in each product.
In the menu System>Taxes, you have a "manage tax categories" button you can use to create tax categories.
Then, in the menu System>Taxes, you can create different tax rules for different tax categories.
And then in your products, you can select different tax categories.
That way, each product will use the tax rules of its tax category.

Hi,

Yes.
You need to configure 2 tax categories, one for taxable items and one for non taxable items.
Then, in each product select the corresponding tax category.
Then, you need to create 3 tax rates: 6%, 7% and the sales tax rate for MD
And finally, you need to create 3 tax rules in System>Taxes:
- one for MD zone without any tax category selected and with the sales tax rate for MD
- one for PA zone for the taxable products tax category with the 6% tax rate
- one for NJ zone for the taxable products tax category with the 7% tax rate
You don't need a tax rule for DE. If there is no tax rules matching, no tax rate will apply to the products.