Remove Comparison Button in Shop

I might be missing something. However, is there any way to remove the comparison button in the shop and keep the comparison page linked in my menu?

You need to edit the item bow layout of your products listing (by default it's listing_img_title) for the product view via the menu Display>Views and remove the comparison area.