Maximum selected checkbox in custom field

Hi,

There is no setting to have such restriction.
You would have to add an onclick attribute to the custom fields with a call to a custom javascript function you would add and which would check whether there are not already three other checkboxes checked.
Based on the custom field table you're using, the view file where to do that customization will be different so I can't tell you what file should it be as you didn't provide that information.
But in any case, it requires programming skills to be able to do that.