The four have resigned from Highland licensing committee after voting to allow David Brown to keep an operator's licence.