Turned out the remote was out of battery