Picking up the remote in any orientation and brushing against any part of it during handling, without physically pushing a button, should never result in accidental input. Picking it up should feel safe.
I hate the new Apple TV remote even more than I hated the old one. I’m the sort of person who likes to hold the remote while watching shows. Not possible with Apple’s too-thin, too-sensitive remote. Marco’s right, it should feel safe. It doesn’t.