crypto: simplify state failure handling

It is more intuitive to return true/false than undefined/false. This also simplifies handling in the JS layer and removes two control-flow branches from C++.

Checklist

• make -j4 test (UNIX), or vcbuild test (Windows) passes
• commit message follows commit guidelines