I had some trouble tracking down in the SmartOS wiki on how to change the root password. I figured it out, and here’s what you need to do:
1. Login to your SmartOS server as root
2. Type in "grep root /usbkey/shadow" and place the output somewhere safe in case you need to revert this change.
3. Type in "/usr/lib/cryptpass YOUR_NEW_PASSWORD" and copy the output. This is your encrypted new root password.
4. Modify /usbkey/shadow (vi /usbkey/shadow) replacing your previous encrypted password with the new one.
5. Save the file (ESC, wq!)
6. Open a new SSH session (Without closing the old one, just in case you messed this up) and test your new password. It should work!