Avamar password recovery

I recently had an issue where, on a new Avamar install, the customer mis-typed the password twice. We had selected the option to make all passwords the same, so we were unable to log into the system at all. We booted the Utility node into Single User Mode by following the instructions here.

After we reset the OS account password we were able to SSH into the Avamar, but we were unable to successfully change all of the passwords. When prompted for the Avamar root account password in the change-passwords script the password we tried didn't work, it was asking us for the incorrectly typed password that set this whole thing in motion.

After SSHing in as admin, we were able to su to root. Then we decrypted the MC Database Tables and this gives us a handful of Avamar account passwords. Because we has chosen to make all passwords the same during the workflow install, we had our mistyped password. Re-encrypt the DB tables and rerun the change-passwords script and everything is ok!


admin@avamar:~/>: su
Password:
root@avamar:/home/admin/#: grep AP /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml
                <entry key="backuprestoreAP" value="{AES}9qRz5orPek4Dq7ybDzh/MA==" />
                <entry key="backuponlyAP" value="{AES}9qRz5orPek4Dq7ybDzh/MA==" />
                <entry key="rootAP" value="{AES}9qRz5orPek4Dq7ybDzh/MA==" />
                <entry key="MCUSERAP" value="{AES}9qRz5orPek4Dq7ybDzh/MA==" />
                <entry key="restoreonlyAP" value="{AES}9qRz5orPek4Dq7ybDzh/MA==" />
                <entry key="viewuserAP" value="{AES}9qRz5orPek4Dq7ybDzh/MA==" />
root@avamar:/home/admin/#: mccipher decrypt --all

**********************************************************************************
* EMC Avamar Management Console (MC).                                            *
* MC Security Tool for Secret Key generation, encryption, decryption and digest. *
**********************************************************************************

WARNING: MCS Preference looks like already decrypted: mcserver:/usr/local/avamar/lib:com/avamar/asn/rmi_ssl_keystore_ap
WARNING: Password is NULL or EMPTY: mcserver:/usr/local/avamar/lib:com/avamar/mc/cac/ldap_login_ap
WARNING: MCS Preference looks like already decrypted: mcserver:/usr/local/avamar/lib:com/avamar/mc/dpn/users/MCUSERAP
WARNING: MCS Preference looks like already decrypted: mcserver:/usr/local/avamar/lib:com/avamar/mc/dpn/users/backuponlyAP
WARNING: MCS Preference looks like already decrypted: mcserver:/usr/local/avamar/lib:com/avamar/mc/dpn/users/backuprestoreAP
WARNING: Password is NULL or EMPTY: mcserver:/usr/local/avamar/lib:com/avamar/mc/dpn/users/mcserver_keystore_ap
WARNING: Password is NULL or EMPTY: mcserver:/usr/local/avamar/lib:com/avamar/mc/dpn/users/mcserver_truststore_ap
WARNING: MCS Preference looks like already decrypted: mcserver:/usr/local/avamar/lib:com/avamar/mc/dpn/users/restoreonlyAP
WARNING: MCS Preference looks like already decrypted: mcserver:/usr/local/avamar/lib:com/avamar/mc/dpn/users/rootAP
WARNING: MCS Preference looks like already decrypted: mcserver:/usr/local/avamar/lib:com/avamar/mc/dpn/users/viewuserAP
WARNING: Password is NULL or EMPTY: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/cac/ldap_login_ap
WARNING: Password is NULL or EMPTY: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/dpn/users/mcserver_keystore_ap
WARNING: Password is NULL or EMPTY: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/dpn/users/mcserver_truststore_ap
WARNING: Can not decrypt some or all MCS Preferences.

All MC DB Tables have been decrypted successfully.

All MCCLI config files have been decrypted successfully.

See MCCipher log for details.

? 2012 EMC Corporation. All rights reserved.
root@avamar:/home/admin/#: grep AP /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml
                <entry key="backuprestoreAP" value="backuprestore1" />
                <entry key="backuponlyAP" value="backuponly1" />
                <entry key="rootAP" value="RootPassword!" />
                <entry key="MCUSERAP" value="MCUser1" />
                <entry key="restoreonlyAP" value="restoreonly1" />
                <entry key="viewuserAP" value="viewuser1" />
root@avamar:/home/admin/#: mccipher encrypt --all

**********************************************************************************
* EMC Avamar Management Console (MC).                                            *
* MC Security Tool for Secret Key generation, encryption, decryption and digest. *
**********************************************************************************

WARNING: Password is NULL or EMPTY: mcserver:/usr/local/avamar/lib:com/avamar/mc/cac/ldap_login_ap
WARNING: Password is NULL or EMPTY: mcserver:/usr/local/avamar/lib:com/avamar/mc/dpn/users/mcserver_keystore_ap
WARNING: Password is NULL or EMPTY: mcserver:/usr/local/avamar/lib:com/avamar/mc/dpn/users/mcserver_truststore_ap
WARNING: Password is NULL or EMPTY: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/cac/ldap_login_ap
WARNING: Password is NULL or EMPTY: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/dpn/users/mcserver_keystore_ap
WARNING: Password is NULL or EMPTY: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/dpn/users/mcserver_truststore_ap
WARNING: Can not encrypt some or all MCS Preferences.

All MC DB Tables have been encrypted successfully.

All MCCLI config files have been encrypted successfully.

See MCCipher log for details.

? 2012 EMC Corporation. All rights reserved.
root@avamar:/home/admin/#: grep AP /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml
                <entry key="backuprestoreAP" value="{AES}9qRz5orPek4Dq7ybDzh/MA==" />
                <entry key="backuponlyAP" value="{AES}9qRz5orPek4Dq7ybDzh/MA==" />
                <entry key="rootAP" value="{AES}9qRz5orPek4Dq7ybDzh/MA==" />
                <entry key="MCUSERAP" value="{AES}9qRz5orPek4Dq7ybDzh/MA==" />
                <entry key="restoreonlyAP" value="{AES}9qRz5orPek4Dq7ybDzh/MA==" />
                <entry key="viewuserAP" value="{AES}9qRz5orPek4Dq7ybDzh/MA==" />
root@avamar:/home/admin/#: