Dell EMC Data Protection patches for Spectre/Meltdown

January 24, 2018 by Abe Covello in Avamar, Data Domain, Security

Dell/EMC has not released any patches for the DPS products yet, but I bet they will be coming soon. After all, Avamar and Data Domain are running on Dell server hardware these days and running Intel processors.

Here is the official product matrix for Dell EMC products, including Avamar, Networker, and some Data Domain products (notably absent is Data Domain physical hardware). The link requires a Dell EMC Support login.

https://emcservice.force.com/CustomersPartners/kA6f1000000FD0gCAG

The important parts are:

Restart Data Domain GUI

September 26, 2017 by Abe Covello in Data Domain

Need to restart the Data Domain GUI? Here's how to restart the services:

Log in to the DD via the CLI as sysadmin, and enter SE privilege mode. The system password is the Data Domain serial number.

# system show serialno

# priv set se

# Enter system password:

Disable http and https services on the Data Domain to bring down the GUI process:

# adminaccess disable http
# adminaccess disable https

Enable http and https services on the Data Domain to re start the GUI process:

# adminaccess enable http
# adminaccess enable https

Wait for a couple of minutes to allow services to fully initialize

Making free space on a Full Data Domain

July 03, 2017 by Abe Covello in Data Domain, Avamar

Have a Data Domain that's full? Need to free up space so your backups continue to run? Check for old snapshots, expire them, then free up space with a file system cleaning.

Had a customer who's replication Data Domain was significantly (20% more!) more utilized than the primary site, and also was completely full. Which was causing replications to fail. When I checked out the snapshots on one of the Mtrees, I saw that there were 10 snapshots from almost a year ago. These snapshots together were using around 16Tb of the available 67Tb of space on the Data Domain.

SSH into the Data Domain as sysadmin. Use the mtree list command to show the Data Domain Mtree structure.

sysadmin@DataDomain# mtree list
Name Pre-Comp (GiB) Status
---------------------------- -------------- ------
/data/col1/avamar-13896542221487697.5 RW
/data/col1/avamar-1477523534 561707.6 RW
/data/col1/backup 0.0 RW
---------------------------- -------------- ------

Checking the smaller of the Mtrees shows what we expect to see

sysadmin@DataDom1# snapshot list mtree /data/col1/avamar-1477523534
Snapshot Information for MTree: /data/col1/avamar-1477523534
----------------------------------------------
NamePre-Comp (GiB) Create Date Retain UntilStatus
----------------- -------------- ----------------- ----------------- -------
cp.20170622161010 637792.0 Jun 22 2017 09:11 Jun 28 2017 08:50 expired
cp.20170622164953 637792.0 Jun 22 2017 09:51 Jun 28 2017 08:51 expired
cp.20170628151023 656374.0 Jun 28 2017 08:11
cp.20170628155016 661756.7 Jun 28 2017 08:51
----------------- -------------- ----------------- ----------------- -------
 
Snapshot Summary
-------------------
Total:4
Not expired:2
Expired:2

However, when we take a look at the larger of the Mtrees, we see where the problem lies.

sysadmin@DataDom1# snapshot list mtree /data/col1/avamar-1389654222
Snapshot Information for MTree: /data/col1/avamar-1389654222
----------------------------------------------
NamePre-Comp (GiB) Create Date Retain Until Status
----------------- -------------- ----------------- ------------ ------
cp.201609101621301076444.1 Sep 10 2016 09:21
cp.201609111532331100654.4 Sep 11 2016 08:32
cp.201609131705211161823.8 Sep 13 2016 10:05
cp.201609171657211257219.9 Sep 17 2016 09:57
cp.201609181543091284386.1 Sep 18 2016 08:43
cp.201609191705561309786.6 Sep 19 2016 10:05
cp.201609201542051339382.0 Sep 20 2016 08:42
cp.201609211541431359362.4 Sep 21 2016 08:41
cp.201609221542311377744.1 Sep 22 2016 08:42
cp.201609281624221432382.5 Sep 28 2016 09:24
----------------- -------------- ----------------- ------------ ------
 
Snapshot Summary
-------------------
Total: 10
Not expired: 10
Expired:0

Ten snapshots from almost a year ago, taking up so much space is bad. So lets manually expire each snapshot.

sysadmin@DataDom1# snapshot expire cp.20160910162130 mtree /data/col1/avamar-1389654222
Snapshot "cp.20160910162130" for mtree "/data/col1/avamar-1389654222" will be retained until Jul3 2017 08:16.

Do this nine more times to mark all of the old snapshots for deletion.

Checking the snapshots now shows them all expired.

sysadmin@DataDom1# snapshot list mtree /data/col1/avamar-1389654222
Snapshot Information for MTree: /data/col1/avamar-1389654222
----------------------------------------------
NamePre-Comp (GiB) Create Date Retain UntilStatus
----------------- -------------- ----------------- ----------------- -------
cp.201609101621301076444.1 Sep 10 2016 09:21 Jul3 2017 08:16 expired
cp.201609111532331100654.4 Sep 11 2016 08:32 Jul3 2017 08:17 expired
cp.201609131705211161823.8 Sep 13 2016 10:05 Jul3 2017 08:17 expired
cp.201609171657211257219.9 Sep 17 2016 09:57 Jul3 2017 08:17 expired
cp.201609181543091284386.1 Sep 18 2016 08:43 Jul3 2017 08:18 expired
cp.201609191705561309786.6 Sep 19 2016 10:05 Jul3 2017 08:18 expired
cp.201609201542051339382.0 Sep 20 2016 08:42 Jul3 2017 08:18 expired
cp.201609211541431359362.4 Sep 21 2016 08:41 Jul3 2017 08:19 expired
cp.201609221542311377744.1 Sep 22 2016 08:42 Jul3 2017 08:19 expired
cp.201609281624221432382.5 Sep 28 2016 09:24 Jul3 2017 08:19 expired
----------------- -------------- ----------------- ----------------- -------
 
Snapshot Summary
-------------------
Total: 10
Not expired:0
Expired: 10

And now we manually kick off the file system cleaning.

sysadmin@DataDom1# filesys clean start nowait
Cleaning started.Use 'filesys clean watch' to monitor progress.

When the cleaning is finished we should be good to go. In this case we cleared the Avamar logs, kicked off an Avamar checkpoint, and resumed the scheduler.

Data Domain - reset sysadmin password

Xiologix

March 10, 2016 by Abe Covello in Data Domain

I believe that the shell-escape command will no longer work as of DDOS 6.1.x without a encrypted password from Dell/EMC that is changed often (daily?). Sorry folks. :(

Every now and then I run into a situation where a customer has forgotten the sysadmin password to their Data Domain. You can't reset the sysadmin password from the CLI or GUI unless you are logged into the sysadmin account, here's how to do it from bash (it is after all just a Linux box), as long as you have another account on the system with admin access.

Note: You will be putting your Data Domain in a situation where you can, with the wrong command, completely destroy your data. If you do so, I'm not responsible, EMC isn't responsible, you're responsible. So be responsible, and be careful. Also, this is all covered in the Data Domain documentation. So I'm not sharing some huge secret here. Mostly just archiving it so I have a convenient place to go back to the next time I have to do this (which is probably in another month or so). Finally, if you don't have another account on the Data Domain apart from the sysadmin, this won't work (as it requires SSHing into the Data Domain) and you'll have to call support.

That being said, here we go.

First, SSH into the Data Domain as a user with admin credentials. Then:

system show serialno
priv set se
(Password is SN)
ctrl-c
ctrl-c
ctrl-c
uname
fi st
df
ctrl-c
ctrl-c
ctrl-c
shell-escape
passwd sysadmin
type in your new password twice, then
exit

That's all there is to it. Have fun, and try not to break anything.