Online migration of ASM disk groups from one SAN to another
Below mentioned are the detailed steps from Database Admin perspective for migrating ASM disks from one SAN to another[from EMC to HP] in a RAC setup.
1. Capture the current ASM disk/diskgroups details as below. Also decide upon the new LUN and ASM disk names. Share these with OS Admins, so that they can create same size LUN's with new SAN.
To get ASM disk, group and size details use the below queries.
select group_number,disk_number,name,mount_status,state,total_mb,free_mb From v$asm_disk order by 1,2;
select GROUP_NUMBER,NAME from v$asm_diskgroup;
To get the LUN-ASM disk mapping details use the below command.
/usr/sbin/oracleasm querydisk -p /dev/mapper/*
This command needs to be run with root privilege and in all the RAC instances just to be sure that you see the same.
ASM Disk Group Existing ASM Disk Existing LUN ASM disk Size in GB New LUN New Disk Name
ASMVOTE01 OCR_VOTE1 OCR_VOTE1p1 1 N_OCR_VOTE1p1 N_OCR_VOTE1
OCR_VOTE2 OCR_VOTE2p1 1 N_OCR_VOTE2p1 N_OCR_VOTE2
OCR_VOTE3 OCR_VOTE3p1 1 N_OCR_VOTE3p1 N_OCR_VOTE3
OCR_VOTE4 OCR_VOTE4p1 1 N_OCR_VOTE4p1 N_OCR_VOTE4
OCR_VOTE5 OCR_VOTE5p1 1 N_OCR_VOTE5p1 N_OCR_VOTE5
ASMDATA01 ASMDATA01 ASM1p1 100 N_ASM1p1 N_ASMDATA01
ASMDATA02 ASM2p1 100 N_ASM2p1 N_ASMDATA02
ASMDATA03 ASM3p1 100 N_ASM3p1 N_ASMDATA03
ASMDATA04 ASM4p1 100 N_ASM4p1 N_ASMDATA04
ASMFRA01 ASMDATA05 ASM5p1 100 N_ASM5p1 N_ASMDATA05
ASMDATA06 ASM6p1 100 N_ASM6p1 N_ASMDATA06
ASMDATA07 ASM7p1 100 N_ASM7p1 N_ASMDATA07
ASMDATA08 ASM8p1 100 N_ASM8p1 N_ASMDATA08
ASMDATA09 ASM9p1 100 N_ASM9p1 N_ASMDATA09
ASMDATA10 ASM10p1 100 N_ASM10p1 N_ASMDATA10
2. Take full database backup.
3. Both old and new SAN LUNs must be presented to all the RAC nodes.
Confirm whether you see the new LUNs under /dev/mapper/ from all the cluster nodes.
4. Once the LUNs are visible we need to stamp them to use them as ASM disks.
Below commands needs to be run with root privilege. This commands should be run only once from anyone of the cluster node.
sudo /usr/sbin/oracleasm createdisk N_OCR_VOTE1 '/dev/mapper/N_OCR_VOTE1p1'
sudo /usr/sbin/oracleasm createdisk N_OCR_VOTE2 '/dev/mapper/N_OCR_VOTE2p1'
sudo /usr/sbin/oracleasm createdisk N_OCR_VOTE3 '/dev/mapper/N_OCR_VOTE3p1'
sudo /usr/sbin/oracleasm createdisk N_OCR_VOTE4 '/dev/mapper/N_OCR_VOTE4p1'
sudo /usr/sbin/oracleasm createdisk N_OCR_VOTE5 '/dev/mapper/N_OCR_VOTE5p1'
sudo /usr/sbin/oracleasm createdisk N_ASMDATA1 '/dev/mapper/N_ASM1p1'
sudo /usr/sbin/oracleasm createdisk N_ASMDATA2 '/dev/mapper/N_ASM2p1'
sudo /usr/sbin/oracleasm createdisk N_ASMDATA3 '/dev/mapper/N_ASM3p1'
sudo /usr/sbin/oracleasm createdisk N_ASMDATA4 '/dev/mapper/N_ASM4p1'
sudo /usr/sbin/oracleasm createdisk N_ASMDATA5 '/dev/mapper/N_ASM5p1'
sudo /usr/sbin/oracleasm createdisk N_ASMDATA6 '/dev/mapper/N_ASM6p1'
sudo /usr/sbin/oracleasm createdisk N_ASMDATA7 '/dev/mapper/N_ASM7p1'
sudo /usr/sbin/oracleasm createdisk N_ASMDATA8 '/dev/mapper/N_ASM8p1'
sudo /usr/sbin/oracleasm createdisk N_ASMDATA9 '/dev/mapper/N_ASM9p1'
sudo /usr/sbin/oracleasm createdisk N_ASMDATA10 '/dev/mapper/N_ASM10p1'
5. Now scan and list the newly added ASM disks on other nodes.
Again this needs to be run with root privilege.
sudo /usr/sbin/oracleasm scandisks
sudo /usr/sbin/oracleasm listdisks
You should be able to see both old and newly added ASM disks from all the nodes.
6. Login to any ASM instance and verify the newly added ASM disks.Make sure you login as sysasm to avoid any privilege issues while logging into ASM instances.
select group_number,disk_number,name,mount_status,state,total_mb,free_mb From v$asm_disk order by 1,2;
7. Rebalancing
a. Add the newly created ASM disks to its mapping ASM diskgroups, which was decided in step 1.
Once the ALTER DISKGROUP is run rebalancing operation will begin in background and the prompt will be returned. Rebalancing of one diskgroup is not dependent on other, so below commands can be run at once.
SQL> alter diskgroup ASMDATA01
add disk 'ORCL:N_ASMDATA1','ORCL:N_ASMDATA2','ORCL:N_ASMDATA3','ORCL:N_ASMDATA4'
rebalance power 11;
SQL> alter diskgroup ASMFRA01
add disk 'ORCL:N_ASMDATA5','ORCL:N_ASMDATA6',
'ORCL:N_ASMDATA7','ORCL:N_ASMDATA8','ORCL:N_ASMDATA9','ORCL:N_ASMDATA10'
rebalance power 11;
SQL> alter diskgroup ASMVOTE01
add disk
'ORCL:N_OCR_VOTE1','ORCL:N_OCR_VOTE2','ORCL:N_OCR_VOTE3' ,'ORCL:N_OCR_VOTE4',
'ORCL:N_OCR_VOTE5' rebalance power 11;
b. Drop the old ASM disks from its corresponding ASM diskgroups.
SQL> alter diskgroup ASMDATA01
drop disk 'ORCL:ASMDATA1','ORCL:ASMDATA2','ORCL:ASMDATA3','ORCL:ASMDATA4'
rebalance power 11;
SQL> alter diskgroup ASMFRA01
drop disk 'ORCL:ASMDATA5','ORCL:ASMDATA6',
'ORCL:ASMDATA7','ORCL:ASMDATA8','ORCL:ASMDATA9','ORCL:ASMDATA10'
rebalance power 11;
SQL> alter diskgroup ASMVOTE01
drop disk
'ORCL:OCR_VOTE1','ORCL:OCR_VOTE2',
'ORCL:OCR_VOTE3' ,'ORCL:OCR_VOTE4','ORCL:OCR_VOTE5' rebalance power 11;
Both steps a and b can be performed in a single command and oracle recommends the same to save time. But to be on the safer side its better to perform them separately.
8. Monitor Rebalancing operation
Login to any ASM instance and monitor rebalancing operation using the below queries.
select * from v$asm_operation;
select * from gv$asm_operation;
9. Verify the diskgroups for the last time.
select group_number,disk_number,mount_status,state,redundancy from v$asm_disk;
select group_number,name,state,type from v$asm_diskgroup;
select group_number,disk_number,name,mount_status,state,total_mb,free_mb from v$asm_disk order by 1,2;
This successfully completes the migration.
10. Once above steps are complete, the old SAN can be dismounted.
Below mentioned are the detailed steps from Database Admin perspective for migrating ASM disks from one SAN to another[from EMC to HP] in a RAC setup.
1. Capture the current ASM disk/diskgroups details as below. Also decide upon the new LUN and ASM disk names. Share these with OS Admins, so that they can create same size LUN's with new SAN.
To get ASM disk, group and size details use the below queries.
select group_number,disk_number,name,mount_status,state,total_mb,free_mb From v$asm_disk order by 1,2;
select GROUP_NUMBER,NAME from v$asm_diskgroup;
To get the LUN-ASM disk mapping details use the below command.
/usr/sbin/oracleasm querydisk -p /dev/mapper/*
This command needs to be run with root privilege and in all the RAC instances just to be sure that you see the same.
ASM Disk Group Existing ASM Disk Existing LUN ASM disk Size in GB New LUN New Disk Name
ASMVOTE01 OCR_VOTE1 OCR_VOTE1p1 1 N_OCR_VOTE1p1 N_OCR_VOTE1
OCR_VOTE2 OCR_VOTE2p1 1 N_OCR_VOTE2p1 N_OCR_VOTE2
OCR_VOTE3 OCR_VOTE3p1 1 N_OCR_VOTE3p1 N_OCR_VOTE3
OCR_VOTE4 OCR_VOTE4p1 1 N_OCR_VOTE4p1 N_OCR_VOTE4
OCR_VOTE5 OCR_VOTE5p1 1 N_OCR_VOTE5p1 N_OCR_VOTE5
ASMDATA01 ASMDATA01 ASM1p1 100 N_ASM1p1 N_ASMDATA01
ASMDATA02 ASM2p1 100 N_ASM2p1 N_ASMDATA02
ASMDATA03 ASM3p1 100 N_ASM3p1 N_ASMDATA03
ASMDATA04 ASM4p1 100 N_ASM4p1 N_ASMDATA04
ASMFRA01 ASMDATA05 ASM5p1 100 N_ASM5p1 N_ASMDATA05
ASMDATA06 ASM6p1 100 N_ASM6p1 N_ASMDATA06
ASMDATA07 ASM7p1 100 N_ASM7p1 N_ASMDATA07
ASMDATA08 ASM8p1 100 N_ASM8p1 N_ASMDATA08
ASMDATA09 ASM9p1 100 N_ASM9p1 N_ASMDATA09
ASMDATA10 ASM10p1 100 N_ASM10p1 N_ASMDATA10
2. Take full database backup.
3. Both old and new SAN LUNs must be presented to all the RAC nodes.
Confirm whether you see the new LUNs under /dev/mapper/ from all the cluster nodes.
4. Once the LUNs are visible we need to stamp them to use them as ASM disks.
Below commands needs to be run with root privilege. This commands should be run only once from anyone of the cluster node.
sudo /usr/sbin/oracleasm createdisk N_OCR_VOTE1 '/dev/mapper/N_OCR_VOTE1p1'
sudo /usr/sbin/oracleasm createdisk N_OCR_VOTE2 '/dev/mapper/N_OCR_VOTE2p1'
sudo /usr/sbin/oracleasm createdisk N_OCR_VOTE3 '/dev/mapper/N_OCR_VOTE3p1'
sudo /usr/sbin/oracleasm createdisk N_OCR_VOTE4 '/dev/mapper/N_OCR_VOTE4p1'
sudo /usr/sbin/oracleasm createdisk N_OCR_VOTE5 '/dev/mapper/N_OCR_VOTE5p1'
sudo /usr/sbin/oracleasm createdisk N_ASMDATA1 '/dev/mapper/N_ASM1p1'
sudo /usr/sbin/oracleasm createdisk N_ASMDATA2 '/dev/mapper/N_ASM2p1'
sudo /usr/sbin/oracleasm createdisk N_ASMDATA3 '/dev/mapper/N_ASM3p1'
sudo /usr/sbin/oracleasm createdisk N_ASMDATA4 '/dev/mapper/N_ASM4p1'
sudo /usr/sbin/oracleasm createdisk N_ASMDATA5 '/dev/mapper/N_ASM5p1'
sudo /usr/sbin/oracleasm createdisk N_ASMDATA6 '/dev/mapper/N_ASM6p1'
sudo /usr/sbin/oracleasm createdisk N_ASMDATA7 '/dev/mapper/N_ASM7p1'
sudo /usr/sbin/oracleasm createdisk N_ASMDATA8 '/dev/mapper/N_ASM8p1'
sudo /usr/sbin/oracleasm createdisk N_ASMDATA9 '/dev/mapper/N_ASM9p1'
sudo /usr/sbin/oracleasm createdisk N_ASMDATA10 '/dev/mapper/N_ASM10p1'
5. Now scan and list the newly added ASM disks on other nodes.
Again this needs to be run with root privilege.
sudo /usr/sbin/oracleasm scandisks
sudo /usr/sbin/oracleasm listdisks
You should be able to see both old and newly added ASM disks from all the nodes.
6. Login to any ASM instance and verify the newly added ASM disks.Make sure you login as sysasm to avoid any privilege issues while logging into ASM instances.
select group_number,disk_number,name,mount_status,state,total_mb,free_mb From v$asm_disk order by 1,2;
7. Rebalancing
a. Add the newly created ASM disks to its mapping ASM diskgroups, which was decided in step 1.
Once the ALTER DISKGROUP is run rebalancing operation will begin in background and the prompt will be returned. Rebalancing of one diskgroup is not dependent on other, so below commands can be run at once.
SQL> alter diskgroup ASMDATA01
add disk 'ORCL:N_ASMDATA1','ORCL:N_ASMDATA2','ORCL:N_ASMDATA3','ORCL:N_ASMDATA4'
rebalance power 11;
SQL> alter diskgroup ASMFRA01
add disk 'ORCL:N_ASMDATA5','ORCL:N_ASMDATA6',
'ORCL:N_ASMDATA7','ORCL:N_ASMDATA8','ORCL:N_ASMDATA9','ORCL:N_ASMDATA10'
rebalance power 11;
SQL> alter diskgroup ASMVOTE01
add disk
'ORCL:N_OCR_VOTE1','ORCL:N_OCR_VOTE2','ORCL:N_OCR_VOTE3' ,'ORCL:N_OCR_VOTE4',
'ORCL:N_OCR_VOTE5' rebalance power 11;
b. Drop the old ASM disks from its corresponding ASM diskgroups.
SQL> alter diskgroup ASMDATA01
drop disk 'ORCL:ASMDATA1','ORCL:ASMDATA2','ORCL:ASMDATA3','ORCL:ASMDATA4'
rebalance power 11;
SQL> alter diskgroup ASMFRA01
drop disk 'ORCL:ASMDATA5','ORCL:ASMDATA6',
'ORCL:ASMDATA7','ORCL:ASMDATA8','ORCL:ASMDATA9','ORCL:ASMDATA10'
rebalance power 11;
SQL> alter diskgroup ASMVOTE01
drop disk
'ORCL:OCR_VOTE1','ORCL:OCR_VOTE2',
'ORCL:OCR_VOTE3' ,'ORCL:OCR_VOTE4','ORCL:OCR_VOTE5' rebalance power 11;
Both steps a and b can be performed in a single command and oracle recommends the same to save time. But to be on the safer side its better to perform them separately.
8. Monitor Rebalancing operation
Login to any ASM instance and monitor rebalancing operation using the below queries.
select * from v$asm_operation;
select * from gv$asm_operation;
9. Verify the diskgroups for the last time.
select group_number,disk_number,mount_status,state,redundancy from v$asm_disk;
select group_number,name,state,type from v$asm_diskgroup;
select group_number,disk_number,name,mount_status,state,total_mb,free_mb from v$asm_disk order by 1,2;
This successfully completes the migration.
10. Once above steps are complete, the old SAN can be dismounted.
