[2025-06-03] Oracle VM VirtualBox의 RAC 환경에서 ASM 디스크 추가 및 asm_power_limit 설정 (19c)

■ [2025-06-03] Oracle VM VirtualBox의 RAC 환경에서 ASM 디스크 추가 및 asm_power_limit 설정 (19c)

 

[2025-06-03] Oracle VM VirtualBox의 RAC 환경에서 ASM 디스크 추가 및 asm_power_limit 설정 (19c)

 

[실습 환경]

 

OS : Oracle Linux Server 7.9 (Linux rdb01d 5.4.17-2102.201.3.el7uek.x86_64)
DB : Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production (Version 19.27.0.0.0)
RAC1 : ol7ora19r1(192.168.0.21) : ORA19R1
RAC2 : ol7ora19r2(192.168.0.22) : ORA19R2

 

0. CRS 내린 후 VM 종료

 

[root@ol7ora19r1][/root]$ crsctl stop crs
[root@ol7ora19r2][/root]$ crsctl stop crs

[root@ol7ora19r1][/root]$ poweroff
[root@ol7ora19r2][/root]$ poweroff

 

1. 공유 디스크 추가

 

Oracle VM Virtual Box -> 도구 -> 미디어 -> 만들기 -> VDI(VirtualBox 디스크 이미지)
-> 고정 크기 -> 경로 지정 (ex:"E:\vm\ol7ora19r\ol7ora19r_storage\ol7ora19r_DATA1_2.vdi") -> 40GB -> 디스크 속성을 "공유 가능"으로 변경

--> 여기 까지 완료 후 각각에 노드에(Oracle VM) 공유 디스크를 연결 해줌
--> 모든 Oracle VM의 전원을 off 시킨 상태에서 해야함

--> RAC1 : ol7ora19r1(192.168.0.21) : DRDB1 설정
VM 설정 -> 저장소 -> 컨트롤러:SATA -> ol7ora19r_DATA1_2.vdi 선택

 

--> RAC2 : ol7ora19r2(192.168.0.22) : DRDB2 설정
VM 설정 -> 저장소 -> 컨트롤러:SATA -> ol7ora19r_DATA1_2.vdi 선택

 

--> 모든 Oracle VM의 전원을 on 시킴

 

2. 공유 디스크 등록 (1번 노드에서만 수행)

 

2-1. ASMLIB 사용 시

[root@ol7ora19r1][/root]$ ls -l /dev/sd*
brw-rw----. 1 root disk 8,   0 Jun  5 16:12 /dev/sda
brw-rw----. 1 root disk 8,   1 Jun  5 16:12 /dev/sda1
brw-rw----. 1 root disk 8,   2 Jun  5 16:12 /dev/sda2
brw-rw----. 1 root disk 8,  16 Jun  5 16:12 /dev/sdb
brw-rw----. 1 root disk 8,  17 Jun  5 16:12 /dev/sdb1
brw-rw----. 1 root disk 8,  32 Jun  5 16:12 /dev/sdc
brw-rw----. 1 root disk 8,  33 Jun  5 16:12 /dev/sdc1
brw-rw----. 1 root disk 8,  48 Jun  5 16:12 /dev/sdd
brw-rw----. 1 root disk 8,  49 Jun  5 16:12 /dev/sdd1
brw-rw----. 1 root disk 8,  64 Jun  5 16:12 /dev/sde
brw-rw----. 1 root disk 8,  65 Jun  5 16:12 /dev/sde1
brw-rw----. 1 root disk 8,  80 Jun  5 16:12 /dev/sdf
brw-rw----. 1 root disk 8,  81 Jun  5 16:12 /dev/sdf1
brw-rw----. 1 root disk 8,  96 Jun  5 16:12 /dev/sdg
brw-rw----. 1 root disk 8,  97 Jun  5 16:12 /dev/sdg1
brw-rw----. 1 root disk 8, 112 Jun  5 16:12 /dev/sdh

 

[root@ol7ora19r1][/root]$ fdisk /dev/sdh
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x0df70104.

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1):
Partition number (1-4, default 1):
First sector (2048-83886079, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-83886079, default 83886079):
Using default value 83886079
Partition 1 of type Linux and of size 40 GiB is set

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

 

[root@ol7ora19r1][/root]$ ls -l /dev/sd*
brw-rw----. 1 root disk 8,   0 Jun  5 16:12 /dev/sda
brw-rw----. 1 root disk 8,   1 Jun  5 16:12 /dev/sda1
brw-rw----. 1 root disk 8,   2 Jun  5 16:12 /dev/sda2
brw-rw----. 1 root disk 8,  16 Jun  5 16:12 /dev/sdb
brw-rw----. 1 root disk 8,  17 Jun  5 16:12 /dev/sdb1
brw-rw----. 1 root disk 8,  32 Jun  5 16:12 /dev/sdc
brw-rw----. 1 root disk 8,  33 Jun  5 16:12 /dev/sdc1
brw-rw----. 1 root disk 8,  48 Jun  5 16:12 /dev/sdd
brw-rw----. 1 root disk 8,  49 Jun  5 16:12 /dev/sdd1
brw-rw----. 1 root disk 8,  64 Jun  5 16:12 /dev/sde
brw-rw----. 1 root disk 8,  65 Jun  5 16:12 /dev/sde1
brw-rw----. 1 root disk 8,  80 Jun  5 16:12 /dev/sdf
brw-rw----. 1 root disk 8,  81 Jun  5 16:12 /dev/sdf1
brw-rw----. 1 root disk 8,  96 Jun  5 16:12 /dev/sdg
brw-rw----. 1 root disk 8,  97 Jun  5 16:12 /dev/sdg1
brw-rw----. 1 root disk 8, 112 Jun  5 16:15 /dev/sdh
brw-rw----. 1 root disk 8, 113 Jun  5 16:15 /dev/sdh1

 

[root@ol7ora19r1][/root]$ oracleasm listdisks
ACFS1_1
CRS1
CRS2
CRS3
DATA1_1
FRA1_1

[root@ol7ora19r1][/root]$ oracleasm createdisk DATA1_2 /dev/sdh1
Writing disk header: done
Instantiating disk: done

[root@ol7ora19r1][/root]$ oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...

[root@ol7ora19r1][/root]$ oracleasm listdisks
ACFS1_1
CRS1
CRS2
CRS3
DATA1_1
DATA1_2
FRA1_1

--2번 노드에서도 확인
[root@ol7ora19r2][/root]$ oracleasm listdisks
ACFS1_1
CRS1
CRS2
CRS3
DATA1_1
DATA1_2
FRA1_1

 

[root@ol7ora19r2][/root]$ ls -l /dev/sd*
brw-rw----. 1 root disk 8,   0 Jun  5 16:21 /dev/sda
brw-rw----. 1 root disk 8,   1 Jun  5 16:21 /dev/sda1
brw-rw----. 1 root disk 8,   2 Jun  5 16:21 /dev/sda2
brw-rw----. 1 root disk 8,  16 Jun  5 16:21 /dev/sdb
brw-rw----. 1 root disk 8,  17 Jun  5 16:21 /dev/sdb1
brw-rw----. 1 root disk 8,  32 Jun  5 16:21 /dev/sdc
brw-rw----. 1 root disk 8,  33 Jun  5 16:21 /dev/sdc1
brw-rw----. 1 root disk 8,  48 Jun  5 16:21 /dev/sdd
brw-rw----. 1 root disk 8,  49 Jun  5 16:21 /dev/sdd1
brw-rw----. 1 root disk 8,  64 Jun  5 16:21 /dev/sde
brw-rw----. 1 root disk 8,  65 Jun  5 16:21 /dev/sde1
brw-rw----. 1 root disk 8,  80 Jun  5 16:21 /dev/sdf
brw-rw----. 1 root disk 8,  81 Jun  5 16:21 /dev/sdf1
brw-rw----. 1 root disk 8,  96 Jun  5 16:21 /dev/sdg
brw-rw----. 1 root disk 8,  97 Jun  5 16:21 /dev/sdg1
brw-rw----. 1 root disk 8, 112 Jun  5 16:21 /dev/sdh
brw-rw----. 1 root disk 8, 113 Jun  5 16:21 /dev/sdh1

--> 2번 노드는 자동으로 인식되는 걸 알 수 있음

2-2. udev 사용 시

 

--RAC 1번
[root@ol7ora19r1][/dev]$ /usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/sdh
1ATA_VBOX_HARDDISK_VBcb172146-14b52cda

 

[root@ol7ora19r1][/dev]$ udevadm info --query=all --name=/dev/sdh | grep ID_PATH | grep -v ID_PATH_TAG
E: ID_PATH=pci-0000:00:0d.0-ata-8.0

 

[root@ol7ora19r1][/dev]$ vi /etc/udev/rules.d/99-oracle-asmdevices.rules
[root@ol7ora19r1][/dev]$ cat /etc/udev/rules.d/99-oracle-asmdevices.rules | grep sdh
KERNEL=="sdh", ENV{ID_PATH}=="pci-0000:00:0d.0-ata-8.0", SYMLINK+="oracleasm/disks/DATA1_2", OWNER="grid", GROUP="asmadmin", MODE="0660"

--> 위의 한줄을 추가시킴

 

[root@ol7ora19r1][/dev]$ udevadm control --reload-rules
[root@ol7ora19r1][/dev]$ udevadm trigger

[root@ol7ora19r1][/dev]$ ls -l /dev/oracleasm/disks | grep sdh
lrwxrwxrwx. 1 root root 9 Jun  1 19:16 DATA1_2 -> ../../sdh

 

--RAC 2번
[root@ol7ora19r2][/root]$ /usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/sdh
1ATA_VBOX_HARDDISK_VBcb172146-14b52cda
[root@ol7ora19r2][/root]$ udevadm info --query=all --name=/dev/sdh | grep ID_PATH | grep -v ID_PATH_TAG
E: ID_PATH=pci-0000:00:0d.0-ata-8.0
[root@ol7ora19r2][/root]$ vi /etc/udev/rules.d/99-oracle-asmdevices.rules
[root@ol7ora19r2][/root]$ cat /etc/udev/rules.d/99-oracle-asmdevices.rules | grep sdh
KERNEL=="sdh", ENV{ID_PATH}=="pci-0000:00:0d.0-ata-8.0", SYMLINK+="oracleasm/disks/DATA1_2", OWNER="grid", GROUP="asmadmin", MODE="0660"

--> 위의 한줄을 추가시킴

[root@ol7ora19r2][/root]$ udevadm control --reload-rules
[root@ol7ora19r2][/root]$ udevadm trigger
[root@ol7ora19r2][/root]$ ls -l /dev/oracleasm/disks | grep sdh
lrwxrwxrwx. 1 root root 9 Jun  1 19:21 DATA1_2 -> ../../sdh

 

--RAC 1번
[root@ol7ora19r1][/dev]$ oracleasm listdisks
ACFS1_1
CRS1
CRS2
CRS3
DATA1_1
DATA1_2
FRA1_1

 

--RAC 2번
[root@ol7ora19r2][/root]$ oracleasm listdisks
ACFS1_1
CRS1
CRS2
CRS3
DATA1_1
DATA1_2
FRA1_1

3. ASM 디스크 추가

 

ASM_POWER_LIMIT는 ASM 인스턴스의 디스크 리밸런싱 속도를 지정함

제한값이 높을수록 리밸런싱 작업이 빨라지지만 CPU를 많이 소비하게 됨
값이 낮을수록 시간은 더 오래 걸리지만 처리 및 입출력 리소스는 더 적게 소모됨

 

디스크 그룹에서 디스크를 추가하거나 삭제할 때 리밸런싱이 이루어짐

백그라운드 프로세스 ARBx는 재조정 활동을 수행함(여기서 x는 숫자임).

 

리밸런싱 작업의 POWER 절이 지정되지 않은 경우 기본 파워는 ASM_POWER_LIMIT 값이 됨
asm_power_limit의 범위는 0에서 1024까지이고 기본값은 1임

 

--sa는 sqlplus "/as sysasm" --의 alias임

[+ASM1:grid@ol7ora19r1][/home/grid]$ sa

SQL*Plus: Release 19.0.0.0.0 - Production on Fri Jun 6 00:27:26 2025
Version 19.27.0.0.0

Copyright (c) 1982, 2024, Oracle.  All rights reserved.

Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.27.0.0.0

[2025-06-06:00:27:26][ol7ora19r1]<SYS@+ASM1> show parameter asm_power_limit;

NAME                                 TYPE                              VALUE
------------------------------------ --------------------------------- ------------------------------
asm_power_limit                      integer                           1

 

[2025-06-06:00:27:26][ol7ora19r1]<SYS@+ASM1> alter diskgroup DATA1 add disk '/dev/oracleasm/disks/DATA1_2';
--> 리밸런싱 파워는 1이됨(지정하지 않으면 asm_power_limit를 따름)

Diskgroup altered.

Elapsed: 00:00:04.95

 

--> 아래의 sql문으로 리밸런싱 모니터링 가능
SELECT INST_ID
     , OPERATION
     , STATE
     , POWER
     , SOFAR
     , EST_WORK
     , EST_RATE
     , EST_MINUTES
  FROM GV$ASM_OPERATION;

--> os에서 asm 리밸런싱 프로세스 확인 가능

ps -ef | grep asm | grep arb

alter diskgroup DATA1 rebalance power 4;
--> 이렇게 리밸런싱 파워를 변경 가능함