[2025-06-03] Oracle VM VirtualBox의 RAC 환경에서 ACFS 구성 (19c)

kyoungoh.lee 2025. 7. 26. 09:52

2025. 7. 26. 09:52

■ [2025-06-03] Oracle VM VirtualBox의 RAC 환경에서 ACFS 구성 (19c)

[실습 환경]

OS : Oracle Linux Server 7.9 (Linux rdb01d 5.4.17-2102.201.3.el7uek.x86_64)
DB : Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production (Version 19.27.0.0.0)
RAC1 : ol7ora19r1(192.168.0.21) : ORA19R1
RAC2 : ol7ora19r2(192.168.0.22) : ORA19R2

1. 공유 디스크 추가

<Oracle VM Virtual Box>

-> 도구 -> 미디어 -> 만들기 -> VDI(VirtualBox 디스크 이미지) -> 고정 크기

-> 경로 지정 (ex:"E:\vm\ol7ora19r\ol7ora19r_storage\ol7ora19r_ACFS1_1.vdi") -> 40GB -> 디스크 속성을 "공유 가능"으로 변경

--> 여기 까지 완료 후 각각에 노드에(Oracle VM) 공유 디스크를 연결 해줌
--> 모든 Oracle VM의 전원을 off 시킨 상태에서 해야함

-- RAC1 : ol7ora19r1(192.168.0.21) : DRDB1 설정
VM 설정 -> 저장소 -> 컨트롤러:SATA -> ol7ora19r_ACFS1_1.vdi 선택

-- RAC2 : ol7ora19r2(192.168.0.22) : DRDB2 설정
VM 설정 -> 저장소 -> 컨트롤러:SATA -> ol7ora19r_ACFS1_1.vdi 선택

--> 모든 Oracle VM의 전원을 on 시킴

2. 공유 디스크 등록

2-1. ASMLIB 사용 시

[root@ol7ora19r1][/root]$ ls -l /dev/sd*
brw-rw----. 1 root disk 8, 0 Jun 3 17:25 /dev/sda
brw-rw----. 1 root disk 8, 1 Jun 3 17:25 /dev/sda1
brw-rw----. 1 root disk 8, 2 Jun 3 17:25 /dev/sda2
brw-rw----. 1 root disk 8, 16 Jun 3 17:25 /dev/sdb
brw-rw----. 1 root disk 8, 17 Jun 3 17:25 /dev/sdb1
brw-rw----. 1 root disk 8, 32 Jun 3 17:25 /dev/sdc
brw-rw----. 1 root disk 8, 33 Jun 3 17:25 /dev/sdc1
brw-rw----. 1 root disk 8, 48 Jun 3 17:25 /dev/sdd
brw-rw----. 1 root disk 8, 49 Jun 3 17:25 /dev/sdd1
brw-rw----. 1 root disk 8, 64 Jun 3 17:25 /dev/sde
brw-rw----. 1 root disk 8, 65 Jun 3 17:25 /dev/sde1
brw-rw----. 1 root disk 8, 80 Jun 3 17:25 /dev/sdf
brw-rw----. 1 root disk 8, 81 Jun 3 17:25 /dev/sdf1
brw-rw----. 1 root disk 8, 96 Jun 3 17:25 /dev/sdg

[root@ol7ora19r1][/root]$ fdisk /dev/sdg
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x8b712d2a.

Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-83886079, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-83886079, default 83886079):
Using default value 83886079
Partition 1 of type Linux and of size 40 GiB is set

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

[root@ol7ora19r1][/root]$ oracleasm createdisk ACFS1_1 /dev/sdg1
Writing disk header: done
Instantiating disk: done

[root@ol7ora19r1][/dev]$ oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...

[root@ol7ora19r1][/dev]$ oracleasm listdisks
ACFS1_1
CRS1
CRS2
CRS3
DATA1_1
FRA1_1

[root@ol7ora19r2][/root]$ oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "ACFS1_1"

[root@ol7ora19r2][/root]$ oracleasm listdisks
ACFS1_1
CRS1
CRS2
CRS3
DATA1_1
FRA1_1

[root@ol7ora19r2][/root]$ ls -l /dev/sd*
brw-rw----. 1 root disk 8, 0 Jun 3 17:25 /dev/sda
brw-rw----. 1 root disk 8, 1 Jun 3 17:25 /dev/sda1
brw-rw----. 1 root disk 8, 2 Jun 3 17:25 /dev/sda2
brw-rw----. 1 root disk 8, 16 Jun 3 17:33 /dev/sdb
brw-rw----. 1 root disk 8, 17 Jun 3 17:33 /dev/sdb1
brw-rw----. 1 root disk 8, 32 Jun 3 17:33 /dev/sdc
brw-rw----. 1 root disk 8, 33 Jun 3 17:33 /dev/sdc1
brw-rw----. 1 root disk 8, 48 Jun 3 17:33 /dev/sdd
brw-rw----. 1 root disk 8, 49 Jun 3 17:33 /dev/sdd1
brw-rw----. 1 root disk 8, 64 Jun 3 17:33 /dev/sde
brw-rw----. 1 root disk 8, 65 Jun 3 17:33 /dev/sde1
brw-rw----. 1 root disk 8, 80 Jun 3 17:33 /dev/sdf
brw-rw----. 1 root disk 8, 81 Jun 3 17:33 /dev/sdf1
brw-rw----. 1 root disk 8, 96 Jun 3 17:33 /dev/sdg
brw-rw----. 1 root disk 8, 97 Jun 3 17:33 /dev/sdg1

2-2. udev 사용 시

--RAC 1번
[root@ol7ora19r1][/dev]$ /usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/sdg
1ATA_VBOX_HARDDISK_VBcb172146-14b52cda

[root@ol7ora19r1][/dev]$ udevadm info --query=all --name=/dev/sdg | grep ID_PATH | grep -v ID_PATH_TAG
E: ID_PATH=pci-0000:00:0d.0-ata-8.0

[root@ol7ora19r1][/dev]$ vi /etc/udev/rules.d/99-oracle-asmdevices.rules
[root@ol7ora19r1][/dev]$ cat /etc/udev/rules.d/99-oracle-asmdevices.rules | grep sdg
KERNEL=="sdg", ENV{ID_PATH}=="pci-0000:00:0d.0-ata-8.0", SYMLINK+="oracleasm/disks/ACFS1_1", OWNER="grid", GROUP="asmadmin", MODE="0660"

[root@ol7ora19r1][/dev]$ udevadm control --reload-rules
[root@ol7ora19r1][/dev]$ udevadm trigger

[root@ol7ora19r1][/dev]$ ls -l /dev/oracleasm/disks | grep sdg
lrwxrwxrwx. 1 root root 9 Jun 1 19:16 ACFS1_1 -> ../../sdg

--RAC 2번
[root@ol7ora19r2][/root]$ /usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/sdg
1ATA_VBOX_HARDDISK_VBcb172146-14b52cda
[root@ol7ora19r2][/root]$ udevadm info --query=all --name=/dev/sdg | grep ID_PATH | grep -v ID_PATH_TAG
E: ID_PATH=pci-0000:00:0d.0-ata-8.0

[root@ol7ora19r2][/root]$ vi /etc/udev/rules.d/99-oracle-asmdevices.rules
[root@ol7ora19r2][/root]$ cat /etc/udev/rules.d/99-oracle-asmdevices.rules | grep sdg
KERNEL=="sdg", ENV{ID_PATH}=="pci-0000:00:0d.0-ata-8.0", SYMLINK+="oracleasm/disks/ACFS1_1", OWNER="grid", GROUP="asmadmin", MODE="0660"

[root@ol7ora19r2][/root]$ udevadm control --reload-rules
[root@ol7ora19r2][/root]$ udevadm trigger
[root@ol7ora19r2][/root]$ ls -l /dev/oracleasm/disks | grep sdg
lrwxrwxrwx. 1 root root 9 Jun 1 19:21 ACFS1_1 -> ../../sdg

--RAC 1번
[root@ol7ora19r1][/dev]$ oracleasm listdisks
ACFS1_1
CRS1
CRS2
CRS3
DATA1_1
FRA1_1

--RAC 2번
[root@ol7ora19r2][/root]$ oracleasm listdisks
ACFS1_1
CRS1
CRS2
CRS3
DATA1_1
FRA1_1

3. ASM 디스크 추가

[+ASM1:grid@rdb01d][/home/grid]$ echo ORACLE_SID=+ASM1
ORACLE_SID=+ASM1
[+ASM1:grid@rdb01d][/home/grid]$ rlwrap sqlplus "/as sysasm"

SQL*Plus: Release 12.2.0.1.0 Production on Sun Jun 1 19:35:11 2025

Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

--drop DISKGROUP ACFS1;
[2025-06-04:16:37:13][ol7ora19r1]<SYS@+ASM1> CREATE DISKGROUP ACFS1 EXTERNAL REDUNDANCY DISK '/dev/oracleasm/disks/ACFS1_1' SIZE 40959M ATTRIBUTE 'compatible.asm' = '19.0', 'compatible.rdbms'='19.0', 'compatible.advm' = '19.0';

Diskgroup created.

--> 생성한 ASM 디스크그룹 및 ASM 디스크의 정보 확인

SET LINE 200
COL DISK_GROUP FOR A10
COL LABEL FOR A10
COL STATE FOR A10
SELECT A.NAME AS DISK_GROUP, D.NAME "LABEL", A.STATE
FROM V$ASM_DISK D

, V$ASM_DISKGROUP A
WHERE D.GROUP_NUMBER=A.GROUP_NUMBER
and A.NAME = 'ACFS1'
ORDER BY 2

;

col COMPATIBILITY form a10
col DATABASE_COMPATIBILITY form a30
col NAME form a20
select group_number, name, compatibility, database_compatibility
from v$asm_diskgroup
where NAME = 'ACFS1';

[+ASM1:grid@ol7ora19r1][/home/grid]$ asmcmd
ASMCMD [+] > lsdg
State    Type    Rebal Sector Logical_Sector Block       AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N         512             512   4096 1048576     40959    40895                0           40895              0             N ACFS1/
MOUNTED NORMAL N         512             512   4096 4194304      6132     5096             2044            1526              0             Y CRS/
MOUNTED EXTERN N         512             512   4096 4194304    131068   126460                0          126460              0             N DATA1/
MOUNTED EXTERN N         512             512   4096 4194304    131068   128952                0          128952              0             N FRA1/

--RAC 2번에도 mount
[+ASM2:grid@ol7ora19r2][/home/grid]$ asmcmd
ASMCMD [+] > lsdg
State    Type    Rebal Sector Logical_Sector Block       AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED NORMAL N         512             512   4096 4194304      6132     5096             2044            1526              0             Y CRS/
MOUNTED EXTERN N         512             512   4096 4194304    131068   126460                0          126460              0             N DATA1/
MOUNTED EXTERN N         512             512   4096 4194304    131068   128952                0          128952              0             N FRA1/

ASMCMD [+] > mount ACFS1
ASMCMD [+] > lsdg
State    Type    Rebal Sector Logical_Sector Block       AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N         512             512   4096 1048576     40959    40851                0           40851              0             N ACFS1/
MOUNTED NORMAL N         512             512   4096 4194304      6132     5096             2044            1526              0             Y CRS/
MOUNTED EXTERN N         512             512   4096 4194304    131068   126460                0          126460              0             N DATA1/
MOUNTED EXTERN N         512             512   4096 4194304    131068   128952                0          128952              0             N FRA1/

[+ASM2:grid@ol7ora19r2][/home/grid]$ crsctl status resource ora.ACFS1.dg -t
--------------------------------------------------------------------------------
Name           Target State        Server                   State details
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ACFS1.dg(ora.asmgroup)
      1        ONLINE ONLINE       ol7ora19r1               STABLE
      2        ONLINE ONLINE       ol7ora19r2               STABLE
--------------------------------------------------------------------------------

4. ACFS 볼륨 추가

ASMCMD [+] > volcreate -G ACFS1 -s 10G ACFS1_VOL1
ASMCMD [+] > volinfo --all
Diskgroup Name: ACFS1

         Volume Name: ACFS1_VOL1
         Volume Device: /dev/asm/acfs1_vol1-440
         State: ENABLED
         Size (MB): 10240
         Resize Unit (MB): 64
         Redundancy: UNPROT
         Stripe Columns: 8
         Stripe Width (K): 1024
         Usage:
         Mountpath:

[+ASM1:grid@ol7ora19r1][/home/grid]$ crsctl stat res ora.ACFS1.ACFS1_VOL1.advm -t
--------------------------------------------------------------------------------
Name           Target State        Server                   State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ACFS1.ACFS1_VOL1.advm
               ONLINE ONLINE       ol7ora19r1               STABLE
               ONLINE ONLINE       ol7ora19r2               STABLE
--------------------------------------------------------------------------------

--RAC1번

[root@ol7ora19r1][/]$ mkdir /acfs1_vol1
[root@ol7ora19r1][/]$ chown oracle:oinstall /acfs1_vol1

--RAC2번

[root@ol7ora19r2][/root]$ mkdir /acfs1_vol1
[root@ol7ora19r2][/root]$ chown oracle:oinstall /acfs1_vol1

--RAC1번에서만

[+ASM1:grid@ol7ora19r1][/]$ mkfs -t acfs /dev/asm/acfs1_vol1-440
mkfs.acfs: version                   = 19.0.0.0.0
mkfs.acfs: on-disk version           = 46.0
mkfs.acfs: volume                    = /dev/asm/acfs1_vol1-440
mkfs.acfs: volume size               = 10737418240 ( 10.00 GB )
mkfs.acfs: Format complete.

[root@ol7ora19r1][/root]$ srvctl add filesystem -d /dev/asm/acfs1_vol1-440 -m /acfs1_vol1 -u oracle -fstype ACFS -autostart ALWAYS
[root@ol7ora19r1][/root]$ crsctl stat res ora.acfs1.acfs1_vol1.acfs -t
--------------------------------------------------------------------------------
Name           Target State        Server                   State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.acfs1.acfs1_vol1.acfs
               OFFLINE OFFLINE      ol7ora19r1               STABLE
               OFFLINE OFFLINE      ol7ora19r2               STABLE
--------------------------------------------------------------------------------

[root@ol7ora19r1][/root]$ srvctl start filesystem -d /dev/asm/acfs1_vol1-440

[root@ol7ora19r1][/root]$ crsctl stat res ora.acfs1.acfs1_vol1.acfs -t
--------------------------------------------------------------------------------
Name           Target State        Server                   State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.acfs1.acfs1_vol1.acfs
               ONLINE ONLINE       ol7ora19r1               mounted on /acfs1_vo
                                                             l1,STABLE
               ONLINE ONLINE       ol7ora19r2               mounted on /acfs1_vo
                                                             l1,STABLE
--------------------------------------------------------------------------------

[root@ol7ora19r1][/root]$ srvctl config filesystem
Volume device: /dev/asm/acfs1_vol1-440
Diskgroup name: acfs1
Volume name: acfs1_vol1
Canonical volume device: /dev/asm/acfs1_vol1-440
Accelerator volume devices:
Mountpoint path: /acfs1_vol1
Mount point owner: oracle
Mount point group: oinstall
Mount permissions: owner:oracle:rwx,pgrp:oinstall:r-x,other::r-x
Mount users: grid
Type: ACFS
Mount options:
Description:
ACFS file system is enabled
ACFS file system is individually enabled on nodes:
ACFS file system is individually disabled on nodes:

[root@ol7ora19r1][/root]$ df -h |grep acfs
/dev/asm/acfs1_vol1-440 10G 570M 9.5G 6% /acfs1_vol1

[root@ol7ora19r2][/root]$ df -h |grep acfs
/dev/asm/acfs1_vol1-440 10G 570M 9.5G 6% /acfs1_vol1

--ACFS 영역에 신규 파일 생성
[ORA19R1:oracle@ol7ora19r1][/acfs1_vol1]$ touch 1.txt

--양쪽 노드에서 파일이 공유됨을 확인

--RAC1번

[ORA19R1:oracle@ol7ora19r1][/acfs1_vol1]$ ls -l 1.txt
-rw-r--r--. 1 oracle oinstall 0 Jun 4 17:21 1.txt

--RAC2번

[ORA19R2:oracle@ol7ora19r2][/home/oracle]$ cd /acfs1_vol1/
[ORA19R2:oracle@ol7ora19r2][/acfs1_vol1]$ ls -l
total 64
-rw-r--r--. 1 oracle oinstall 0 Jun 4 17:21 1.txt
drwx------. 2 root root 65536 Jun 4 17:17 lost+found

'Oracle > ASM' 카테고리의 다른 글

[2025-06-06] ASM 내에 파일을 원격지의 ASM 위치로 copy 하기 (19c) (0)	2025.08.14
[2025-06-06] ORACLE RAC 환경에서 ASM Disk Group명 변경 (19c) (4)	2025.08.13
[2025-06-03] Oracle VM VirtualBox의 RAC 환경에서 ASM 디스크 추가 및 asm_power_limit 설정 (19c) (3)	2025.08.10
[2025-06-04] Oracle 19c 2Node RAC 환경에서 Instance 및 ASM Instance의 Password 파일 생성 (ASM 영역에 생성) (2)	2025.07.27
Oracle RAC 12cR2 환경에서 ASM 인스턴스의 관리 권한이 sysasm 인지 확인하기 (0)	2025.06.01

db-expert

[2025-06-03] Oracle VM VirtualBox의 RAC 환경에서 ACFS 구성 (19c)

'Oracle > ASM' 카테고리의 다른 글

+ Recent posts

티스토리툴바