Grid Infrasturuer Redundant Interconnect

The grid infrastrure redundant interconnect and ora.cluster_interconnect.haip without any 3-party failover (like bond, ipmp, etc) is supported by GI starting 11.2.0.2.  Multiple private network adapter can be defined during the installation or after via oifcfg.  Gi can activate 4 private network adapter at maximum.  Oracle GI automatically picks local addresses from reserved 169.254.*.* subnet for HAIP.  By default, interconnect traffic is load balanced across all active interconnect interface and will fail over to other adapters if one fails.

The redundant interconnect is based on a highly available virtual IP is assigned to each private network.  It will use virtual private IP 169.254.0.0 instead of real private IP.  For pre 11.2.0.2 instance, the init.ora parameter cluster_interconects by default is still use the real private IP.   It can be updated to take advantage of the new feature.

[root@rac0]# ./oifcfg getif

eth2  173.11.33.0  global  cluster_interconnect

eth3  173.11.33.0  global  cluster_interconnect

bond0  10.5.1.0    global  public

[root@rac01]# ./oifcfg iflist -p -n

eth2  173.11.33.0                PRIVATE  255.255.255.0

eth2  169.254.0.0                UNKNOWN  255.255.128.0 è is started by resource haip

eth3  172.17.33.0                PRIVATE  255.255.255.0

eth3  169.254.0.0                UNKNOWN  255.255.128.0 è is started by resource haip

bond0  10.5.1.0                   PRIVATE  255.255.255.0

[oracle@rac01]$ ./crsctl stat res -t -init

--------------------------------------------------------------------------------

NAME           TARGET  STATE        SERVER                   STATE_DETAILS      

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.asm

      1        ONLINE  ONLINE       rac01              Started            

ora.cluster_interconnect.haip

HAIP can be validated via ifconfig –a or in ASM and database instance alerts.

ifconfig –a

eth2:1    Link encap:Ethernet  HWaddr 64:31:50:50:BC:FA 

          inet addr:169.254.xx.xx  Bcast:169.254.127.255  Mask:255.255.128.0

          UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1

          Interrupt:170 Memory:f8000000-f8012800

eth3:1    Link encap:Ethernet  HWaddr 64:31:50:50:BC:FC 

          inet addr:169.254.xxx.xxx  Bcast:169.254.255.255  Mask:255.255.128.0

          UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1

          Interrupt:178 Memory:f6000000-f6012800

adrci> set homepath diag/asm/+asm/+ASM1

adrci> show alert -P "MESSAGE_TEXT like '%eth2:1%'"

ADR Home = /apps/oracle/diag/asm/+asm/+ASM1:

*************************************************************************

Output the results to file: /tmp/alert_10218_4735_+ASM1_12.ado

2011-05-27 15:00:39.753000 -05:00

Private Interface 'eth2:1' configured from GPnP for use as a private interconnect.

  [name='eth2:1', type=1, ip=169.254.x.x, mac=6-31-50-4e-054, net=169.254.0.0/17, mask=255.255.128.0, use=haip:cluster_interconnect/62]

2011-05-27 16:28:08.301000 -05:00

Private Interface 'eth2:1' configured from GPnP for use as a private interconnect.

  [name='eth2:1', type=1, ip=169.254.x.x, mac=64-31-50-4e-054, net=169.254.0.0/17, mask=255.255.11.0, use=haip:cluster_interconnect/62]