Need to configure the cross site replication for infinispan 13 cache server

[hemachandiran-linux]

[Need to configure the cross site replication for infinispan 13 cache server]

[tristantarrant]

Read the docs https://infinispan.org/docs/stable/titles/xsite/xsite.html

[hemachandiran-linux]

Hi Tristantarrant,
thanks for the reply. Already we have gone through this link. but we are facing issue. Can you please help me with little more few simple details

[tristantarrant]

You need to specify what the problem is. Configuration, logs, etc

[hemachandiran-linux]

Hi tristantarrant,
Sorry for the late reply,
setup1 = === we have 5 vm's configured as a PROD servers
setup 2 ==== we have 5 vm's configured as a BAckup servers

Here is my that infinispan.xml file for cross site configuration. we have setup 1(which have 5vms) and setup 2(GR site have 5vm's).
we need to configure the cross site for setup 1 to setup 2. But we have already configured the clustering for within the setup's

here we are trying to test between two servers(10.184.42.173 & 10.184.42.174) in Setup 2 itself. But we got issue with the below conf. Please find the infinispan.xml config file we have configured in our server. Please help us to resolve this issue ASAP.

=======================================================================================

[tristantarrant]

I don't see your configuration. Did you base it off infinispan-xsite.xml which is included in our default distribution? Also you are not saying what doesn't work and you did not include any logs.

[hemachandiran-linux]

Please find my conf file infinispan.xml

and infinispan-xsite.xml-- this config file we left the default conf as it is.. ..

[tristantarrant]

You still have not provided any logs indicating what the problem is.

[hemachandiran-linux]

`2022-09-28 11:42:37,107 ERROR (MPING-5,_innolx42173-20532:LON) [org.jgroups.protocols.MPING] JGRP000191: failed receiving packet from /10.184.42.173:37564: java.lang.ArrayIndexOutOfBoundsException: -17845 java.lang.ArrayIndexOutOfBoundsException: -17845
at org.jgroups.conf.ClassConfigurator.create(ClassConfigurator.java:129)
at org.jgroups.Message.readHeader(Message.java:836)
at org.jgroups.Message.readFrom(Message.java:709)
at org.jgroups.protocols.MPING.run(MPING.java:198)
at java.lang.Thread.run(Thread.java:748)

2022-09-28 11:42:44,491 ERROR (MPING-5,_innolx42173-20532:LON) [org.jgroups.protocols.MPING] JGRP000191: failed receiving packet from /10.184.42.173:37564: java.lang.ArrayIndexOutOfBoundsException: -17845 java.lang.ArrayIndexOutOfBoundsException: -17845
at org.jgroups.conf.ClassConfigurator.create(ClassConfigurator.java:129)
at org.jgroups.Message.readHeader(Message.java:836)
at org.jgroups.Message.readFrom(Message.java:709)
at org.jgroups.protocols.MPING.run(MPING.java:198)
at java.lang.Thread.run(Thread.java:748)

2022-09-28 11:42:44,864 ERROR (MPING-5,_innolx42173-20532:LON) [org.jgroups.protocols.MPING] JGRP000191: failed receiving packet from /10.184.42.173:37564: java.lang.ArrayIndexOutOfBoundsException: -17845 java.lang.ArrayIndexOutOfBoundsException: -17845
at org.jgroups.conf.ClassConfigurator.create(ClassConfigurator.java:129)
at org.jgroups.Message.readHeader(Message.java:836)
at org.jgroups.Message.readFrom(Message.java:709)
at org.jgroups.protocols.MPING.run(MPING.java:198)
at java.lang.Thread.run(Thread.java:748)

[cisapp@innolx42173 bin]$ 2022-09-28 11:42:53,749 ERROR (MPING-5,_innolx42173-20532:LON) [org.jgroups.protocols.MPING] JGRP000191: failed receiving packet from /10.184.42.173:37564: java.lang.ArrayIndexOutOfBoundsException: -17845 java.lang.ArrayIndexOutOfBoundsException: -17845
at org.jgroups.conf.ClassConfigurator.create(ClassConfigurator.java:129)
at org.jgroups.Message.readHeader(Message.java:836)
at org.jgroups.Message.readFrom(Message.java:709)
at org.jgroups.protocols.MPING.run(MPING.java:198)
at java.lang.Thread.run(Thread.java:748)

^C
[cisapp@innolx42173 bin]$ 2022-09-28 11:42:56,112 ERROR (MPING-5,_innolx42173-20532:LON) [org.jgroups.protocols.MPING] JGRP000191: failed receiving packet from /10.184.42.173:37564: java.lang.ArrayIndexOutOfBoundsException: -17845 java.lang.ArrayIndexOutOfBoundsException: -17845
at org.jgroups.conf.ClassConfigurator.create(ClassConfigurator.java:129)
at org.jgroups.Message.readHeader(Message.java:836)
at org.jgroups.Message.readFrom(Message.java:709)
at org.jgroups.protocols.MPING.run(MPING.java:198)
at java.lang.Thread.run(Thread.java:748)`

Error message after configured the cross site for site 1(10.184.42.173) to site 2 (10.184.42.174)
`infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:infinispan:config:13.0 https://infinispan.org/schemas/infinispan-config-13.0.xsd
urn:infinispan:server:13.0 https://infinispan.org/schemas/infinispan-server-13.0.xsd" xmlns="urn:infinispan:config:13.0" xmlns:server="urn:infinispan:server:13.0">

`

[tristantarrant]

You are declaring a relay-global stack for xsite, but you are then using tcp in the default-stack attribute of the remote-sites element. I don't see the cache-container transport element declaration in your screenshot, but I assume it is pointing to the xsite stack.

[hemachandiran-linux]

We have server setup 1(Primary) and setup 2(Geo Red).
Each setup we have 5 VMS/servers with same subnet.
In setup 1 we have jboss business installed in one server and 3 infinispan cache server installed in 3 servers.
Another 3 infinispan cache server is installed in 3 other servers which is loacted in Setup 2.
But when we started the JBOSS business , it is started establishing the connection to all 6 infinispan servers, but that should not be happen for our configuration. Since we have configured clustering for infinispan cache in Jboss Business application HOTROD properties like this""""HOTROD_APPLICATION_CACHE_URL=10.184.42.168:11222;10.184.42.169:11222;10.184.42.170:11222"""

Also, We have configured the cross site replication for pcache in setup 1 and setup 2 ...there we have configured all 6 infinispan server ip's in infinispam.xml file. Please find the below configuration of infinispan.xml file.

<jgroups>
       <stack name="prod">
         <TCP bind_port="7800" bind_addr="10.184.42.168"/>
         <TCPPING initial_hosts="${jgroups.tcpping.initial_hosts:10.184.42.168[7800],10.184.42.169[7800],10.184.42.170[7800]}"
              port_range="1" />
         <MERGE3 />
         <FD_SOCK />
         <FD_ALL timeout="3000" interval="1000" timeout_check_interval="1000" />
         <VERIFY_SUSPECT timeout="1000" />
         <pbcast.NAKACK2 use_mcast_xmit="false" xmit_interval="100" xmit_table_num_rows="50"
                         xmit_table_msgs_per_row="1024" xmit_table_max_compaction_time="30000" />
         <UNICAST3 xmit_interval="100" xmit_table_num_rows="50" xmit_table_msgs_per_row="1024"
                   xmit_table_max_compaction_time="30000" />
         <pbcast.STABLE stability_delay="200" desired_avg_gossip="2000" max_bytes="1M" />
         <pbcast.GMS print_local_addr="false" join_timeout="${jgroups.join_timeout:2000}" />
         <UFC_NB max_credits="3m" min_threshold="0.40" />
         <MFC_NB max_credits="3m" min_threshold="0.40" />
         <FRAG2 />
       </stack>
  <stack name="relay-global" extends="tcp">
      <TCPPING initial_hosts="10.184.42.168[7801],10.184.42.169[7801],10.184.42.170[7801],10.184.42.173[7801],10.184.42.174[7801],10.184.42.175[7801]"
               stack.combine="REPLACE"
               stack.position="MPING"/>
    </stack> -->
    <stack name="xsite" extends="udp">
      <relay.RELAY2 site="LON" xmlns="urn:org:jgroups" max_site_masters="1000" can_become_site_master="true"/>
      <remote-sites default-stack="relay-global">
         <remote-site name="NYC"/>
         <remote-site name="LON"/>
      </remote-sites>
    </stack>
   </jgroups>

Netstat of JBOSS

[xxx@innolx42167 ~]$ netstat -anlp | grep 11222
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp 0 0 10.184.42.167:56384 10.184.42.173:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:56386 10.184.42.173:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:54214 10.184.42.169:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:43876 10.184.42.168:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:56366 10.184.42.173:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:56376 10.184.42.173:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:56390 10.184.42.173:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:54230 10.184.42.169:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:56362 10.184.42.173:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:56382 10.184.42.173:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:39156 10.184.42.170:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:56372 10.184.42.173:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:56388 10.184.42.173:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:56380 10.184.42.173:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:56368 10.184.42.173:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:56370 10.184.42.173:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:39172 10.184.42.170:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:56364 10.184.42.173:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:43892 10.184.42.168:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:56374 10.184.42.173:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:56396 10.184.42.173:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:56394 10.184.42.173:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:56392 10.184.42.173:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:56378 10.184.42.173:11222 ESTABLISHED 54618/java
tcp 0 0 10.184.42.167:56360 10.184.42.173:11222 ESTABLISHED 54618/java

[tristantarrant]

By default, the Hot Rod clients will automatically handle server topology changes. Read the section on client intelligence: https://infinispan.org/docs/stable/titles/hotrod_java/hotrod_java.html#hotrod-client-intelligence_hotrod-java-client

[tristantarrant]

I think you also want to set port_range to 0 otherwise the clusters will discover each other.