11.6 Memory fault tolerance

Read(13) Label: memory data zone, init.dfx,

With data-intensive computations, we can load each part of a cluster composite table into the corresponding node’s memory and access the memories to speed up a query processing. The shared data is loaded into memory data zones at the startup of the nodes. Then we access data globally for all processes by memory data zone numbers.

First we need to initiate the related nodes to load a cluster composite table from memories by executing init.dfx. The SPL file is located in a node’s main path or search path, in which you can define how to load the shared data:

 

A

B

1

/Shared initiation code

2

if z > 0

>zone(z)

3

 

=file("salespart.ctx":z)

4

 

=B3.create().cursor().fetch()

5

 

>env(SALES, B4)

The init.dfx file defines two parameters – z and zoneN, which are passed in by node numbers at the initiation.

In the above init.dfx file, zone(z) function numbers the current memory data zone for a later call. file(fn:z) function searches the current data zone z on the node for file fn. env(v, data) function stores the retrieved file in global variable v in the memory data zone.

Let’s look at how to query a cluster composite table stored in multiple memory data zones:

 

A

1

[127.0.0.1:8281, 127.0.0.1:8291]

2

=hosts@i(A1,2)

3

=memory(A2,SALES)

4

=A3.cursor().fetch()

Here we use two locally-initiated nodes. The port numbers are 8281 and 8291. In A2, hosts@i(h, n) function searches the nodes’ main processes (sequence h) to get available nodes that hold data zones numbered 1 and 2 respectively; @i option enables executing init.dfx file on each available node to load their part of the cluster composite table to the memory. Below is the sequence of available nodes:

You can view the initiation status on the node information window, as shown below:

In A3, memory(h, v) function puts together the global variables (v) in the available nodes’ memory data zones to get the whole cluster composite table. A4 accesses memory data zones to query the cluster composite table. An in-memory cluster composite table is queried in the same manner as an external one is processed. Below is A4’s result:

An in-memory cluster composite table consisting of data coming from multiple nodes requires high stability. In case that the cluster composite table fails due to a node malfunction, we introduce memory fault tolerance. We make a backup of the cluster composite table to store it on one or more backup nodes. If a node breaks down, a backup node will respond to provide that part of the data. For example, we add a node 8301 to store the cluster composite table distributed in node1 and node2’s memory data zones in memory data zone node3, as shown below:

Run node3 and close node1 to execute the code below:

 

A

1

[127.0.0.1:8281, 127.0.0.1:8291, 127.0.0.1:8301]

2

=hosts@i(A1,2)

3

=memory(A2,SALES)

4

=A3.cursor().fetch()

Execution goes on properly and A4 returns same and correct result. Here’s A2’s result:

 

Node 8301 replaces node 8281 to provide that part of the composite table for a smooth execution.