Architecture-aware Coding for Distributed Storage: Repairable Block Failure Resilient Codes

In large scale distributed storage systems (DSS) deployed in cloud computing, correlated failures resulting in simultaneous failure (or, unavailability) of blocks of nodes are common. In such scenarios, the stored data or a content of a failed node can only be reconstructed from the available live nodes belonging to available blocks. To analyze the resilience of the system against such block failures, this work introduces the framework of Block Failure Resilient (BFR) codes, wherein the data (e.g., file in DSS) can be decoded by reading out from a same number of codeword symbols (nodes) from a subset of available blocks of the underlying codeword. Further, repairable BFR codes are introduced, wherein any codeword symbol in a failed block can be repaired by contacting to a subset of remaining blocks in the system. File size bounds for repairable BFR codes are derived, and the trade-off between per node storage and repair bandwidth is analyzed, and corresponding minimum storage regenerating (BFR-MSR) and minimum bandwidth regenerating (BFR-MBR) points are derived. Explicit codes achieving the two operating points for a special case of parameters are constructed wherein the underlying regenerating codewords are distributed to BFR codeword symbols according to combinatorial designs. Finally, BFR locally repairable codes (BFR-LRC) are introduced, an upper bound on the resilience is derived and, optimal code construction are provided by a concatenation Gabidulin and MDS codes. Repair efficiency of BFR-LRC is further studied via the use of BFR-MSR/MBR codes as local codes. Code constructions achieving optimal resilience for BFR-MSR/MBR-LRCs are provided for certain parameter regimes. Overall, this work introduces the framework of block failures along with optimal code constructions, and the study of architecture-aware coding for distributed storage systems.


A. Background
Increasing demand for storing and analyzing big-data as well as several applications of cloud computing systems require efficient cloud computing infrastructures.Under today's circumstances where the data is growing exponentially, it is crucial to have storage systems that guarantee no permanent loss of data.However, one inevitable nature of the storage systems is node failures.In order to provide resilience against such failures, redundancy is introduced in the storage.
Classical redundancy schemes range from replication to erasure coding.Erasure coding allows for better performance in terms of reliability and redundancy compared to replication.To increase repair bandwidth efficiency of erasure coded systems, regenerating codes are proposed in the seminal work of Dimakis et al. [1].In such a model of distributed storage systems (DSS), the file of size M is encoded to n nodes such that any k ≤ n nodes (each with α symbols) allow for reconstructing the file and any d ≥ k nodes (with β ≤ α symbols from each) reconstruct a failed node with a repair bandwidth γ = dβ.The trade-off between per node storage (α) and repair bandwidth (γ) is characterized and two ends of the trade-off curve are named as minimum storage regenerating (MSR) and minimum bandwidth regenerating (MBR) points [1].Several explicit codes have been proposed to achieve these operating points recently [2]- [8].
Another metric for an efficient repair is repair degree d and regenerating codes necessarily have d ≥ k.Codes with locality and locally repairable codes with regeneration properties [9]- [18] allow for a small repair degree, wherein a failed node is reconstructed via local connections.Instances of such codes are recently considered in DSS [19], [20].Small repair degree has its benefits in terms of the implementation of DSS since a failed node requires only local connections.In particular, when more nodes are busy for recovery operations, this in turn creates additional access cost in the network.
In large-scale distributed storage systems (such as GFS [21]), correlated failures are unavoidable.As analyzed in [22], these simultaneous failures of multiple nodes affect the performance of computing systems severely.The analysis in [22] further shows that these correlated failures arise due to failure domains.For example, nodes connected to the same power source or nodes belonging to the same rack exhibit these structured failure bursts.The unavailability periods are transient, and largest failure bursts almost always have significant rack-correlation.For example, in Fig. 1 there are three racks in a DSS, each connected to the same switch.Assume that topof-rack (TOR) switch of the first rack is failed; hence, all the disks in the rack are assumed to be failed or unavailable (same failure domain).Now consider that, while the first rack is unavailable, a user asks for some data D stored in one of the disks in the first rack.If both the data and its corresponding redundancy were stored all together in the first rack, then the user would not be able to access the data until the TOR switch works properly.On the other hand, assume that redundancy is distributed to the other two racks; then, the user could connect to those two racks in order to reconstruct the data.Furthermore, if the disk storing the data fails in the first rack, then the repair process could be performed similarly since the failed disk could connect to other racks to download some amount of data for repair process.This architecture is also relevant to disk storage, where the disk is divided into sectors, each can be unavailable or under failure.In order to overcome from failures having such patterns, a different approach is needed.
In terms of data recovery operations, such an architecture can be considered as a relaxation Fig. 1: A data-center architecture where top-of-rack (TOR) switch of the first rack fails of regenerating coded system.In particular, considering a load balancing property for the racks utilized in the recovery operations, we focus on the following model.Regenerating codes allow the failed node to connect any d nodes for repair purposes whereas we consider failed node connecting to a total of d nodes in some restricted subset of nodes, i.e., any d 2 nodes in the second and third racks respectively for the example in Fig. 1.Similarly, any k property of regenerating codes for data-reconstruction is also relaxed i.e., DC can connect to k 3 nodes in each rack in the example above.The outcome of such a relaxation is directly related to the performance of the DSS in terms of per node storage and repair bandwidth trade-off.For example, consider a DSS where a file of size M is stored over n = 10 nodes such that any k = 4 nodes are enough to reconstruct the data.In addition, any d = 4 nodes are required to regenerate a failed node.For such a system that uses regenerating code, trade-off curve can be obtained as in Fig. 2c.Now consider that, n = 10 nodes are distributed into two distinct groups such that each block has n 2 = 5 nodes.Note that, a failed node in one of the blocks can now be repaired by connecting any d = 4 nodes in the other block.Also, DC can connect k 2 = 2 nodes per block to reconstruct the original message M. For such a relaxation, we can obtain the corresponding trade-off curve between per node storage and repair bandwidth as in Fig. 2c.Observe that the new MSR point, called BFR-MSR, has significantly lower repair bandwidth than MSR point as a result of this relaxation.In this paper, we further show that the gap between the repair bandwidth of BFR-MSR and MBR point can be made arbitrarily small for large systems while keeping the per node storage of BFR-MSR point same as MSR point.Therefore, such a relaxation of regenerating codes allow for simultaneous achievability of per-node storage of MSR point and repair bandwidth of MBR point.

B. Contributions and organization
The contributions of this paper can be summarized as follows: • We develop a framework to analyze resilience against block failures in DSS with node repair efficiencies.We consider a DSS with a single failure domain, where nodes belonging to the same failure group form a block of the codeword.• We introduce block failure resilient (BFR) codes, which allow for data collection from any b c = b − ρ blocks, where b is the number of blocks in the system and ρ is the resilience parameter of the code.Considering a load-balancing among blocks, a same number of nodes are contacted within these b c blocks.(A total of k = k c b c nodes and downloading α -i.e., all -symbols from each.)This constitutes data collection property of BFR codes.(ρ = 0 case can be considered as a special case of batch codes introduced in [23].) and MBR properties simultaneously is asymptotically possible.(This is somewhat similar to the property of Twin codes [24], but here the data collection property is different.)Then, for a system with b ≥ 3 blocks case, we consider utilizing multiple codewords, which are placed into DSS via a combinatorial design (projective planes) based codeword placement algorithm. placement.
• We also analyze the case where regenerating codes are used to improve repair efficiency of BFR-LRC codes.These codes are called BFR-MSR/MBR-LRC and they have a better performance in terms of repair bandwidth.We identify the upper bound on the file size that can be stored for both cases and also propose a construction that utilizes DCBD based construction to achieve the studied bound for certain parameter regimes.
• We provide codes with table-based type of relaxation for repairable BFR in order to operate within wider set of parameters.
The rest of the paper is is organized as follows.In Section II, we introduce BFR codes and provide preliminaries.Section III focuses on file size bounds for efficient repair where we study BFR-MSR and BFR-MBR points.In Section IV, we provide explicit code constructions.We discuss BFR-LRC and local regeneration in BFR-LRC in Section V. We extend our discussion in Section VI where we analyze repair time of DSS with BFR as well as other codes e.g., regenerating codes, and also propose a relaxation for the BFR model, where we provide explicit codes achieving the performance of BFR-MSR/MBR.Finally we conclude in Section VII.

C. Related Work
In the seminal work of Dimakis et al. [1], regenerating codes are presented, which are proposed to improve upon classical erasure codes in terms of repair bandwidth.The authors focus on a setting similar to maximum distance separable (MDS) codes as regenerating codes also have any k out of n property which allows data collector to connect any k nodes to decode data stored over n nodes; however, they show that by allowing a failed node to connect d ≥ k nodes, they can significantly reduce the repair bandwidth.Thus, a trade-off between per node storage and repair bandwidth for a single node repair is presented.Such a repair process is referred to as functional repair since the newcomer node may not store the same data as the failed node; however in terms of functionality, they are equivalent.In [1], [6], [7], functional repair is studied to construct codes achieving the optimality at the two ends of the trade-off curve.In [3], the focus is on exact repair where the failed node is repaired such that the newcomer stores exactly same data as the failed node stored.The proposed code construction is optimal for all parameters (n, k, d) for minimum bandwidth regeneration (MBR) point as well as optimal for (n, k, d ≥ 2k − 2) for minimum storage regeneration (MSR) point.It can be deducted that since n − 1 ≥ d necessarily, these codes' rate is bounded by 1  2 + 1 2n for MSR point.[4] utilizes two parity nodes to construct exact regeneration codes that are optimal in repair bandwidth, whereas [2], [8] focus on exact-repair for systematic node repair through permutation-matrix based constructions.Our BFR model here can be seen as a structured relaxation of the data collection and repair process of regenerating codes as discussed in the previous subsection.
Combinatorial designs are utilized in constructing fractional repetition codes in [25].In this construction, MDS codewords are replicated and these replicated symbols are placed to storage nodes according to the combinatorial designs.The resulting coding scheme can be viewed as the relaxation of regenerating codes where the repairs are performed table-based (instead of "any d" property).Other works that utilize combinatorial designs include [26]- [28].For the constructions proposed in this paper, we consider having (in certain cases precoded versions of) regenerating codewords placed to storage nodes according to combinatorial designs.As compared to fractional repetition codes, this allows to have bandwidth efficient repair with"any d r property in the block failure context studied here.In the last part of the sequel, we revisit and provide codes with a table-based type of relaxation for repairable BFR model, where instead of "any d r " per helper block we consider "any d" per helper sub-block.
Proposed recently in [11], local codes operate node repairs within some local group thereby reducing repair degree.In [9], minimum distance of locally repairable codes are studied for scalar case only and Pyramid codes are shown to be optimal in terms of minimum distance [14].[13] generalizes the bound by allowing multiple local parities per local group for the scalar linear code.Next, [10] focuses on the minimum distance bound for vector codes which have only one local parity per group as well as characterize the trade-off between per node storage and resilience.In [12], [29], the authors generalize the bound on minimum distance for vectors codes which can have multiple parity nodes per local group and characterize the trade-off between per node storage and resilience.Recently.[30], has proposed locally repairable codes with small field sizes by considering polynomial construction of Reed-Solomon codes.For LRCs, when the number of failures exceed the corresponding minimum distance of the local group, the local recovery would not be possible.Our BFR model here can be considered as the opposite of locality property, where non-local nodes have to be contacted for recovery operations.
Recently, another type of erasure codes called Partial-MDS are proposed for RAID systems to tolerate not just disk failures but also sector failures [31].Considering a stripe on a disk system as an r × n array where disks are represented by columns and all disks have r sectors, PMDS codes tolerate an erasure of any m sectors per row plus any s additional elements.Later, relaxation of PMDS codes are studied in [32] which allow erasure of any m columns plus any s additional elements.These codes are called Sector-Disk (SD) codes.Our BFR model can also be considered as a class of SD codes since BFR construction tolerates ρ block failures which can be treated as disks (columns) in SD code construction.However, for the full access case, BFR codes do not have any additional erasure protection beyond m = ρ disk failures.And, for the partial access case, BFR codes allow additional erasure protection but the code tolerates any additional c − k c number of erasures per block (disk) rather than s number of erasures that can occur anywhere over the remaining disks.On the other hand, repair process in SD codes utilize the row-wise parities (one sector from other disks) to correct the corresponding sectors in the failed disks, but repairable BFR allows the failed node (sector) in a block (disk) to contact any set of nodes in other blocks.
We note that the blocks in our model can be used to model racks in DSS.Such a model is related to the work in [33] which differentiates between within-rack communication and crossrack communication.Our focus here would correspond to the case where within the failure May 18, 2016 DRAFT group, communication is much higher than the cross-group communication, as no nodes from the failed/unavailable group can be contacted to regenerate a node.

II. BACKGROUND AND PRELIMINARIES
A. Block failure resilient codes and repairability Consider a code C which maps M symbols (over F q ) in f (file) to length n codewords (nodes) c = (c 1 , . . ., c n ) with c i ∈ F α q for i = 1, . . ., n.These codewords are distributed into b blocks each with block capacity c = b n nodes per block.We have the following definition.
Definition 1 (Block Failure Resilient (BFR) Codes).An (n, b, M, k, ρ, α) block failure resilient (BFR) code encodes M elements in F q (f ) to n codeword symbols (each in F α q ) that are grouped into b blocks such that f can be decoded by accessing to any k b−ρ nodes from each of the b − ρ blocks.
We remark that, in the above, ρ represents the resilience parameter of the BFR code, i.e., the code can tolerate ρ block erasures.Due to this data collection (file decoding) property of the code, we denote the number of blocks accessed as b c = b − ρ and number of nodes accessed per block as k c = k bc .Noting that k c ≤ c should be satisfied, we differentiate between partial block access, k c < c, and full block access k c = c.Throughout the paper, we assume b|n.i.e., c is integer, and (b − ρ)|k, i.e., k c is integer.
Remarkably, any MDS array code [34] can be utilized as BFR codes for the full access case.In fact, such an approach will be optimal in terms of minimum distance, and therefore for resilience ρ.However, for k c < c, MDS array codes may not result in an optimal code in terms of the trade-off between resilience ρ and code rate M nα .Concatenation of Gabidulin codes and MDS codes as originally proposed in [12] gives optimal BFR codes for for all parameters.For completeness, we provide this coding technique adapted to generate BFR codes in Appendix A. We remark that this concatenation approach is used for locally repairable codes in [12], for locally repairable codes with minimum bandwidth node repairs in [16], thwarting adversarial errors in [35], [36], cooperative regenerating codes with built-in security mechanisms against node capture attacks in [37] and for constructing PMDS codes in [38].In this work, we focus on repairable BFR codes, as defined in the following.

B. Information flow graph
The operation of a DSS employing such codes can be modeled by a multicasting scenario over an information flow graph [1], which has three types of nodes: 1) Source node (S): Contains original file f .2) Storage nodes, each represented as x i with two sub-nodes((x in i , x out i )), where x in is the sub-node having the connections from the live nodes, and x out is the storage sub-node, which stores the data and is contacted for node repair or data collection (edges between each x in i and x out i has α-link capacity).3) Data collector (DC) which contacts x out sub-nodes of k live nodes (with edges each having ∞-link capacity).(As described above, for BFR codes these k nodes can be any k b−ρ nodes from each of the b − ρ blocks.)Then, for a given graph G and DCs DC i , the file size can be bounded using the max flow-min cut theorem for multicasting utilized in network coding [1], [39].
where flow(S → DC i , G) represents the flow from the source node S to DC i over the graph G.
Therefore, M symbol long file can be delivered to a DC, only if the min cut is at least M. In the next section, similar to Dimakis et al., [1], we consider k successive node failures and evaluate the min-cut over possible graphs, and obtain file size bounds for DSS operating with BFR-RC.

C. Vector codes
An (n, M, d min , α) q vector code C ⊆ (F α q ) n is a collection of M vectors of length nα over F q .A codeword c ∈ C consists of n blocks, each of size α over F q .We can replace each αlong block with an element in F q α to obtain a vector c = (c 1 , c 2 , . . ., c n ) ∈ F n q α .The minimum distance, d min , of C is defined as minimum Hamming distance between any two codewords in C. Definition 4. Let c be a codeword in C selected uniformly at random from M codewords.The minimum distance of C is defined as where , and H(•) denotes q-entropy.
A vector code is said to be maximum distance separable (MDS) code if α | log q M and d min = n − logqM α + 1.A linear (n, M, d min , α) q vector code is a linear subspace of F αn q of May 18, 2016 DRAFT dimension M = log q M .An [n, M, d min , α] q array code is called MDS array code if α | M and d min = n − M α + 1.The encoding process of an (n, M = q M , d min , α) q vector code can be summarized by The encoding function is defined by an M × nα generator matrix

D. Maximum rank distance codes
Gabidulin codes [40], are an example of class of rank-metric codes, called maximum rank distance (MRD) codes.(These codes will be utilized later in the sequel.) Let F q m be an extension field of F q .An element v ∈ F q m can be represented as the vector , for a fixed basis {b 1 , . . ., b m } of the extension field F q m .Using this, a vector v = (v 1 , . . ., v N ) ∈ F N q m can be represented by an m × N matrix V = [v i,j ] over F q , which is obtained by replacing each v i of v by its vector representation Definition 5.The rank of a vector v ∈ F N q m , rank(v), is defined as the rank of its m × N matrix representation V (over F q ).Then, rank distance is given by An [N, K, D] q m rank-metric code C ⊆ F N q m is a linear block code over F q m of length N with dimension K and minimum rank distance D. A rank-metric code that attains the Singleton bound D ≤ N − K + 1 in rank-metric is called maximum rank distance (MRD) code.Gabidulin codes can be described by evaluation of linearized polynomials.Definition 6.A linearized polynomial f (y) over F q m of q-degree t has the form where a i ∈ F q m , and a t = 0.

Process of encoding a message
Gabidulin code over F q m has two steps: Step 2: Evaluate f (y) at {y 1 , y 2 , . . ., y n } where each y i ∈ F q m , to obtain a codeword c = (f (y 1 ), . . ., f (y N )) ∈ F N q m .Remark 7.For any a, b ∈ F q and v 1 , v 2 ∈ F q m , we have May 18, 2016 DRAFT Remark 8. Given evaluations of f (•) at any K linearly independent (over F q ) points in F q m , one can reconstruct the message vector.Therefore, an [N, K, D] Gabidulin code is an MDS code and can correct any D − 1 = N − K erasures.

E. Locally repairable codes
Recently introduced locally repairable codes reduce repair degree by recovering a symbol via contacting small number of helper nodes for repair.
Remark 11.The last requirement in Definition 10 implies that each element j ∈ Γ(i) can be written as a function of any set of r elements in Γ(i)\ {j}.

III. FILE SIZE BOUND FOR REPAIRABLE BFR CODES
In this section, we perform analysis to obtain file size bounds for repairable BFR codes.We will focus on different cases in order to cover all possible values of parameters d r , k c , ρ and σ.
We denote the whole set of blocks in a DSS by B, where |B| = b.A failed node in block i can be recovered from any b − σ blocks.Denoting an instance of such blocks by B r i , we have This repair process requires a total number of d nodes and equal number of nodes from each block in B r i , d r = d b−σ is contacted.Data collector, on the other hand, connects to b − ρ blocks represented by B c to reconstruct the stored data.In this process, total number of k nodes are connected to retrieve the stored data and the same number of nodes from each block in B c , k c = k b−ρ are connected.In the following, we analyze the information flow graph for a total number of k c (b−ρ) failures and repairs and characterize the min-cut for the corresponding graphs to upper bound the file size that can be stored in DSS.
Our analysis here differs from the classical setup considered in [1], in the sense that the scenarios considered here require analysis of different cases (different min-cuts in information flow graph) depending on the relation between d r and k c (d r ≥ k c vs. d r < k c ).Before detailing the analysis for each case (in Section III-A and III-B), we here point out why this phenomenon occurs.First, consider the case d r ≥ k c , a failed node in block i requires connections from the blocks j ∈ B r i and that are previously failed and repaired and connected to DC.At this point, block i needs to contact some additional nodes in block j (d r − k c number of nodes) that may not be connected to DC.On the other hand, when d r < k c , then, for the min-cut, any failed node in block i would contact to d r nodes in block j, which are already connected to DC.Therefore, the storage systems exhibits different min-cuts depending on the values of d r , k c , ρ and σ.In the following, we will analyze each case separately, obtain the min-cuts as well as corresponding corner points in per-node storage and repair bandwidth trade-off.Section III-A focuses on d r ≥ k c case, where the number of helper nodes from each block is greater than or equal to the number of nodes connected to DC from each block.Section III-B details the remaining case, i.e., d r < k c .

A. Case I: d r ≥ k c
We first note that d ≥ k. (This follows similarly to the analysis for regenerating codes, as otherwise one can regenerate each node by contacting d < k nodes, and obtain the stored data with less than k nodes.) Since we only assume that d r ≥ k c , no relation between ρ and σ can be concluded.Accordingly, we further split our analysis into two cases as follows.
1) Case I.A: σ ≤ ρ: The first case we focus on is having σ ≤ ρ, which implies that the number of blocks participating to the repair process is greater than or equal to the number of blocks that are connected to DC, i.e., b − σ ≥ b − ρ.
Theorem 12.The upper bound on the file size when d r ≥ k c and σ ≤ ρ is given by for α = M k or α = dβ.
Proof: Denote the order of node repairs with corresponding (node) indices in the ordered set O, where |O| = k c (b − ρ).We will show that any order of failures results in the same cut value.We have This follows by considering that block i connects to i − 1 blocks, each having k c nodes that are repaired and connected to DC, which makes the cut value of (d − (i − 1)k c )β.We denote the stored data of node j in block l as X l,j and downloaded data to this node as R l,j .The set O includes an ordered set of node incides (i, j) contacted by DC, where the order indicates the sequence of failed&repaired process.(For instance, O = {(1, 1), (1, 2), (2, 3), • • • } refers to a scenario in which third node of second blocks is repaired after second node of first block.)We consider DC connecting to the nodes in where each block has repairs sequentially.Due to data reconstruction property, it follows that H(F|X O ) = 0. Accordingly, we have We consider the following bound on the latter term H(X O ) to obtain C O , i.e., "cut value" for the repair order given by O. (Note that the bounds given below correspond to introducing "cuts" in the corresponding information flow graph.)We denote as the nodes contacted in i-th block.We consider each repaired node contacts the previously repaired nodes in O. Accordingly, we consider the entropy calculation given by Therefore, as the bound on H(X (i,j) ) above holds for any j such that (i, The equivalence analysis above can be extended to any pair of orders.In the following, we show this through the following lemmas.We first observe that any order can be obtained from another by swapping adjacent failures.Lemma 13.Any given failure order O * can be obtained by permuting the elements in the order O, and the underlying permutation π O→O * operation can be decomposed into stages of swapping of adjacent elements. We provide an example here, the lemma above generalizes this argument to any pair of orders.Consider O = (1, 2, 3) and O * = (3, 2, 1), the permutation π O→O * is given by position mappings {1 → 3, 2 → 2, 3 → 1}.This can be obtained by composition of three permutations {1 → 1, 2 → 3, 3 → 2}, for swapping 2 and 3, {1 → 2, 2 → 1, 3 → 3}, for swapping 1 and 3, and {1 → 1, 2 → 3, 3 → 2}, for swapping 1 and 2.
Utilizing Lemma 13, it remains to show that the cut value remains the same if we interchange adjacent failures in any failure order O.We show that this holds in the following lemma.Lemma 14.For any order O swapping any adjacent failures does not result in min-cut value C O to change.
To show this, consider two orders O 1 and O 2 , where O 2 is obtained by swapping order of failures at locations j and j + 1, O 2 = π(O 1 ).Then, we can say that the cut values up to j − 1 and the cut values after j + 1 individually same for both O 1 and O 2 .Furthermore, there are two possible cases for swapped failures j and j +1; i) either both are from the same block, ii) they are from different blocks.For the former case, the swapping does not affect the cut values for failures j and j +1.In the latter, first note that when α = M k , there is no change in cut value, Combining two lemmas, we conclude that every order of failures has the same cut value as O, which is C O .
The key distinction to be made in our scenario is that all of the nodes that are failed and repaired are utilized in the repair process of the later failures, which is why the order of failures does not matter.Note that for the special case of k c = 1, the above bound reduces to the classical bound given in [1].We obtain the following corner points in the trade-off region.
Corollary 15.For d r ≥ k c and σ ≤ ρ, corresponding BFR-MSR and BFR-MBR points can be found as follows.
2) Case I.B: σ > ρ: For the case of having σ > ρ, we have Noting that the helper nodes are chosen from the ones that are already connected to DC, we consider that B c ⊃ B r .Here, the min-cut analysis similar to the one given in the proof of Lemma 17 is same as having a system with b = b − ρ, ρ = 0, σ = σ − ρ since the analysis for the file size bound only utilizes b−ρ blocks and repairs occur by connecting to a subset of these b−ρ blocks.That is, the remaining ρ blocks do not contribute to the cut value.Therefore, we conclude that Remark 18.We conjecture that the order given in the proof above corresponds to the order producing the min-cut.We verified this with numerical analysis for systems having small n values.Although a general proof of this conjecture is not established yet 2 , we were able to construct codes achieving the stated bound.Therefore we conjecture the following MSR/MBR points for this case.
Utilizing the bound given in Lemma 17, we obtain following result (proof is similar to Corollary 15 and omitted for brevity).
Conjecture 19.For d r ≥ k c and σ > ρ, corresponding BFR-MSR and BFR-MBR points can be found as follows.Assume that O i− contains d r failures from each of b − ρ blocks but corresponding C O i− is not the minimum.We have already shown that the failure order among the nodes in O i− does not matter as long as the list contains d r failures from b − ρ blocks.This follows from d r = k c case analyzed in Theorem 12. Assume an order spanning b−ρ+t blocks for t > 0. This ordering will include nodes that are not connected to DC and can be omitted.This means that an order that minimizes the cut value needs to contain a block that has at least d r + 1 failures.Denote such an ordering by O i−− , assume without loss of generality that this block j has t ≥ 1 additional failures.In such a case, we observe that the cut value for failed nodes in other blocks would not be affected by this change since each such node is already connected to d r nodes of block j.Furthermore, by removing a failed node of some other block (other than j) from O i− , the cut values corresponding to other nodes in the list would only increase since the failures can benefit at most d r − 1 nodes of that block as opposed to d r in O i− .Hence, in order to minimize C O i− , O i− needs to include exactly d r failures from each of b − ρ blocks.Also, it can be observed (α

C. BFR-MSR and BFR-MBR Points for Special Cases
The general case is analyzed in the previous section, and we here focus on special cases for BFR-MSR and BFR-MBR points.The first case analyzed below has the property that corresponding BFR-MSR codes, achieves both per-node storage point of MSR codes and repair bandwidth of MBR codes simultaneously when 2d k.
1) Special Case for I.B: ρ = 0, σ = 1, d r ≥ k c and b = 2: Consider the 2-block case (b = 2) as in Fig. 3, and assume 2|k.The file size M can be upper bounded with the repair procedure shown in Fig. 3, which displays one of the "minimum-cut" scenarios, wherein any two consecutive node failures belong to different blocks.Assuming d ≥ k 2 , we obtain Achieving this upper bound (12) with equality would yield maximum possible file size.One particular repair instance is shown in Fig. 3, and we note that the order of node repairs does not matter as the sum of the corresponding cut values would be the same with different order of failures as long as we consider connection from data collector to k 2 repaired nodes from each block.
For BFR-MSR point, α = α BFR-MSR = M k .In the bound (12), we then have α BFR-MSR ≤ (d − k 2 )β BFR-MSR .Achieving equality would give the minimum repair bandwidth for the MSR case.Hence, BFR-MSR point is given by Note that, this coincides with that of (7) where we set b = 2, ρ = 0 and σ = 1 therein.BFR-MBR codes, on the other hand, have the property that dβ = α with minimum possible dβ while achieving the equality in (12).Inserting dβ = α in (12), we obtain that This coincides with that of (8) where we set b = 2, ρ = 0 and σ = 1 therein.We now consider the case where 2 k (as compared to previous section where we assumed k c = k b−ρ ), and characterize trade-off points for all possible system parameters in this special case.First consider the special case of k = 3 and two different order of failures, one with first failure in first block, second failure in second block, third failure in first block and the other one with first and second failures from first block, third failure from second block.Accordingly, observe that the cuts as min(α, dβ) + 2 min(α, (d − 1)β) and 2 min(α, dβ) + min(α, (d − 2)β) respectively.For MSR case, first sum would require α = (d − 1)β, whereas second sum requires α = (d − 2)β, resulting in higher repair bandwidth.Henceforth, one needs to be careful even though cut values are same for both orders of failures in both MSR (3α) and MBR ((3d − 2)β) cases.
The corresponding trade-off point are summarized below by following analysis similar to the one above.
May 18, 2016 DRAFT Here, we compare γ BFR-MSR and γ MBR .We have min(γ k-odd BFR-MSR , γ k-even BFR-MSR ) ≥ γ MBR = 2Md k(2d−k+1) , and, if we have 2d − k 1, then γ k-odd BFR-MSR ≈ γ k-even BFR-MSR ≈ γ MBR .This implies that BFR-MSR codes with b = 2 achieves repair bandwidth of MBR and per-node storage of MSR codes simultaneously for systems with d 1.On Fig. 4a and 4b, we depict the ratio of γ k-odd BFR-MSR and γ k-even BFR-MSR to γ MBR respectively, where we keep k constant and vary d as 2k ≥ d ≥ k.Also, we only even d values are shown in both figures.It can be observed that ratio gets closer to 1 as we increase k.Next, we provide the generalization of critical points to b ≥ 2 case in the following.
We observe that γ BFR-MSR ≤ γ MSR = Md k(d−k+1) for b ≤ k, which is the case here as we assume b | k.Also, we have k+1 which is always true.Hence, γ BFR-MSR is between γ MSR and γ MBR , see Fig. 2.

A. Transpose code: b=2
Construction I (Transpose code): 2 , and placement of nα 2 symbols denoted by  (in block b = 1) stores symbols {x i,j : Note that, when the stored symbols in nodes of block 1 is represented as a matrix, the symbols in block 2 corresponds to transpose of that matrix.(We therefore refer to this code as transpose code.) Due to this transpose property, the repair of a failed node i in the first block can be performed by connecting all the nodes in the second block and downloading only 1 symbol from each node.That is, we have dβ = α.Consider now that the file size M = kd − ( k 2 ) 2 , and an [N = α 2 , K = M] MDS code is used to encode file f into symbols denoted with x i,j , i, j = 1, . . ., α.Here, BFR data collection property for reconstructing the file is satisfied, as connecting any k c = k 2 nodes from each block assures at least K distinct symbols.This can be shown as follows: Consider α×α matrix X, where i-throw, j-th column has the element x i,j .Rows of X correspond to nodes of block 1, and columns of X correspond to nodes of block 2. Any k 2 rows (or any k 2 columns) provide total of kα 2 symbols.And k 2 rows and k 2 columns intersect at k 2 symbols.Therefore, total number of symbols from any k 2 rows and k 2 columns is M. Note that the remaining system parameters are d r = n 2 ≥ k c , ρ = 0 and σ = 1.Henceforth, this code is a BFR-MBR code as the operating point in (19), is achieved with M = kd − ( k 2 ) 2 and dβ = α for β = 1 (scalar code).A similar code to this construction is Twin codes introduced in [24], where the nodes are split into two types and a failed node of a a given type is regenerated by connecting to nodes only in the other type.During construction of Twin codes, the message is first transposed and two different codes are applied to both original message and it's transposed version separately to obtain code symbols.On the other hand, we apply one code to the message and transpose resulting symbols during placement.Also, Twin codes, as opposed to our model, do not have balanced node connection for data collection.In particular, DC connects to only (a subset of k nodes from) a single type and repairs are conducted from k nodes.On the other hand, BFR codes, for b = 2 case, connects to k 2 nodes from each block and repairs are from any d nodes in other block.This construction, however, is limited to b = 2 and in the following section we propose utilization of block designs to construct BFR codes for b > 2.   Consider that the file f of size M is partitioned into 3 parts f 1 , f 2 and f 3 each of size M = M 3 .Each partition f i is encoded with an [ñ = 10, k = 4, d = 5, α, β] regenerating code C, where the resulting partition codewords are represented with P 1 = {p 1,1:ñ } for f 1 , P 2 = {p 2,1:ñ } for f 2 , and P 3 = {p 3,1:ñ } for f 3 .These symbols are grouped in a specific way and placed into nodes within blocks as represented in Fig. 6b, where each node contains two symbols each coming from two different partitions.We set the BFR code parameters as Assume block b = 1 (denoted as Block 1) is unavailable and its first node, which contains codeword c 1 , has to be reconstructed.Due to underlying regenerating code, contacting 5 nodes of Block 2 and accessing to p 1,6:10 regenerates p 1,1 .Similarly, p 2,1 can be reconstructed from Block 3. Any node reconstruction can be handled similarly, by connecting to remaining 2 blocks and repairing each symbol of the failed node by corresponding d nodes in each block.As we have k = 6, DC, by connecting to 2 nodes from each block, obtains a total of 12 symbols, which consist of 4 different symbols from each of P 1 , P 2 and P 3 .As the embedded regenerating code has k = 4, all 3 partitions (f 1 , f 2 and f 3 ) can be recovered, from which f can be reconstructed.
In the following construction, we generalize the BFR-RC construction above utilizing projective planes for the case of having ρ = 0, σ = 1.As defined in Section III, this necessarily requires d r > k c .)We first introduce projective planes in the following and then detail the code construction.
Definition 23 (Balanced incomplete block design [41]).A (v, κ, λ)-BIBD has v points distributed into blocks of size κ such that any pair of points are contained in λ blocks.
Corollary 24.For a (v, κ, λ)-BIBD, In the achievable schemes of this work, we utilize a special class of block designs that are called projective planes [41].Projective planes have the property that every pair of blocks intersect at a unique point (as λ = 1).In addition, due to Corollary 24, in projective planes, every point occurs in r = p + 1 blocks, and there are b = v = p 2 + p + 1 blocks.Construction II (Projective plane based placement of regenerating codes): v , is then encoded using [ñ, k, d, α, β] regenerating code C. We represent the resulting partition codewords with P i = p i,1:ñ for i = 1, . . ., v. We then consider index of each partition as a point in a (v = p 2 + p + 1, κ = p + 1, λ = 1) projective plane.(Indices of symbol sets P J and points J of the projective plane are used interchangeably in the following.)We perform the placement of each symbol to the system using this projective plane mapping.(The setup in Fig. 6b can be considered as a toy model.Although the combinatorial design with blocks given by {p 1 , p 2 }, {p 3 , p 1 }, {p 3 , p 2 } has projective plane properties with p = 1, it is not considered as an instance of a projective plane [41].)In this placement, total of ñ symbols from each partition P i are distributed to r blocks evenly such that each block contains ñ r nodes where each node stores α = κα symbols.Note that blocks of projective plane give the indices of partitions P i stored in the nodes of the corresponding block in DSS.That is, all nodes in a block stores symbols from unique subset of P = {P 1 , . . ., P v } of size κ.(For instance, in Fig. 6b, the first block of the block design has part {p 1 , p 2 }, and accordingly symbols from partitions P 1 and P 2 are placed into node of Block 1.) Here, as each point in the block design is repeated in r blocks, the partition codewords span r blocks.Overall, the system can store a file of size M = v M with b = v blocks.(Note that, r = κ = p + 1 and b = v = p 2 + p = 1 for projective planes.See Definition 25.)We set the parameters as where we choose parameters to satisfy r−1 | d, r | ñ (for splitting partition codewords evenly to r blocks) and r | k (for data collection as detailed below).We have as d ≥ k and hence the required condition d r > k c is satisfied Node Repair: Consider that one of the nodes in a block is to be repaired.Note that the failed node contains κ symbols, each coming from a distinct partition.Using the property of projective planes that any 2 blocks has only 1 point in common, any remaining block can help for in the regeneration of 1 symbol of the failed node.Furthermore, as any point in the block design has a repetition degree of r, one can connect to r − 1 blocks, d r = d r−1 nodes per block, to repair one symbol of a failed node.Combining these two observations; we observe that node regeneration can be performed by connecting (r − 1)κ blocks.In particular, substituting r = κ = p + 1, we see that connecting to p 2 + p = b − 1 blocks allows for reconstructing of any node of a failed block.
Data Collection: DC connects to k c = k r nodes per block from all b c = b blocks, i.e., a total of k = b r k = v r k nodes each having encoded symbols of κ = r partitions.These total of v k symbols include k symbols from each partition, from which all partitions can be decoded, and hence the file f , can be reconstructed.
Remark 26.We note that, when ñ = r in Construction II, each node in the system stores different partition subsets of P = {P 1 , • • • , P v }.This translates into having c = 1 node per block.This special case of proposed construction is equivalent to layered codes studied in [26].
In that work, layering helps to construct codes with exact repair properties.In Construction II, on the other hand, multiple nodes in the system can have the same type (representing the same subset of partitions), and this enables to achieve different operating points for the block failure model.

1) BFR-MSR:
To construct a BFR-MSR code, we set each sub-code C in Construction II as an MSR code, which has This, together with (20), results in the following parameters of our BFR-MSR construction We remark that if we utilize ZigZag codes [2] as the sub-code C above, we have and having d = ñ − 1 requires connecting to 1 node per block for repairs in our block model.In addition, product matrix MSR codes [3] require d ≥ 2 k − 2, and they can be used as the sub-code C, for which we do not necessarily have d = r − 1.We observe from ( 18) and ( 22) that the BFR-MSR point is achieved for k = p + 1, implying k = b, i.e.DC connects necessarily 1 node per block for data reconstruction when our Construction II gives BFR-MSR code.
2) BFR-MBR: To construct a BFR-MBR code, we set each sub-code C in Construction II as a product matrix MBR code [3], which has This, together with (20), results in the following parameters of our BFR-MBR construction From ( 19) and ( 24), we observe that the BFR-MBR point is achieved for k = p + 1.
C. Duplicated Block Design Based BFR Codes (ρ = 0 and σ < b − 1) In this section, BFR codes for special case of having ρ = 0 is constructed.We note that ρ = 0 implies that DC contact all b blocks to retrieve the stored data.Before detailing the code construction, we first introduce a block design referred to as duplicated combination block design [26].
Definition 27 (DCBD).Let (κ, ṽ) denote the parameters for a block design, where ṽ points from all possible sets of blocks each with κ points.Then, duplicated combination block design (DCBD) (with repetition r ) is a block design where the given block design is duplicated r times with different labeling points.(Here, total of v = rṽ points are splitted into r groups of ṽ points, where each group generates sub-blocks according to the given block design.) Example 28.DCBD with ṽ = 5, κ = 4 and r = 3 is given below.
It can be observed that each sub-block consists ṽ κ blocks, each containing a different set of κ points.Also, the same combination is repeated r times (with different labels for points, namely {6, 7, 8, 9, 10} and {11, 12, 13, 14, 15}).Each row here corresponds to a block of DCBD, where sub-blocks aligned similarly in columns represent the underlying (κ, ṽ) block design.We refer to the sub-blocks as repetition groups in the following.
The following example (with b = 5 and σ = 2) illustrates a repair scenario.Assume that the failed node is in the first block and it will be regenerated by blocks 2,3 and 4 (as b − σ = 3).Considering the first repetition group, it can be observed that symbols of each of b − σ = 3 partitions (P 3 , P 4 and P 5 ) can be found in b − σ − 1 = 2 of these blocks, whereas the remaining σ − 1 = 1 partition (P 1 ) can be found in in all b − σ blocks.(In the representation below, numbers represent the indices for partitions P i .And, the three highlighted rows for the first repetition group includes partitions P 1 , P 3 , P 4 , P 5 that are relevant to the symbols stored in the block corresponding to the first row below.) In order to have a successful regeneration for each node to be regenerated, we also require the following condition in this construction.= k α symbols per partition is collected, from which each partition can be decoded via underlying sub-code C, and the stored file f can be reconstructed.

1) BFR-MSR:
To construct a BFR-MSR code, we set each sub-code C in Construction III as an MSR code, which has This, together with (26), results in the following parameters of our BFR-MSR construction From (7), we obtain that Construction III results in optimal BFR-MSR codes when k = b σ (i.e., k = b−1 σ ).

2) BFR-MBR:
To construct a BFR-MBR code, we set each sub-code C in Construction III as a product matrix MBR code [3], which has This, together with (26) results in the following parameters of our BFR-MBR construction

A. Locality in BFR
In this section, we focus on BFR model with repair locality constraints, i.e., only a local set of blocks are available to regenerate the content of a given node.This model is suitable for both disk storage and distributed (cloud/P2P) storage systems.For example, a typical data center architecture includes multiple servers, which form racks which further form clusters, see Fig. 8.We can think of each cluster as a local group of racks where each rack contains multiple servers.Hence, in Fig. 8, we can model the data center as having b blocks (racks) where b L blocks form a local group (cluster) and there are c nodes (servers) in each block.
In this section, we extend our study of data recovery with the block failure model to such scenarios with locality constraints.We assume that DSS maintains the data of size M with at most ρ blocks being unavailable.Hence, from any b − ρ blocks, data collection can be performed by contacting any k c nodes from each of such blocks.In other words, DC can contact some set of local groups, B * , to retrieve the file stored in the DSS with |B * | = b − ρ.Let c i,j denote the part of the codewords associated with i th local group's j th block accessed by DC, which consists of k c nodes.Note that, we consider k c = c for full access and k c < c for partial access as before.Then, we can denote by c i the codeword seen by DC corresponding to local group B i , which has a size j ≤ b L .Therefore, ) denotes the codeword corresponding to the one seen by DC, when accessing k c (b − ρ) = k nodes.Definition 30.Let c be a codeword in C selected uniformly, the resilience of C is defined as We remark that resilience ρ of locally repairable BFR code dictates the number of block failures that the system can tolerate (analogous to minimum distance providing the maximum number of node failures).May 18, 2016 DRAFT scenario corresponds to Thus, the worst case happens when N K L − α 1 − 1 local groups with all of their blocks erased and one additional local group with b L − β 1 − 1 blocks erased.Then, that scenario corresponds to Remark 37. We note that the above construction is similar to that of [12], here modified for the block failure model to achieve resilience optimal construction with improved rate.

C. Local Regeneration for BFR Codes
In the previous section, MDS codes are utilized in local groups.These codewords, however, are not repair bandwidth optimal.As an alternative, regenerating codes can be used in local groups to construct codes which have better trade-off in terms of repair bandwidth.Differentiating between the two important points, we denote BFR-LRC codes with regenerating code properties which operate at minimum per-node storage point as BFR-MSR-LRC.Similarly, the codes operating at minimum repair bandwidth point are called BFR-MBR-LRC.
Let Definition 38 (Uniform rank accumulation (URA) codes).Let G be a generator matrix for a BFR code C. Columns of G produce the codewords henceforth we can think of each α columns (also referred to as thick column) of G as a representation of a node storing α symbols.Then, a block can be represented by c such thick columns.Let S i be an arbitrary subset of i such blocks.C is an URA code, if the restriction G| S i of G to S i , has rank ρ i that is independent of specific subset S i and it only depends on |S i |.
Remark 39.Generally, URA codes are associated with rank accumulation profile to calculate the rank of any subset S i .However, rank accumulation profile is not required but rather rank accumulation is enough for a code to be considered as URA code.We note that for BFR-MSR/MBR codes, we do not have a specific rank accumulation profile but that does not rule out BFR-MSR/MBR codes being URA since they still obey the rank accumulation property.Specifically, we note that H(b S i ) = f (|S i |) for both BFR-MSR/MBR, which makes them URA codes.
Following similar steps as introduced in [16], resilience upper bound can be derived when local codes are URA codes.Consider the finite length vector (b 1 , . . ., b b L ) and its extension with b L period as b i+jb L = b i , 1 ≤ i ≤ b L , j ≥ 1.Let H(b S ) denote the entropy of set of blocks S, For integers µ ≥ 0 and 1 ≤ φ ≤ b L , let H(µb L + φ) = µK L + H(φ).(Note that, due to URA property, entropy is only a function of number of blocks here.)For the inverse function H (inv) , we set H (inv) (ϕ) for ϕ ≥ 1, to be largest integer S such that H(b S ) ≥ ϕ.Then, we have for μ ≥ 0 and where Theorem 40.The resilience of BFR-LRC is upper bounded by When URA codes are used as local codes, we have the following file size bound for resilience optimal codes Since BFR-MSR is a class of URA codes, we can upper bound the resilience of BFR-MSR-LRC as follows, where for BFR-MSR codes we have for some µ ≥ 0 and 1 ≤ φ ≤ K L and ϕ is determined from Then, we can derive the file size bound for optimal BFR-MSR-LRC as M ≤ H(b − ρ), i.e.; where 2) Local Regeneration with BFR-MBR Codes: In the following, we will focus on the case where the local groups form BFR-MBR codes.When BFR-MBR codes are utilized, the dimension of the local code is given by Using the fact that BFR-MBR is URA code, the upper bound on the resilience of BFR-MBR-LRC is given by where for BFR-MBR codes we have for some µ ≥ 0 and 1 ≤ φ ≤ K L and ϕ is determined from Now, the file size bound for an optimal BFR-MBR-LRC is given by where switches.)Since in schemes that uses BFR a failed node requires d r β BFR amount of data from each of b−σ blocks, the repair delay (normalized with file size M) can be calculated as follows.
RT BFR = max where BW i is the bandwidth for block i and B h is the set of blocks that help in the repairs.We assume the repairs through blocks are performed in parallel.Throughout this section, we'll assume that all bandwidths are identical (BW i = BW, ∀i), hence the repair time of a failed node is given by drβ BFR MBW .In an identical setting, we can also analyze the repair delay of regenerating codes.Note that regenerating codes do not require any symmetric distribution of helper nodes among blocks.Hence repair delay of regenerating codes is, where a system s refers to a selection of d i (the number of helper nodes in block i) such that the sum of helper nodes is equal to d.In other words, repair delay will be affected by the block with the most helper node.Furthermore, the average repair delay can be calculated as We may encounter some d values, which are not attainable in the BFR model since for BFR codes we require d ≤ n − c because of the assumption that a failed node does not connect any nodes in the same block.Furthermore, to compare the codes in a fair manner, we assume that regenerating codes connect any d from b − σ blocks.Henceforth, in our comparisons, we instead calculate repair delay for regenerating codes as ) where the helper nodes are chosen from b − σ blocks but the number of helpers in each block is not necessarily the same, only requirement is to have the sum of the helpers equal to d.Average repair time can be calculated similar to (52) where the difference being the system s satisfies the condition that helpers are chosen from b − σ blocks.
One can allow symmetric distribution of regenerating codes among d − σ blocks as well, similar to BFR codes.We'll denote these codes by MSR-SYM or MBR-SYM in the following.
We examine the case where b = 7 and n = 21, which means each block contains c = 3 nodes.We also set σ = 3 so that b − σ = 4.Note that since we are comparing relative values, the values we assign to BW and M does not change the result as long as they are same across all comparisons.In other words, one can think our results as normalized repair delays.At first, we find out all possible ρ, d r , k c , d and k values accordingly from Section III.After identifying all possible parameter sets, we calculate repair times for both BFR-MSR and BFR-MBR.For regenerating codes, we choose all possible parameter sets as long as d ≤ n−σc since we impose regenerating codes to perform repairs from b − σ blocks.For each parameter set, we calculate repair time of all possible helper node allocations and then calculate the mean of those.The mean is reported for each data point in Fig. 9 for one set of parameters for both MSR and MBR codes.Finally, we allow symmetric distribution of helper nodes among b − σ blocks, and report MSR-SYM and MBR-SYM.Repair delay vs. storage overhead results are depicted on Fig. 9.In Fig. 9a we indicate all data points, whereas in Fig. 9b, lower envelope when zoomed in to the storage overheads less than 13.As expected, one can observe that there are more data points for regenerating codes than BFR, since BFR requires b−σ|d, which limits the number of possible sets of parameters for BFR.First, lower envelopes of MSR and MSR-SYM are the best repair times for MSR is achieved when d gets it's highest possible value, (b − σ)c.In other words, all nodes are utilized hence there is no different possible connection schemes for MSR.Interestingly, given storage overhead, it can be observed that in some cases MSR codes perform better than MBR codes eventhough MBR codes minimizes repair bandwidth.On the other hand, when distributed symmetrically across blocks, MBR-SYM outperforms both MBR and MSR-SYM in all cases.When we compare BFR-MSR and BFR-MBR, we can observe that BFR-MBR has lower repair delay for all cases but still they perform same when storage overhead gets larger.Furthermore, we observe that unlike MBR-MSR comparison, BFR performs more regularly, meaning BFR-MBR is better than BFR-MSR always.Next, if we compare all schemes, it can be observed that convex hulls of BFR-MBR, MBR and MBR-SYM follows the same line.Note that repair delay of BFR-MSR is below MSR and it performs same as storage overhead increases.Also, BFR-MBR outperforms MSR-SYM  and performs identical to MBR-SYM.For lower storage overheads, we can observe that BFR codes operate well (both BFR-MSR and BFR-MBR) whereas existing RC or RC-SYM codes (MBR and MBR-SYM codes do not even exist for overhead below 3.23) performs worse than BFR.
Finally, note that within BFR schemes, we may encounter different α and β values depending on the parameters.Differentiating between cases, let BFR1 denote the schemes with d r ≥ k c and ρ ≥ σ, BFR2 denote d r ≥ k c and ρ < σ, and BFR3 denote d r < k c and ρ ≥ σ.The resulting β values for these schemes may not be the same for same k and d.In Fig. 10, we examine these BFR schemes (for storage overhead less than 10).We observe that the convex hull for BFR-MBR is a line and all BFR-MBR schemes operate on that line.Furthermore,different MSR schemes can perform better depending on the storage overhead.

B. Relaxed BFR
It's challenging to find general constructions that allow for repair and data collection schemes to operate with any ρ and σ blocks with any subset of d r or k c .Hence, in this section, we relax "any" requirement of BFR to table-based recovery schemes that guarantee an existence of helper/data recovery nodes for every possible repair/data collection scenarios.By altering this, we are able to use another combinatorial design and obtain relaxed BFR codes (R-BFR) for wider set of parameters. 3) Resolvable Balanced Incomplete Block Design (RBIBD): Definition 44.A parallel class is the set of blocks that partition the point set.A resolvable balanced incomplete block design is a (v, κ, λ) − BIBD whose blocks can be partitioned into parallel classes.
An example of (9, 3, 1) − RBIBD is given below, where each column (consisting of 9 points in 3 blocks) forms a parallel class.
2) R-BFR-RC with RBIBD: In this section, we show that RBIBDs can be used to construct R-BFR codes for any ρ ≥ 0 and σ ≥ 1. Considering RBIBDs (with λ = 1) as defined above, we construct blocks each containing the same number of nodes that store symbols belonging to different partitions.For instance, utilizing (54), a block can be formed to contain nodes, where 4 nodes store symbols in the form of {1, 2, 3}, (referred to as "type" below), 4 nodes store symbols of the type {4, 5, 6} and the other 4 nodes store symbols of the type {7, 8, 9}.We refer to blocks of the same type as sub-block.Assume that one of the nodes that store the symbols of the type {1, 2, 3} is failed.A newcomer may download symbols from any other subset of blocks but instead of connecting to any d r nodes from each block, we consider connecting to any dr 3 nodes of type {1, 4, 7}, any dr 3 nodes of type {2, 5, 8} and any dr 3 nodes of type {3, 6, 9} in the second block and so on. 4Similarly, DC can connect any kc 3 from each sub-blocks.Therefore, requirement of any set of nodes from a block is changed to any subset of nodes from a sub-block.Note that, RBIBD still preserves any ρ and σ properties.
In the general case of any ρ and σ, we still have the same relationship as before, c.f., ( With this construction, the same steps provided for BFR-MSR and BFR-MBR cases in the previous section can be followed.In the general case, we have three cases depending on the values of d r , k c , σ and ρ and the corresponding cut values can be found using Theorems 12, 20, and Lemma 17. Solving for these minimum storage and bandwidth points, optimal k values can be found.We provide these result in the following subsections. Using the relationships (56) and (57) together with (4), ( 7) and ( 10) we obtain the optimal as This together with (55) results in the following parameters for our R-BFR-MBR construction Similar to above, we can solve for optimal value in MBR case using (59) and (60) together with (5), ( 8) and (11)

VII. CONCLUSION
We introduced the framework of block failure resilient (BFR) codes that can recover data stored in the system from a subset of available blocks with a load balancing property.Repairability is then studied, file size bounds are derived, BFR-MSR and BFR-MBR points are characterized, explicit code constructions for a wide parameter settings are provided for limited range of σ and ρ.We then analyzed BFR for broader range of parameters and characterized the file size bounds in these settings and also proposed code constructions achieving the critical points on the tradeoff curve.Locally repairable BFR codes are studied where the upper bound on the resilience of DSS is characterized and two-step encoding process is proposed to achieve optimality.We finally analyzed repair delay of BFR codes and compare those with regenerating codes, and provide constructions with table-based repair and data recovery properties.
Constructions reported here are based on combinatorial designs, which necessitate certain parameter sets.As a future work, general constructions for the BFR mode can be studied further.Also, the file size bound for the case of having d r ≥ k c and σ > ρ is established here, and we conjecture that this expression corresponds to the min-cut.The proof for this conjecture resisted our efforts thus far.Furthermore, repair delay with a uniform BW assumption is studied here.Different bandwidth and more realistic communication schemes (e.g., queuing models) can be be studied.Further, system implementations can be performed and more realistic analysis can be made for disk storage and distributed (cloud/P2P) storage scenarios.
• Split resulting N symbols {x 1 , . . ., x N } into b blocks each with k c symbols.We represent this operation by double indexing the codeword symbols, i.e., x i,j is the symbol at block i and j for i = 1, . . ., b, j = 1, . . ., k c .We also denote the resulting sets with the vector notation, x i,1:kc = (x i,1 , x i,2 , . . ., x i,kc ) for block i.
• Use an [n = c, k = k c , d] MDS array code for each block to construct additional parities.
Representing the output symbols as y i,1:c we have y i,1:c = x i,1:kc G MDS for each block i, where G MDS is the encoding matrix of the MDS code over F q .For instance, if a systematic code is used, x i,1:kc is encoded into the vector y i,1:n = (x i,1 , . . ., x i,k C , p i,1 , . . ., p i,c−kc ) for each block i = 1, . . ., b.
In the resulting code above if one erases ρ blocks and any c − k c symbols from the remaining blocks, the remaining (b − ρ)k c symbols form linearly independent evaluations of the underlying linearized polynomial which can be decoded due to to the Gabidulin code from which the data symbols can be recovered and hence, by re-encoding, the pre-erasure of the version of the system can be recovered.

Proof:
In order to get an upper bound on the resilience of an BFR-LRC, the definition of resilience given in (33) is utilized similar to proof in [9], [10].We iteratively construct a set B * ⊂ B so that H(c(B * )) < M. The algorithm is presented in Fig. 11.Let b i and h i represent the number of blocks and entropy included at the end of the i-th iteration.We define and Assume that the algorithm outputs at (l + 1) th iteration then it follows from (62) and (63) that The analysis of algorithm is divided into two cases as follows.
• Case 1: [Assume that the algorithm exits without ever entering line 7.] We have

Fig. 2 :
Fig. 2: (a) Node repair in regenerating codes.(b) Node repair in relaxation of regenerating codes.(c) Trade-off curves in toy example.

Definition 2 (
Block Failure Resilient Regenerating Codes (BFR-RC)).An (n, b, M, k, ρ, α, d, σ, β) block failure resilient regenerating code (BFR-RC) is an (n, b, M, k, ρ, α) BFR code (data collection property) with the following repair property: Any node of a failed block can be reconstructed by accessing to any d r = d b−σ nodes of any b r = b − σ blocks and downloading β symbols from each of these d = b r d r nodes.We assume (b − σ)|d, i.e., d r is integer.(Note that d r should necessarily satisfy d b−σ = d r ≤ c = n b in our model.)We consider the trade-off between the repair bandwidth γ = dβ and per node storage α similar to the seminal work of Dimakis et al. [1].In particular, we define α BFR-MSR = M k as the minimum per node storage and γ BFR-MBR = α BFR-MBR as the minimum repair bandwidth for an (n, b, M, k, ρ, α, d, σ, β) BFR-RC.

Definition 9 (
Punctured Vector Codes).Given an (n, M, d min , α) q vector code C and a set S ⊂ [n], C| S is used to denote the code obtained by puncturing C on [n]\S.In other words, codewords of C| S are obtained by keeping those vector symbols in c = (c 1 , . . ., c n ) ∈ C which have their indices in set S.
j) ∈ O (i) , we have (b), from which we obtain the following bound on file size M M ≤ b−ρ i=1 k c min {α, (d − (i − 1)k c )β} .Interchanging the failure order in O at indices k c and k c + 1, we obtain another order O * .Using the same bounding technique above, the resulting cut value can be calculated as C O * = (k c − 1) min {α, dβ} + min {α, (d − (k c − 1))β} + min {α, (d − 1)β} + (k c − 1) min {α, (d − k c )β} + b−ρ i=3 k c min {α, (d − (i − 1)k c )β}.Note that the first term here corresponds to the first set of k c − 1 nodes in the first blocks, second term corresponds to the first node repaired in the second block, third term corresponds to the remaining node in the first block, fourth terms corresponds to remaining k c − 1 nodes in the second block, and the last term corresponds to the remaining blocks.We observe that C O = C O * , i.e., the total cut values induced by O and O * are the same.Consider C O * − C O , which evaluates to C O * −C O = min {α, (d − 1)β}−min {α, dβ}+min {α, (d − (k c − 1))β}−min {α, (d − k c )β} .We observe that C O * − C O = 0 at both α = M k and α = dβ operating points.(For α = M k , all min {} terms result in α and cancel each other.For the α = dβ point, min {} terms remove α (as α = dβ) and remaining terms cancel each other.) hence we'll only focus on α = dβ case.Assume for the O 1 we have (d−i)β for i th and (d−i * )β for (i + 1) th failures.Then, O 2 should have (d − (i * − 1))β and (d − (i + 1))β respectively.Note that the sums are still the same, (d − i)β + (d − i * )β = (d − (i * − 1))β + (d − (i + 1))β, hence we can conclude that swapping any two adjacent failures does not change the min-cut value,

Lemma 17 .
The upper bound on the file size when d r ≥ k c and σ > ρ is given byM ≤ b−σ i=1 k c min {α, β(d − (i − 1)k c )} + b−ρ i=b−σ+1 k c min {α, β(d − (b − σ)k c )} .(6)Proof: Let O be an order such that first k c failures occur in the first block, next k c failures occur in the second block and so on.Denote by C O the total cut value induced by O. (The analysis detailed in the proof of Theorem 12 is followed here.)Consider the node indexed by i = k c (b − σ) so that order O can be split into two parts as O i− and O i+ , where O i− represents the failures up to (and including) the node i, and O i+ represents the remaining set of failures.Since any node failure contacts to b − σ blocks and noting that O i− includes exactly b − σ blocks, any node failure in O i+ would contribute to the cut value as min {α, (d − (b − σ)k c )β}.On the other hand, C O i− follows from Theorem 12. Combining C O i− and C O i+ , we get (6).
) B. Case II: d r < k c In this case, having d r < k c necessarily implies b − σ > b − ρ ⇐⇒ ρ > σ, as d ≥ k for regenerating codes (as discussed in previous section).Therefore, we only focus on ρ > σ case in this section.Theorem 20.The optimal file size when d r < k c and ρ > σ is given by M ≤ b−ρ i=1 d r min {α, (d − (i − 1)d r )β} + b−ρ i=1 (k c − d r ) min {α, (d − (b − ρ − 1)d r )β} .(9) Proof: Let O be an order and let index i = d r (b − ρ) so that order O can be split into two parts as O i− and O i+ where O i− represents the failures up to (and including) index i and O i+ represents the remaining set of failures.For index i, C O i− takes its minimum possible value if O i− contains exactly d r failures from each of b − ρ blocks.We show this by a contradiction.

•
Construction III (DCBD based BFR-RC): Consider a file f of size M.• Divide M into (b − σ) b b−1 parts of equal size M, i.e., M(b − σ)b = M. • Encode each part f i using an [ñ, k = M, d]regenerating code (referred to as the sub-code C).Place the resulting partition codewords according to DCBD design (with ṽ = b, κ = b − 1 and r = b − σ) such that each block has c = ñ b−1 nodes, where each node stores κ = κr symbols, each coming from a different partition.Overall, the system stores a file of size M = b(b − σ) M over b blocks.We set the parameters as

27 )
Node Repair: Generalizing above argument, consider that one of the nodes in a block is to be repaired by contacting to b − σ blocks.A failed node contains κ = (b − 1)(b − σ) symbols, each coming from a distinct partition codeword.The properties of underlying (DCBD) block design, (considering the first repetition group), implies that there exists b − σ partitions of the failed node that are contained only in b − σ − 1 of the blocks contacted for repair.The remaining b − 1 − (b − σ) = σ − 1 partitions (of each repetition group) are contained in all of the contacted b − σ blocks.These partitions are referred to as the common partitions of a repetition group in b − σ contacted blocks.(In the example above, partitions P 1 , P 6 , P 11 are the common partitions for the first, second and third repetition group respectively.)Using this observation for DCBD based construction, (i.e., considering all repetition groups), consider obtaining σ − 1 common partitions from b − σ − 1 blocks of each of the r = b − σ repetition groups.In addition, consider obtaining remaining relevant partitions (b − σ partitions per repetition group) from these r = b−σ partition groups (total of r(b−σ)(b−σ −1) partitions).These σ − 1 common partitions per repetition group over r = b − σ repetition groups are contacted evenly.Namely, each other relevant partition are contacted from only b − σ − 1 blocks, the common partitions among b − σ blocks are contacted only b − σ − 1 times (i.e., by not contacting to any common point at all in only one repetition group from a block and since there are b − σ repetition groups and b − σ contacted blocks, we can do this process evenly for all blocks).Henceforth, from each block same amount of symbols (and same amount of symbols from each partition) is downloaded.In total, there are (σ − 1)(b − σ)(b − σ − 1) common points and each block contributes the transmission of (σ − 1)(b − σ − 1) common partitions.Hence,

Lemma 29 .
Construction III requires the necessary condition ñ b−1 ≥ d b−σ−1 for repair feasibility.Proof: Given v − 1 combinations of v points, any two combinations differs only in one point.Also, any combination is missing only one point.If one collects b − σ ≥ 2 of such combinations in Construction III, then the partition with least number of instances is b − σ − 1. (This follows as the first combination is missing only one partition which is necessarily included in the second combination.)Then, by contacting any b − σ blocks, one can recover partitions of failed node from these b − σ blocks.(There exist at least b − σ − 1 number of blocks containing nodes storing symbols from a given partition.)Since each block has c = ñ b−1 symbols from each partition, we require ñ b−1 ≥ d b−σ−1 to have repair feasibility in Construction III.Therefore, a failed node can be regenerated from d = d(b−σ) b−σ−1 nodes and downloading β = (b − σ − 1)(b − 1) β symbols from each block.(Note that, d r = d b−σ = d b−σ−1 .)For each partition of the failed node, d β symbols are downloaded, from which one can regenerate each partition.Note that, repeating combinations multiple times enables us to have uniform downloads from the nodes during repairs.Data Collection: DC connects to k c = k b = k b−1 nodes per block (as ρ = 0), and downloads total of k c α symbols from each block.These symbols include k α b−1 symbols from each of (b−1)(b−σ) partitions.Therefore, from all blocks, (b−1) k α b−1

Fig. 8 : 2 b+σ 2 − 1 (
Fig. 8: Data center architecture.Each rack resembles a block and each cluster forms a local group for node repair operations in the BFR model.
G 1 , . . ., G b b L represent the disjoint set of indices of blocks where G i represents local group i, which has b L blocks.A failed node in one of the blocks in G i is repaired by contacting any b L − σ L blocks within the group.A newcomer downloads β symbols from d L b L −σ L nodes from each of b L − σ L blocks.That is, the local group has the properties of repair bandwidth efficient BFR codes as studied in Section III.
where µ = b−ρ b L and φ = b − ρ − µb L .Note that if φ ≥ min {b L − ρ L , b L − σ L }, then H( φ i=1 b i ) = K L since from any such φ blocks, one can regenerate all symbols or retrieve the content stored in the corresponding local group.Therefore, the case of having φ ≥ min{b L − ρ L , b L − σ L } results in M = (µ + 1)K L and we will mainly focus on the otherwise in the following.1) Local Regeneration with BFR-MSR Codes: At first, we analyze the case where BFR-MSR codes are used inside local groups to have better trade-off in terms of repair bandwidth.When BFR-MSR codes are used, dimension of local code is given by

Fig. 9 :
Fig. 9: Repair time vs. storage overhead comparisons for b = 7, n = 21 and σ = 3.(a) Data points for possible cases of each node.(b) Lower envelope of Fig. 9a when zoomed in.