Approximate Gradient Coding with Optimal Decoding

In distributed optimization problems, a technique called gradient coding, which involves replicating data points, has been used to mitigate the effect of straggling machines. Recent work studied approximate concerns coding schemes where replication factor is too low recover full exactly. Our motivated by challenge creating that simultaneously well in both adversarial and stochastic models. To end, we introduce novel codes based on expander graphs, each machine receives exactly two blocks points. We analyze decoding error random straggler setting, when optimal coefficients are used. show our achieve an decays exponentially factor. nearly smaller than any existing code with similar performance setting. convergence bounds setting for descent under standard assumptions using codes. rate improves upon block-box bounds. can converge down noise floor scales linearly gradient. demonstrate empirically near-optimal faster algorithms do not use coefficients.