| It has been demonstrated that higher order recurrent neu-ral networks exhibit an underlying fractal attractor as an artifact of their dynamics. These fractal attractors ooer a very eecent mechanism to encode visual memories in a neu-ral substrate, since even a simple twelve weight network can encode a very large set of diierent images. The main problem in this memory model, which so far has r...

Method One of the obstacles in designing gradient coil is the general discretization procedure that approximates the continuous current densities by a number of current carrying conductors. The shape of the conductors’ layout determines the properties of the gradient coil such as the gradient strength, field quality characteristics inside the FoV, level of shielding, coil resistance, slew rate,...

On modern large-scale parallel computers, the performance of Krylov subspace iterative methods is limited by global synchronization. This has inspired the development of s-step (or communication-avoiding) Krylov subspace method variants, in which iterations are computed in blocks of s. This reformulation can reduce the number of global synchronizations per iteration by a factor of O(s), and has...

In the paper, we present an algorithm framework for the more general problem of minimizing the sum f(x) + ψ(x), where f is smooth and ψ is convex, but possible nonsmooth. At each step, the search direction of the algorithm is obtained by solving an optimization problem involving a quadratic term with diagonal Hessian and Barzilai-Borwein steplength plus ψ(x). The method with the nomonotone line...

In mathematics, the conjugate gradient method is an algorithm for the numerical solution of particular systems of linear equations, namely those whose matrix is symmetric and positive-definite. The conjugate gradient method is often implemented as an iterative algorithm, applicable to sparse systems that are too large to be handled by a direct implementation or other direct methods such as the ...

We propose two numerical methods, namely the block relaxation and majorization method, for the problem of nearest correlation matrix with factor structure, which is highly nonconvex. In the block relaxation method, the subproblem is of the standard trust region problem, which is solved by Steighaug’s truncated conjugate gradient method or by the trust region method of [21]. In the majorization ...

We deal with the iterative solution of linear systems arising from so-called dual-dual mixed finite element formulations. The linear systems are of a two-fold saddle point structure; they are indefinite and ill-conditioned. We define a special inner product that makes matrices of the two-fold saddle point structure, after a specific transformation, symmetric and positive definite. Therefore, th...

We still need to show that the directions p0, p1, . . . , pn−1 generated by Algorithms 1 and (2) are conjugate wrt A. If so, then by Theorem 11.3, this algorithm will terminate in n steps. The next theorem shows this property, along with two other important properties: (i) the residuals ri are mutually orthogonal, and (ii) each pk and rk is contained in the Krylov subspace of degree k for r0, d...

The celebrated Nesterov’s accelerated gradient method offers great speed-ups compared to the classical gradient descend method as it attains the optimal first-order oracle complexity for smooth convex optimization. On the other hand, the popular AdaGrad algorithm competes with mirror descent under the best regularizer by adaptively scaling the gradient. Recently, it has been shown that the acce...