This paper presents a novel incremental algorithm that combines Q-learning, a well-known dynamic programming-based reinforcement learning method, with the TD() return estimation process, which is typically used in actor-critic learning, another well-known dynamic programming-based reinforcement learning method. The parameter is used to distribute credit throughout sequences of actions, leading ...

In the Spallation Neutron Source (SNS) ring, multi-turn injection is employed to obtain a large transverse beam size which significantly reduces the space-charge tune shift of the accumulated beam. Careful choice of the painting scheme and bump function is required to obtain the desired beam profile together with low beam loss. In this paper we examine, both analytically and numerically, the ef...

The aim of this work is to estimate dense displacement fields over long video shots. Put in sequence they are useful for representing point trajectories but also for propagating (pulling) information from a reference frame to the rest of the video. Highly elaborated optical flow estimation algorithms are at hand, and they were applied before for dense point tracking by simple accumulation, howe...

Fixed multi-beam base stations can significantly increase the capacity of WCDMA systems. Previous capacity studies have not included important Radio Network Algorithms (RNA) such as Admission Control (AC) which are essential to guarantee an acceptable Quality Of Service (QoS). Some of these RNA may have to be changed to secure the full gains of fixed multi-beam antennas. In this paper two downl...

Due to lack of resources such as transmission power and bandwidth in satellite systems, resource allocation problem is a very important challenge. Nowadays, new heterogeneous network includes one or more satellites besides terrestrial infrastructure, so that it is considered that each satellite has multi-beam to increase capacity. This type of structure is suitable for a new generation of commu...

Reinforced concrete coupled walls are a common lateral load resisting system used in multi-storey buildings. The effect of the coupling beams can improve seismic performance, but at the same time adds complexity to the design procedure. A case study coupled wall building is designed using Force-Based Design (FBD) and Direct Displacement-Based Design (DDBD) and in the case of the latter a step b...

Multi-point laser ignition (LI) is one of the key potential advantages that LI can offer over conventional spark ignition (SI). Previous research found that multi-point ignition produced more stable and faster combustion. This paper demonstrates a diffractive multi-point LI technology which is experimentally attempted at Liverpool University Laser Group. Sparks with arbitrary geometrical locati...