Most standard algorithms for prediction with expert advice depend on a parameter called the learning rate. This learning rate needs to be large enough to fit the data well, but small enough to prevent overfitting. For the exponential weights algorithm, a sequence of prior work has established theoretical guarantees for higher and higher data-dependent tunings of the learning rate, which allow f...

Stochastic gradient descent (SGD), which updates the model parameters by adding a local gradient times a learning rate at each step, is widely used in model training of machine learning algorithms such as neural networks. It is observed that the models trained by SGD are sensitive to learning rates and good learning rates are problem specific. To avoid manually searching of learning rates, whic...

In a conventional SOM, it is of utmost importance that a certain and consistently decreasing learning rate function be chosen. Decrease the learning rate too fast, the map will not get converged and the performance of the SOM may take a steep fall, and if too slow, the procedure would take a large amount of time to get carried out. For overcoming this problem, we have hereafter proposed a const...

2 Rate-Distortion Basics 2 2.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2.2 Gaussian Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2.1 Sphere-Packing Intuition . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.3 Proving Information Rate Distortion = Rate Distortion . . . . . . . . . . . . 4 2.3.1 Convexity of R(...

We develop a new theoretical framework to analyze the generalization error of deep learning, and derive a new fast learning rate for two representative algorithms: empirical risk minimization and Bayesian deep learning. The series of theoretical analyses of deep learning has revealed its high expressive power and universal approximation capability. Although these analyses are highly nonparametr...

The field of optimization and machine learning are increasingly interplayed and optimization in different problems leads to the use of machine learning approaches‎. ‎Machine learning algorithms work in reasonable computational time for specific classes of problems and have important role in extracting knowledge from large amount of data‎. ‎In this paper‎, ‎a methodology has been employed to opt...

Multi-task learning (MTL) models have demonstrated impressive results in computer vision, natural language processing, and recommender systems. Even though many approaches been proposed, how well these balance different tasks on each parameter still remains unclear. In this paper, we propose to measure the task dominance degree of a by total updates parameter. Specifically, compute exponentiall...

LowRate DDoS (LDDoS) is a variation of attack that sends fewer packets than conventional attacks. However, by sending smaller number and using unique period, low-rate very effective in reducing the quality an internet network-based service due to full access. On other hand, with its nature also makes it difficult detect because looks more mixed normal user The Deep Learning model will be used t...