Studying Multi-head Attention in the Style Tokens Paper

I am trying to understand the multi-head attention introduced in the paper, “Attention Is All You Need“. The purpose to understand the multi-head attention is understand the style token layer, which contains multi-head attention and was introduced in the paper, “Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis“. Multi-head attention is […]

Digital Signal Processing | Coursera

1.2.a Discrete-time signals Discrete-time signals Discrete-time signal:= A sequence of complex numbers Dimension = 1 (for now) Notation: where is an integer Two-sided sequences: is one-dimensional “time”. Analysis: Periodic measurement approach Discrete-time signals can be created by an analysis process where we take periodic measurements of a physical phenomenon. Synthesis: Stream of generated samples Delta […]

Survey on knowledge graph embedding

Papers Q. Wang, Z. Mao, B. Wang and L. Guo, “Knowledge Graph Embedding: A Survey of Approaches and Applications,” in IEEE Transactions on Knowledge and Data Engineering, vol. 29, no. 12, pp. 2724-2743, 1 Dec. 2017. Maximilian Nickel, Kevin Murphy, Volker Tresp, Evgeniy Gabrilovich. A Review of Relational Machine Learning for Knowledge Graphs. Proc. IEEE, […]

Sequence Modeling | Deep Learning Specialization | Coursera

Course planning Week 1: Recurrent neural networks Learn about recurrent neural networks. This type of model has been proven to perform extremely well on temporal data. It has several variants including LSTMs, GRUs and Bidirectional RNNs, which you are going to learn about in this section. Lectures: Recurrent neural networks C4W1L01 Why sequence models C4W1L02 […]

Neural Networks and Deep Learning | Deep Learning Specialization | Coursera

Lecture Planning Week 1:?Introduction to Deep Learning Welcome to the Deep Learning Specialization C1W1L01 Welcome Introduction to Deep Learning C1W1L02 Welcome C1W1L03 What is a neural network? C1W1L04 Supervised Learning with Neural Networks C1W1L05 Why is Deep Learning taking off? C1W1L06 About this Course C1W1R1 Frequently Asked Questions C1W1L07 Course Resources C1W1R2 How to use […]

Curriculum Learning | Bengio et al. | ICML 2009 | 2009

Brief information Authors:?Yoshua Bengio, J?r?me Louradour, Ronan Collobert, Jason Weston Published year: 2009 Publication: ICML 2009 Abstract Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. We formalize such training strategies in the context of […]

One-Shot Imitation Learning | Yan Duan et al. | 2017

Summary Abstract Ideally, robots should be able to learn from very few demonstrations of any given task, and instantly generalize to new situations of the same task, without requiring task-specific engineering. In this paper, we propose a meta-learning framework for achieving such capability, which we call one-shot imitation learning. Task examples: to stack all blocks […]

Conditional Generative Adversarial Nets | M. Mirza, S. Osindero | 2014

Introduction Conditional version of Generative Adversarial Nets (GAN) where both generator and discriminator are conditioned on some data y (class label or data from some other modality). Architecture Feed y into both the generator and discriminator as additional input layers such that y and input are combined in a joint hidden representation.