Dual Rectified Linear Units (DReLUs): A replacement for tanh activation functions in Quasi-Recurrent Neural Networks

Godin, F.F.GodinDegrave, JonasJonasDegraveDambre, JoniJoniDambreDe Neve, WesleyWesleyDe Neve2021-10-252021-10-252018-120167-8655https://imec-publications.be/handle/20.500.12860/30773Dual Rectified Linear Units (DReLUs): A replacement for tanh activation functions in Quasi-Recurrent Neural NetworksJournal articlehttps://www.sciencedirect.com/science/article/abs/pii/S0167865518305646