Programmable Tanh- and ELU-Based Photonic Neurons in Optics-Informed Neural Networks

Abstract

We demonstrate an integrated opto-electronic (Omicron Epsilon) device that can be programmed to provide a set of nonlinear activation functions (AFs) and present its operation within programmable tanh- and ELU-based photonic neurons at line rates up to 10 GBd. The OE activation module provides a set of well-known activation functions that are typically used in DL training models, including the tanh-, ELU- and inverted ELU-like functions. Its performance is experimentally evaluated when incorporated in a 4-input wavelength division multiplexed (WDM) photonic neuron and operating with non-deterministic data patterns, providing "noisy" tanh, ELU and inverted ELU AFs with an error-distribution that has in all cases a standard deviation of <0.49. We also evaluate the trainability of these "noisy" AFs and present for the first time an optics-informed training framework that incorporates the pattern-induced AF variations into the training process, yielding the first noise-aware training scheme where the noise emerges at the nonlinear AF NN segment. The performance analysis of the optics-informed training framework for all three AFs was carried out via Deep Learning setups suitable for classifying the Fashion MNIST and the CIFAR-10 datasets. This analysis has shown that the employment of traditional training schemes leads to significant accuracy degradations, which can be, however, almost completely waived when employing the optics-informed training framework, leading to accuracy values that are almost identical to the reference accuracy values obtained when ideal and noise-less AFs are used.