Must Know Tips for Deep Learning Neural Networks, Part 2

Previous post            Tweet Tags: Convolutional Neural Networks, Deep Learning, Tips, Tutorial Deep learning is white hot research topic. Add some solid deep learning neural network tips and tricks from a PhD researcher. By Xiu-Shen Wei, Nanjing University. Editor’s note: Yesterday we posted the first 4 tips & ticks of this fascinating post. Be sure to have a look before reading the second half. 5. Activation Functions One of the crucial factors in deep networks is activation function, which brings the non-linearity into networks. Here we will introduce the details and characters of some popular activation functions and give advices later in this section. The sigmoid non-linearity has the mathematical form 𝜎(x)=1/(1+e-x). It takes a real-valued number and “squashes” it into range between 0 and 1. In particular,…


Link to Full Article: Must Know Tips for Deep Learning Neural Networks, Part 2