Feature normalization missing for handwritten number challenge
I tried to reach the 94-96% accuracy with 30 neurons in the hidden layer, loss function=objectives.L2, activation function=actvations.Sigmoid and eta=3. There was no convergence at all.
So after a week trying to reach the 94-96% accuracy, I figured out that the missing feature normalization might be the problem. I guess it's because of some overflow in the sigmoid function.
After I added feature normalization, I got the 94-96% accuracy.