What is the difference between Deep and Shallow networks?

A Deep Neural Network consists of two or more hidden layers. While a single layer perceptron can be used as a binary classifier for data that is linearly separable, a Neural Network with one hidden layer is the usual case in Traditional Machine Learning.  This is also the minimum number of layers required to be useful for such a case, and this configuration is called a ‘Shallow Network’. As more layers are added to the Neural Network, the model moves towards being a ‘Deep Network’.