10

I was reading the papers on deep learning. Most of them refer to unsupervised learning.

They also say the neurons are pre-trained using unsupervised RBM network. Later they are fine tuned using Back propagation algorithm (supervised).

So can we solve supervised learning problems using deep learning??

I am trying to find out if deep learning can be applied for document classification problem. I know there are pretty good classifiers available. But my goal is to find out whether we can use deep learning for this purpose or not.

OmG
  • 18,337
  • 10
  • 57
  • 90
alex
  • 1,421
  • 1
  • 16
  • 19

7 Answers7

15

In short - yes, it can be, and is often used in supervised manner. Exactly as Ben J described - the "depth" of the architecture is used in the unsupervised manner in order to create very abstract representation of your data, which can be later on used as either preprocessing layer (no fine-tuning) or as an initialization for the neural network (with fine-tuning, Hinton-like). In particular you can use such approach for texts.

There is interesting very recent paper by Hinton regarding modeling the text documents with DBMs: http://www.cs.toronto.edu/~rsalakhu/papers/uai13.pdf

There are many resources available online, in particular pylearn library implements such approaches, including "classic" DBN (Deep Belief Network) http://deeplearning.net/tutorial/DBN.html

Furthermore, it is also possible to actually make classification using just stacked RBMs, this model is called "classification RBM". More details can be obtained here: http://machinelearning.org/archive/icml2008/papers/601.pdf

Sadi
  • 50
  • 1
  • 9
lejlot
  • 64,777
  • 8
  • 131
  • 164
2

Deep learning is about learning unknown concepts so are typically used in terms of finding patterns in sets of data. This is unsupervised since these patterns are not necessarily known a-priori. In supervised learning, however, the type of pattern you require is easily understood a-priori in the form of training patterns which fit the data you are trying to learn about. These patterns become the basis for fitting your model (e.g. a neural network trained using back-propagation) to your data. There is no true discovery of new concepts and components. So from this perspective, I would say that no, deep learning cannot be applied to solving supervised learning problems.

Having said that, you might be able to use it to find interesting patterns in your data. You could then use these interesting patterns as a basis for training using a standard supervised approach. Perhaps this is what they did above, where you mention

"They also say the neurons are pre-trained using unsupervised RBM network. Later they are fine tuned using Back propagation algorithm (supervised)."

Without having read what you read, perhaps they started with an unsupervised algorithm to find the most interesting data and in doing so performed a form of dimensionality reduction, leading to data that was easier to train than the original data, using a supervised algorithm.

Ben J
  • 1,367
  • 2
  • 15
  • 33
1

Besides the good reference provided above, there is another paper by Yann Le Cunn's group that does text classification just by encoding characters without using any external feature extraction library. It works simply by encoding at the character level. They claim 98% accuracy.

http://arxiv.org/pdf/1502.01710v2.pdf

London guy
  • 27,522
  • 44
  • 121
  • 179
1

Yes, Definately. Actually in industry deep learning is mostly used to solve supervised problems. RBM and such unsupervised things is not widely applied outside academic circle. To know more about deep learning with supervised learning, you can read papers after 2012, start from anything related to ImageNet Challenge.

These are main milestones of deep neural networks:

These are main milestone of neural networks

GhostPotato
  • 31
  • 1
  • 1
  • 9
1

Absolutely yes, you can use deep learning for both supervised and unsupervised learning. For example the well-known Network with is named CNN is used to unsupervised Learning. maybe you need learn something about text mining fundamentally to comprehend deep learning model that classify documents. I found this link interesting, I'm hoping that help you too: https://machinelearningmastery.com/best-practices-document-classification-deep-learning/

Mozhgan vazifehdoost
  • 13
  • 4
0

Yes Deep learning can be used for supervised learning. but i think what you're trying to understand is automatic feature extraction that happens inside the neural network that's what makes deep learning unique, unlike any other machine learning model that require manual human feature extraction

Youssef Khemiri
  • 49
  • 7
-1

As the basis of deep learning is a neuron, e just have supervised algorithms like the logistic regression in a neuron and it means, YES.

H. Hadipour
  • 1
  • 2