![\"\"](\"https:\/\/www.datakeen.co\/wp-content\/uploads\/2018\/02\/3_Algorithms_Deep_Learning.png\")
<\/a><\/p>\n<\/p>\n
\u201cDeep” Learning has attracted much attention during these past years. And for a good reason: this subset of machine learning has stood out impressively in several research fields: facial recognition, speech synthesis, machine translation, and many others.<\/span><\/p>\n These research fields have in common to be perceptual problems related to our senses and our expression. They have long represented a real challenge for researchers because it is extremely difficult to model vision or voice by means of algorithms and mathematical formulas.<\/span><\/p>\n As a result, the first models that have been implemented in these fields have been constructed with a good deal of business expertise (in speech recognition: decomposition into phonemes, in machine translation: application of grammatical and syntactic rules). Years of research have been dedicated to the exploitation and processing of these non-structured data in order to derive meaning.<\/span><\/p>\n The problem is that these new representations of data invented by researchers have failed to generalize at full extent to any text, sound or image. If you used Google Translate before 2014, year when they switched to a 100% deep learning model, you will remember the obvious limitations at the time.<\/span><\/p>\n Deep learning places itself directly on top of raw data without distortion or pre-aggregation. Then, thanks to a very large number of parameters that self-adjust over learning, will learn from implicit links existing in the data.<\/span><\/p>\n Before going into details of three different algorithms * used in deep learning for different use cases, let\u2019s start by simply defining the model at the heart of deep learning: the “neural network”.<\/span><\/p>\n * We also talk about different network architectures.<\/span><\/p>\n <\/p>\n <\/p>\n Let me begin by saying that neural networks have very little to do with the neural system and the brain. The analogy between a neuron and a one-neuron neural network is essentially graphic, insofar as there is a flow of information from one end to the other network.<\/span><\/p>\n The first layer of a neural network is called the input layer. It is through this layer that your data will enter the network. Prior to “feeding” the network with your data you will need to change it to numbers if they are not already.<\/span><\/p>\n We’ll take the example of sentiment analysis on textual data.<\/span><\/p>\n Let\u2019s say you have 10,000 comments on your ecommerce website about products sold:<\/span><\/p>\n With your team you have labeled 1000 of them (we’ll see that you can also rely on pre-trained neural networks) into 3 classes (satisfied | neutral | dissatisfied). This number of 3 classes, often taken in the sense of analysis, is an example and you can actually set more.<\/span><\/p>\n – “I loved it, very good taste”;<\/span><\/p>\n – “I didn\u2019t like the packaging that much”;<\/span><\/p>\n – “I thought it was pretty good”<\/span><\/p>\n The final layer, called output layer, will provide you with the classification “satisfied \/ neutral \/ dissatisfied”.<\/span><\/p>\n And all layers between the input and output layer, layers called “hidden” are all different representations of the data. A representation may be the number of words in a sentence, the number of punctuation (?!) in a sentence, etc. You will not have to specify the network these representations; if statistically they help to correctly classify the sentences the network will teach alone.<\/span><\/p>\n <\/p>\n To illustrate these layers take another example: that of the estimated price of a home.<\/span><\/p>\n As we can see here we take four input variables: the size of the house, number of bedrooms, the postal code and the degree of richness of the area. The output is not seeking to classify but to predict a number: the price of the house. This is a problem known as regression.<\/span><\/p>\n The italicized words to examples of representations that the neural network will make the data after having seen many.<\/span><\/p>\n The parameters of the network are updated thanks to a process called “backpropagation”. And the more hidden layers there are in the network the “deeper” it is, hence the name “Deep” Learning.<\/span><\/p>\n Let us now see 3 different types of architectures of neural networks.<\/span><\/p>\n <\/p>\n These networks are used for all use cases around image or video which include face recognition or image classification.<\/span><\/p>\n For example Bai Du (the equivalent of Google in China) has set up portals powered by face recognition to let enter only employees of the company.<\/span><\/p>\n Snapchat and many mobile applications have leveraged the breakthroughs of deep learning and CNNs to increase their face recognition capacities in order to add extra layers on your face such as funny bunny ears and a pink nose.<\/span><\/p>\n The name \u201cconvolution\u201d comes from a mathematical operation: convolution between functions.<\/span><\/p>\n Put simply, the convolution applies a filter to the input image, the filter parameters are learned through the learning. A learnt filter will be able of detecting features in an image, for example angles, and use them to classify at best the image.<\/span><\/p>\n The image is first decomposed into 3 channels (R, G, B) pixels per pixel, we obtain three matrices of size <\/span>n x n<\/b> (where <\/span>n<\/b> is the number of pixels).<\/span><\/span><\/p>\n Below is an example of a convolution with a <\/span>6 x 6<\/b> size matrix:<\/span><\/span><\/p>\n It is important to note two important advantages inherent to convolutional networks:<\/span><\/p>\n <\/p>\n <\/p>\n <\/p>\n Recurrent neural networks are at the heart of many substantial improvements in areas as diverse as speech recognition, automatic music composition, sentiment analysis, DNA sequence analysis, machine translation.<\/span><\/p>\n <\/p>\n The main difference with other neural networks is that they take into account a sequence of data, often a sequence evolving over time. For example in the case of analyzing temporal data (time series) the network will still have in memory a part or all of the observations previous to the data being analyzed.<\/span><\/p>\n <\/p>\n The pattern of this network is produced here:<\/span><\/p>\n <\/p>\n Instead of taking into account separately input data (in the way a CNN would analyse image per image) the recurrent network takes into account data previously processed.<\/span><\/p>\n Some architectures, called bidirectional, can also take into account future data. For instance when analyzing text to identify named entities (people, companies, countries, etc.) the network would need to see the words of the whole sentence.<\/span><\/p>\n Example:<\/span><\/p>\n The beginning of the sentence (underlined) is not enough to identify who is \u2018Jean\u2019.<\/span><\/p>\n <\/p>\n <\/p>\n Autoencoders are applied mainly to anomaly detection (for example to detect fraud in banking or to find faults in an industrial production line). They can also be used in dimensionality reduction (close to the objective of a Principal Component Analysis). Indeed the goal of autoencoders is to teach the machine what constitutes “normal” data.<\/span><\/p>\n The architecture of our network is the following:<\/span><\/p>\n The network will therefore represent data through one or more hidden layers so that the output will be as close as possible to the input data.<\/span><\/p>\n The objective to find the same data back as the output of the network is characteristic of autoencoders (analogous to the identity function f (x) = x).<\/span><\/p>\n The encoding and decoding stage it is not however specific to autoencoders. Indeed, they are found in machine translation in recurrent neural networks.<\/span><\/p>\n After training the network with enough data it will be possible to identify suspicious or anomalous observations when they exceed a certain threshold compared to the new “standard”.<\/span><\/p>\n <\/p>\n Conclusion:<\/b><\/span><\/p>\n We saw 3 major types of neural networks:<\/span><\/p>\n Other architectures exist such as GANs, generative adversarial networks, which are composed of a model generating candidates for a given task, for example image creation, and another that evaluates the different outputs. Or Reinforcement Learning, a method used by Deepmind to train their Alpha Go and Alpha Go Zero models.<\/span><\/p>\n Obviously there are limits: for example it is possible to fool convolutional network by adding a particular sound to image undetectable to the human eye but can be fatal for a model that has not been sufficiently tested robustness. New architectures such as capsule networks have merged to face this particular problem.<\/span><\/p>\n All in all it is certain that deep learning has a bright future with many business applications to come.<\/span><\/p>\n Ga\u00ebl Bonnardot,<\/span> [\/et_pb_text][\/et_pb_column][\/et_pb_row][\/et_pb_section]<\/p>\n","protected":false},"excerpt":{"rendered":" \u201cDeep” Learning has attracted much attention during these past years. And for a good reason: this subset of machine learning has stood out impressively in several research fields: facial recognition, speech synthesis, machine translation, and many others. These research fields have in common to be perceptual problems related to our senses and our expression. […]<\/p>\n","protected":false},"author":1,"featured_media":3691,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_yoast_wpseo_focuskw":"","_yoast_wpseo_title":"","_yoast_wpseo_metadesc":"","_et_pb_use_builder":"on","_et_pb_old_content":"\u00a0<\/h3>\n
1. Neural networks<\/b><\/h3>\n
<\/a><\/p>\n<\/a><\/p>\n\u00a0<\/h3>\n
2. Convolutional Neural Networks (CNN)<\/b><\/span><\/h3>\n
<\/a><\/p>\n\n
<\/a><\/p>\n<\/blockquote>\n\n
3. Recurrent neural networks (RNN)<\/h3>\n
<\/a><\/p>\n\n
4. Autoencoders<\/b><\/span><\/h3>\n
<\/a><\/p>\n\n
<\/a><\/p>\n
<\/span>
<\/span>Cofounder and CTO at Datakeen<\/strong><\/span><\/em>
<\/span>
<\/span>At Datakeen we seek to simplify the use and understanding of new machine learning paradigms by the business functions of all industries.<\/span>
<\/span>
<\/span>Contact us for more information: contact@datakeen.co<\/span><\/span><\/p>\n<\/a>\r\n\r\nL'apprentissage \u201cprofond\u201d ou \u201cdeep learning\u201d fait beaucoup parler de lui ces derni\u00e8res ann\u00e9es. Et pour cause, ce sous ensemble de l'apprentissage machine ('machine learning'') s'est impos\u00e9 de mani\u00e8re impressionnante dans plusieurs champs de recherche: reconnaissance faciale, synth\u00e8se vocale, traduction automatique, et bien d'autres.<\/span>\r\n\r\nCes champs de recherche ont pour point commun d'\u00eatre des probl\u00e8mes perceptifs, li\u00e9s \u00e0 nos sens et \u00e0 notre expression. Ils ont ainsi represent\u00e9 pendant longtemps un veritable d\u00e9fi pour les chercheurs car il est extr\u00eamement difficile de traduire la vue ou la voix aux moyens d'algorithmes et de formules math\u00e9matiques.<\/span>\r\n\r\nIl en r\u00e9sulte que les premiers mod\u00e8les qui ont \u00e9t\u00e9 mis en place dans ces domaines ont \u00e9t\u00e9 construits \u00e0 partir d'une certaine expertise m\u00e9tier (pour la reconnaissance vocale: la d\u00e9composition en phon\u00e8mes, pour la traduction machine: le passage par des r\u00e8gles grammaticales et syntaxiques). Des ann\u00e9es de recherche ont \u00e9t\u00e9 consacr\u00e9es \u00e0 l'exploitation et \u00e0 la transformation de ces donn\u00e9es non structur\u00e9es de mani\u00e8re \u00e0 en tirer du sens.<\/span>\r\n\r\nLe probl\u00e8me est que ces nouvelles repr\u00e9sentaitons des donn\u00e9es invent\u00e9s par les chercheurs se sont heurt\u00e9s \u00e0 la g\u00e9n\u00e9ralisation: \u00e0 tout texte, image ou son. Si vous avez utilis\u00e9 Google Tranlate avant 2014, ann\u00e9e \u00e0 laquelle ils ont bascul\u00e9 \u00e0 100% sur de l'apprentissage profond, vous vous souviendrez des limites \u00e9videntes \u00e0 l'\u00e9poque.<\/span>\r\n\r\nL'apprentissage profond propose de se placer directement au niveau des donn\u00e9es sans d\u00e9formation ou aggr\u00e9gation pr\u00e9alable. Puis, au moyen d'un nombre extr\u00eamement grand de param\u00e8tres qui s'auto-ajustent au fil de l'apprentissage, le r\u00e9seau va apprendre de lui m\u00eame les liens implicites qui existent dans les donn\u00e9es.<\/span>\r\n\r\nAvant de rentrer dans le d\u00e9tail de trois diff\u00e9rents algorithmes* utilis\u00e9s en apprentissage profond pour diff\u00e9rents cas d'usage, commencons par d\u00e9finir simplement le mod\u00e8le au coeur de l'apprentissage profond: le \u201cr\u00e9seau de neurones\u201d.<\/span>\r\n\r\n*On parle \u00e9galement d'architectures diff\u00e9rente de r\u00e9seaux.<\/span>\r\n1. Les R\u00e9seaux de Neurones\r\n<\/b><\/h3>\r\nDisons le tout de suite les r\u00e9seaux de neurones n'ont que tr\u00e8s peu \u00e0 voir avec le syst\u00e8me neuronal et le cerveau. L'analogie entre un neurone et un r\u00e9seau de neurones \u00e0 une couche est essentiellement graphique, dans la mesure o\u00f9 il y a un flux d'information d'un bout \u00e0 l'autre du r\u00e9seau.<\/span>\r\n\r\n
<\/a>La premi\u00e8re couche d'un r\u00e9seau de neurones est celle d'entr\u00e9e (input). C'est par cette couche que vont rentrer les donn\u00e9es dont vous disposez. Avant de pouvoir \u201cnourrir\u201d le r\u00e9seau, il faudra pr\u00e9alablement transformer vos donn\u00e9es en nombres si elles n'en sont pas d\u00e9j\u00e0.<\/span>\r\n\r\nNous allons prendre l'exemple de l'analyse de sentiment d'un texte.<\/span>\r\n\r\nVous avez 10 000 commentaires sur votre site \u00e0 propos de produits vendus:<\/span>\r\nAvec votre \u00e9quipe vous avez labellis\u00e9 1000 d'entre eux (nous allons voir que vous pouvez \u00e9galement vous appuyer sur des r\u00e9seaux de neurones pr\u00e9-entra\u00een\u00e9s) en 3 classes (satisfait | neutre | insatisfait). Ce nombre de 3 classes, souvent utilis\u00e9 dans l'analyse de sentiment, est un exemple et vous pouvez en r\u00e9alit\u00e9 en d\u00e9finir plus.<\/span>\r\n\r\n- \u201cJ'ai ador\u00e9, tr\u00e8s bon go\u00fbt\u201d;<\/span>\r\n- \u201cJe n'ai pas beaucoup aim\u00e9 l'emballage\u201d;<\/span>\r\n- \u201cJ'ai trouv\u00e9 \u00e7a plut\u00f4t bon\u201d<\/span>\r\n\r\nLa couche finale, dite de sortie (output), va vous fournir la classification \u201csatisfait \/ neutre \/ insatisfait\u201d.<\/span>\r\n\r\nEt toutes les couches entre la couche d'entr\u00e9e et de sortie, des couches dites \u201ccach\u00e9es\u201d, sont autant de repr\u00e9sentations diff\u00e9rentes des donn\u00e9es. Par repr\u00e9sentation cela peut \u00eatre le nombre de mots dans une phrase, le nombre de ponctuations (?!) dans une phrase, etc. Vous n'aurez pas \u00e0 pr\u00e9ciser au r\u00e9seau ces repr\u00e9sentations; si statistiquement celles-ci aident \u00e0 classifier correctement les phrases le r\u00e9seau les apprendra tout seul.<\/span>\r\n\r\n<\/a>\r\n\r\nPour illustrer ces couches prenons un autre exemple: celui de l'estimation du prix d'une maison.<\/span>\r\nComme nous pouvons le voir on prend ici 4 variables en entr\u00e9e: la superficie de la maison, le nombre de chambres, le code postal et le degr\u00e9 de richesse du quartier. En sortie on cherche non plus \u00e0 classifier mais \u00e0 pr\u00e9dire un nombre: le prix de la maison. C'est un probl\u00e8me dit de regression.<\/span>\r\nLes mots en italique renvoient \u00e0 des exemples de repr\u00e9sentations que le r\u00e9seau de neurones va faire des donn\u00e9es apr\u00e8s en avoir vu un grand nombre.<\/span>\r\n\r\nLes param\u00e8tres du r\u00e9seau sont mis \u00e0 jour gr\u00e2ce \u00e0 un processus appel\u00e9 \"r\u00e9tropropagation\". Plus il y a de couches cach\u00e9es dans le r\u00e9seau, plus on dit qu'il est \"profond\", d'o\u00f9 le nom d'apprentissage \"profond\".<\/span>\r\n\r\nVoyons maintenant 3 types architectures diff\u00e9rents de r\u00e9seaux de neurone.<\/span>\r\n<\/h3>\r\n
2. Les r\u00e9seaux de neurones dits convolutifs (CNN)<\/b><\/span><\/h3>\r\n\u00a0\r\n\r\nCes r\u00e9seaux sont utilis\u00e9s pour tout usage autour de l'image ou de la vid\u00e9o dont fait partie la reconnaissance faciale ou encore la classification d'image.<\/span>\r\n\r\nL'entreprise Bai Du (l'\u00e9quivalent de Google en Chine) a par exemple mis en place des portiques actionn\u00e9s par la reconnaissance visuelle qui laissent passer uniquement leurs employ\u00e9s.<\/span>\r\n\r\nSnapchat et de nombreuses applications mobiles ont utilis\u00e9 la perc\u00e9e de l'apprentissage et des CNN pour augmenter leurs fonctionnalit\u00e9s de \u201cfiltres\u201d.<\/span>\r\n\r\n
<\/a>Le nom r\u00e9seau convolutif renvoit \u00e0 un terme math\u00e9matique: le produit de convolution.<\/span>\r\nEn termes simples, l'id\u00e9e est qu'on applique un filtre \u00e0 l'image d'entr\u00e9e, les param\u00e8tres du filtre seront appris au fur et \u00e0 mesure de l'apprentissage. Un filtre appris permettra par exemple de d\u00e9tecter les angles dans une image si les angles servent \u00e0 classifier au mieux l'image.<\/span>\r\n\r\nL'image est d'abord d\u00e9compos\u00e9 dans les 3 cannaux (R,G,B) pixels par pixels, on obtient donc 3 matrices de taille n x n<\/strong> (o\u00f9 n<\/strong> est le nombre de pixels).<\/span>\r\n\r\nVoici un exemple de convolution avec une matrice de taille 6 x 6<\/strong> :<\/span>\r\n\r\n:<\/span><\/span>\r\n<\/a><\/blockquote>\r\nIl est important de noter deux avantages importants inh\u00e9rents aux r\u00e9seaux convolutifs :<\/span>\r\n\r\n \t
3. Les r\u00e9seaux de neurones dits r\u00e9currents (RNN)<\/h3>\r\n\u00a0\r\n\r\nLes r\u00e9seaux de neurones r\u00e9currents sont au coeur de bon nombre d'am\u00e9liorations substantiels dans des domaines aussi divers que la reconnaissance vocale, la composition automatique de musique, l'analyse de sentiments, l'analyse de s\u00e9quence ADN, la traduction automatique.<\/span>\r\n\r\nLa diff\u00e9rence principal avec les autres r\u00e9seaux de neurones vient du fait que ces derniers tiennent compte de l'encha\u00eenement successif des donn\u00e9es, bien souvent de leur encha\u00eenement dans le temps. Par exemple dans le cas de l'analyse d'une s\u00e9rie de mesures de capteurs (s\u00e9ries temporelles) le r\u00e9seau aura encore en m\u00e9moire tout ou partie des observations pr\u00e9c\u00e9dentes.<\/span>\r\n\r\nLe sch\u00e9ma de ce r\u00e9seau est produit ici:<\/span>\r\n\r\n\u00a0\r\n\r\n
<\/a>\r\n\r\nAu lieu de prendre en compte les donn\u00e9es d'entr\u00e9e de mani\u00e8re s\u00e9par\u00e9e (comme un CNN analyse image par image) le r\u00e9seau r\u00e9current lui prend en compte les donn\u00e9es d'entr\u00e9e pass\u00e9es.<\/span>\r\n\r\nCertaines architectures, dites bidirectionnelles, peuvent aussi prendre en compte les donn\u00e9es futures. Par exemple lors d'une analyse de texte o\u00f9 on cherche \u00e0 trouver des entit\u00e9s nomm\u00e9es (noms de personnes, soci\u00e9t\u00e9s, pays, etc.) n\u00e9cessite de voir les mots de toute la phrase.<\/span>\r\n\r\nExemple:<\/span>\r\n\r\n \t
4. Les Auto encodeurs<\/b><\/span><\/h3>\r\n\u00a0\r\n\r\nLes auto encodeurs sont appliqu\u00e9s principalement \u00e0 la d\u00e9tection d'anomalie ( par exemple pour d\u00e9tecter la fraude en banque ou bien pour trouver des anomalies dans une ligne de production industrielle ). Ils peuvent \u00e9galement servir \u00e0 la r\u00e9duction de dimension (proche dans l'id\u00e9e d'une Analyse en Composante Principale). En effet le but des auto encodeurs est d'apprendre \u00e0 la machine en quoi consiste des observations \u201cnormales\u201d.\r\nL'architecture de notre r\u00e9seau est le suivant:<\/span>\r\n\r\n
<\/a>\r\n\r\nLe r\u00e9seau va donc repr\u00e9senter les donn\u00e9es au moyen d'une ou plusieurs couches cach\u00e9es de sorte \u00e0 ce qu'en sortie on retrouve les m\u00eames donn\u00e9es qu'en entr\u00e9e.<\/span>\r\n\r\nL'objectif de retrouver en sortie les memes donn\u00e9es qu'en entr\u00e9e est caract\u00e9ristique des auto-encodeurs (analogue \u00e0 la fonction identit\u00e9 f(x)=x).<\/span>\r\nLa phase d'encodage et de d\u00e9codage n'est elle pas propre aux auto-encodeurs. En effet, on les retrouve dans la traduction automatique dans des r\u00e9seaux de neurones r\u00e9currents.<\/span>\r\n\r\nApr\u00e8s avoir entra\u00een\u00e9 le r\u00e9seau avec suffisamment de donn\u00e9es il sera possible d'identifier des observations suspectes ou anormales lorsque celles-ci d\u00e9passent un certain seuil par rapport \u00e0 la nouvelle \u201cnorme\u201d.<\/span>\r\n\r\n\u00a0\r\n\r\nConclusion:<\/b><\/span>\r\n\r\nNous avons vu 3 grands types de r\u00e9seau de neurones :<\/span>\r\n\r\n \t
<\/a>\r\n\r\nIl est certain que l'apprentissage profond a encore de beaux jours devant lui avec de nombreuses nouvelles applications pour les entreprises \u00e0 venir.<\/span>\r\n\r\nGa\u00ebl Bonnardot,<\/span>