What is the difference between AI and ML?

What is the difference between Artificial Intelligence and Machine Learning?

My students often ask the question: “What is the difference between Artificial Intelligence (AI) and Machine Learning (ML) – and is deep learning (DL) belonging to either AI or ML?”. In the following I provide a I) brief answer, a II) formal short answer and III) a more elaborated answer:

I) Brief answer: It is the same and it is different. Deep Learning can also belong to both, and: both are necessary. This explains well why HCI-KDD is so enormously important: Human–Computer Interaction (HCI) deals mainly with aspects of human perception, human cognition, human intelligence, sense-making and the interaction between human and machine. Knowledge Discovery from Data (KDD), deals mainly with machine intelligence, and with the development of algorithms for automatic and interactive data mining [1].

II) A formal short answer:

Deep Learning is part of  Machine Learning  is part of Artificial Intelligence

DL  \subset ML  \subset AI

This follows the popular Deep Learning book by Ian Goodfellow, Yoshua Bengio and Aaron Courville published by MIT Press 2016 [2]:
http://www.deeplearningbook.org/contents/intro.html

and here is the explanation:

III) A more elaborated answer:

Artificial Intelligence (AI) is the field working on understanding intelligence. The motto of Google Deep Mind is “understand intelligence – then understand everything else” (Demis Hassabis) – consequently the study of human intelligence is of utmost importance for understanding machine intelligence. The long-term goal of AI is in general intelligence (“strong AI”). AI has a strong connection to cognitive science and is a very old scientific field. After a first hype between 1950 and 1980 and a following AI-winter, it has regained hype status because of the practical success made by machine learning and particularly by the success of deep learning very recently (although going back to the early days of AI, e.g. [3]. Recently the DARPA described it well (DARPA Perspective on Artificial Intelligence by John LAUNCHBURY – excellent video, I highly recommend my students to watch it:

According to DARPA there are three waves of AI: the first wave as kind of a programmed ability to process information, i.e. engineers handcraft a set of rules to represent knowledge in well-defined domains. The structure of this knowledge is defined by human experts and specifics in the domain are explored by computers. The second wave of AI is the success of statistical learning, i.e. engineers create statistical models for specific problem domains and train them, preferably on very big data sets. (BTW: John LAUNCHBURY emphasizes the importance of geometric models for machine learning, e.g. manifolds in topological data analysis). Currently neural networks (deep learning) show tremendously interesting successes (see e.g. a recent work from our own group [4]. The future third wave will have to focus on explainable ai, i.e. contextual adaptation, and make models able to explain how an algorithms came to a decision (see my post on transparceny and trust in machine learning and our recent paper [7], and see our iML project page). In essence ALL three waves are necessary in the future and the combination of various methods promise success!

Machine Learning (ML) is a very practical field and deals with applying artificial intelligence for the design and development of algorithms that can learn from data, to gain knowledge from experience and improve their learning behaviour over time – for more details please refer to [5].

Whilst AI is the broader fundament and encompasses all underlying scientific theories of human learning vs. machine learning, ML itself is a very practical field with uncountable practical applications – the introduction by Sebastian Thrun (Stanford) and Katie Malone (Moderator at Linear Digressions) brings this beautiful to the point and makes it important how important machine learning is for business:

 

Deep Learning (DL) is one methodological family of ML based on, e.g. artificial neural networks (ANN), deep belief networks, recurrent neural networks, or to give a precise example of a feed forward ANN: the multilayer perceptron (MLP), which is a very simple mathematical function mapping a set of input data to output data. The concept behind is representations learning by introducing other representations that are expressed in terms of simpler representations. Maybe, this is how our brain works [6], but we do not know yet.

A nice example is the recognition of a cat (at 2m11s):

However, this immediately let us understand the huge shortcomings of these approaches: While these algorithms nicely recognize a cat, they cannot explain why it is a cat. The algorithm is unable to explain why it come to this conclusion. Consequently, the next level of machine learning and artificial intelligence is in explainable AI, see transparency.

A final note to my students: computational intelligence (you may call it either Artificial Intelligence (AI) or Machine Learning (ML) may help to solve problems, particularly in areas where humans have limited capacities (e.g. in high dimensional spaces, large numbers, big data, etc.); however, we must acknowledge that the problem-solving capacity of the human mind is still unbeaten in certain aspects (e.g. in the lower dimensions, little data, complex problems, etc.). A strategic aim to find solutions for data intensive problems is effectively the combination of our two areas: Human–Computer Interaction (HCI) and Knowledge Discovery (KDD).

A proverb attributed perhaps incorrectly to Albert Einstein (many proverbs are attributed to famous persons to make them appealing) illustrates this perfectly: “Computers are incredibly fast, accurate, but stupid. Humans are incredibly slow, inaccurate, but brilliant. Together they may be powerful beyond imagination”. Consequently, the novel approach to combine HCI & KDD in order to enhance human intelligence by computational intelligence fits perfectly to AI and ML together [1].

References:

[1]          Holzinger, A. 2013. Human–Computer Interaction and Knowledge Discovery (HCI-KDD): What is the benefit of bringing those two fields to work together? In: Cuzzocrea, Alfredo, Kittl, Christian, Simos, Dimitris E., Weippl, Edgar & Xu, Lida (eds.) Multidisciplinary Research and Practice for Information Systems, Springer Lecture Notes in Computer Science LNCS 8127. Heidelberg, Berlin, New York: Springer, pp. 319-328, doi:10.1007/978-3-642-40511-2_22.

[2]          Goodfellow, I., Bengio, Y. & Courville, A. 2016. Deep Learning, Cambridge (MA), MIT Press.

[3]          Mcculloch, W. S. & Pitts, W. 1943. A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biology, 5, (4), 115-133, doi:10.1007/BF02459570.

[4]          Singh, D., Merdivan, E., Psychoula, I., Kropf, J., Hanke, S., Geist, M. & Holzinger, A. 2017. Human Activity Recognition Using Recurrent Neural Networks. In: Holzinger, Andreas, Kieseberg, Peter, Tjoa, A. Min & Weippl, Edgar (eds.) Machine Learning and Knowledge Extraction: Lecture Notes in Computer Science LNCS 10410. Cham: Springer International Publishing, pp. 267-274, doi:10.1007/978-3-319-66808-6_18.

[5]          Holzinger, A. 2017. Introduction to Machine Learning and Knowledge Extraction (MAKE). Machine Learning and Knowledge Extraction, 1, (1), 1-20, doi:10.3390/make1010001.

[6]          Hinton, G. E. & Shallice, T. 1991. Lesioning an attractor network: Investigations of acquired dyslexia. Psychological review, 98, (1), 74.

[7]   Holzinger, A., Plass, M., Holzinger, K., Crisan, G.C., Pintea, C.-M. & Palade, V. 2017. A glass-box interactive machine learning approach for solving NP-hard problems with the human-in-the-loop. arXiv:1708.01104.

What is machine learning?

Many services of our every day life rely meanwhile on machine learning. Machine learning is a very practical field and provides powerful technologies that allows machines (i.e. computers) to learn from prior data, to extract knowledge, to generalize and to make predictions – similar as we humans can do (see the MAKE intro). There is a very nice and highly recommendable info graphic available by the Royal Society [1]. This includes also an interactive quiz, which can be found here:

Royal Society Infographic “What is machine learning?”

This is part of a larger info campaign about machine learning from the Royal Society:

https://royalsociety.org/topics-policy/projects/machine-learning/

[1] The Royal Society was formed by a group of natural scientists influenced by Francis BACON (1561-1626).  The first ‘learned society’ meeting on 28 November 1660 followed a lecture at Gresham College by Christopher WREN. Joined by Robert BOYLE and John WILKINS and others, the group received royal approval by King Charles II (1630-1685) in 1663 and was known since as ‘The Royal Society of London for Improving Natural Knowledge’.

Today the Royal Society is a registered charity and the governing body of the Society is its Council and its members are elected by and from the Fellowship. Important to mention is that the Royal Society has an international character: “Science is an inherently international activity. The Society’s aim  is to reinforce the importance of science to build partnerships between nations, promote international relations and science’s role in culture and society”

Deep Learning Playground openly available

TensorFlow – part of the Google brain project – has recently open sourced on GitHub a nice playground for testing and learning the behaviour of deep learning networks, which also can be used following the Apache Licence:

http://playground.tensorflow.org

Background: TensorFlow is an open source software library for machine learning. There is a nice video “large scale deep learning” by Jeffrey Dean.  TensorFlow is  an interface for expressing machine learning algorithms along with an implementation for executing such algorithms on a variety of heterogeneous systems, ranging from smartphones to high-end computer clusters and  grids of thousands of computational devices (e.g. GPU). The system has been used for research in various areas of computer science (e.g. speech recognition, computer vision, robotics, information retrieval, natural language processing, geographic information extraction, computational drug discovery). The TensorFlow API and a reference implementation were released as an open-source package under the Apache 2.0 license on 9th November 2015 and is available at www.tensorflow.org

Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J. & Devin, M. 2016. TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems. arXiv preprint arXiv:1603.04467.

It is also discussed on episode 24 of talking machines.

 

 

Happy Scientific 2016

We wish you a prosperous scientific 2016 with a lot of crazy ideas and successful breakthrough discoveries !

Happy New 2016

Happy New Year from the Holzinger Group HCI-KDD

Merry Christmas and a Happy 2015 from the Holzinger Group

Merry Christmas and a Happy 2015 from the Holzinger Group

Merry Christmas and a Happy 2015 from the Holzinger Group

hci4all.at is now hci-kdd.org

Our website appears in a new light!

We welcome our visitors to our new hci-kdd.org website. In order to keep pace with time, our website is now responsive with a fresh design.

HCI4ALL is now HCI-KDD

Our digital world, full of mobile computing devices and ubiquitous sensor networks produce increasingly large, complex, high-dimensional and weakly structured data sets, and increasing volumes of unstructured information. These enormous amounts of data require novel, efficient and interactive solutions for knowledge discovery/data mining.  A synergistic combination of methodologies and approaches of two areas offer ideal conditions towards unraveling such problems: Human-Computer Interaction (HCI) and Knowledge Discovery/Data Mining (KDD), with the goal of supporting human intelligence with machine learning, putting the “human-in-the-loop “. Our goal is to interactively discover new, previously unknown insights into the data and we are passionate on extending advanced methods including time (e.g. information entropy) and space (e.g. computational topology), along with user-centered software engineering methods to create interactive software for mobile applications & content analytics techniques.

Science is to test crazy ideas – Engineering is to put these ideas into Business!