AI a modern approach, my understanding and key points

 To start with the AI field one thing has fascinated me is always been this book by Russell and Norvig. As a beginner, I love to read this book and note down some of my important points. These notes might help others to just get a zest of AI.

Epoch 1: Introduction

What is AI?

There are several definitions and perspective in consideration, depending on the perspective the definitions are as follows:

Thinking Humanly

“The exciting new effort to make computers

think . . . machines with minds, in the

full and literal sense.” (Haugeland, 1985)

“[The automation of] activities that we

associate with human thinking, activities

such as decision-making, problem solving,

learning . . .” (Bellman, 1978)

AI and Cognitive computing are the fields which associate in this category, here is the diagram which shows exact elaboration

Acting Humanly

“The art of creating machines that perform

functions that require intelligence

when performed by people.” (Kurzweil,

1990)

“The study of how to make computers do

things at which, at the moment, people are

better.” (Rich and Knight, 1991)

The Turing test is an ultimate test that confirms if the machine is Intelligent or not. The diagram elaborates this test.

Thinking Rationally

“The study of mental faculties through the

use of computational models.”

(Charniak and McDermott, 1985)

“The study of the computations that make

it possible to perceive, reason, and act.”

(Winston, 1992)

Rational thinking is a way of reasoning and inferring something which would lead to the decision. The diagram throws a limelight for the same.

Acting Rationally

“Computational Intelligence is the study

of the design of intelligent agents.” (Poole

et al., 1998)

“AI . . . is concerned with intelligent behavior

in artifacts.” (Nilsson, 1998)

An intelligent agent will not only analyze and take a decision but also will act for the same. The diagram below shows an insight to this approach.

THE FOUNDATIONS OF ARTIFICIAL INTELLIGENCE

Philosophy

Certain questions are considerable when we think about the Intelligence.

Is a thought always followed by certain action?

Is there something called as soul or mind which forms the consciousness?

There many such questions which are relevant in the understanding of the intelligence. Can animals think? Do they have consciousness? It is said that humans are gifted with the consciousness. The mind is very important aspect, for example pulling back hands or body part from a hot object, this is something which is done consciously by the mind. Another example would be, consider you are feeling cold, its windy outside the home, the reasoning mind takes decision to wear a winter jacket or a sweater. But how does it happen that thinking is sometimes accompanied by action and sometimes not, sometimes by motion, and sometimes not? It looks as if almost the same thing happens as in the case of reasoning and making inferences about unchanging objects.

 

Mathematics

 

What can be calculated?

 

Formal rules to conclude certain decisions

 

The above-mentioned points or questions are the basic ideas behind the mathematical influence on AI. The Boolean logic formulates one of the vital components in the mathematics behind AI. The first nontrivial algorithm is thought to be Euclid’s algorithm for computing greatest common divisors. Further the algorithms form the thesis of development of any program, be it AI or a simple logical software. Besides logic and computation, the third great contribution of mathematics to AI is the theory of probability.

Economics

 

How should we make decisions to maximize payoff?

How should we do this when others may not go along?

How should we do this when the payoff may be far in the future?

 

Most people think of economics as being about money, but economists will say that they are really studying how people make choices that lead to preferred outcomes.

Decision theory, which combines probability theory with utility theory, provides a formal and complete framework for decisions (economic or otherwise) made under uncertainty—that is, in cases where probabilistic descriptions appropriately capture the decision maker’s environment.

 

Neuroscience

How do brains process information?

 

Neuroscience is the study of the nervous system, particularly the brain. Although the exact way in which the brain enables thought is one of the great mysteries of science, the fact that it does enable thought has been appreciated for thousands of years because of the evidence that strong blows to the head can lead to mental incapacitation.

 

Neuron

The parts of a nerve cell or neuron. Each neuron consists of a cell body, or soma, that contains a cell nucleus. Branching out from the cell body are several fibers called dendrites and a single long fiber called the axon. The axon stretches out for a long distance, much longer than the scale in this diagram indicates. Typically, an axon is 1 cm long (100 times the diameter of the cell body) but can reach up to 1 meter. A neuron makes connections with 10 to 100,000 other neurons at junctions called synapses. Signals are propagated from neuron to neuron by a complicated electrochemical reaction. The signals control brain activity in the short term and enable long-term changes in the connectivity of neurons. These mechanisms are thought to form the basis for learning in the brain. Most information processing goes on in the cerebral cortex, the outer layer of the brain. The basic organizational unit appears to be a column of tissue about 0.5 mm in diameter, containing about 20,000 neurons and extending the full depth of the cortex about 4 mm in humans).

The truly amazing conclusion is that a collection of simple cells can lead to thought, action, and consciousness or, brains cause minds. The only real alternative theory is mysticism: that minds operate in some mystical realm that is beyond physical science.

 

Psychology

 

How do humans and animals think and act?

Cognitive psychology, which views the brain as an information-processing device. Key steps of a knowledge-based agent: (1) the stimulus must be translated into an internal representation, (2) the representation is manipulated by cognitive processes to derive new internal representations, and (3) these are in turn retranslated back into action.

If the organism carries a “small-scale model” of external reality and of its own possible actions within its head, it is able to try out various alternatives, conclude which is the best of them, react to future situations before they arise, utilize the knowledge of past events in dealing with the present and future, and in every way to react in a much fuller, safer, and more competent manner to the emergencies which face it. (Craik, 1943)

It is now a common (although far from universal) view among psychologists that “a cognitive theory should be like a computer program” (Anderson, 1980); that is, it should describe a detailed information processing mechanism whereby some cognitive function might be implemented.

 

Computer engineering

 

How can we build an efficient computer?

 

For artificial intelligence to succeed, we need two things: intelligence and an artifact. The computer has been the artifact of choice. Since the invent of computer, each generation of computer hardware has brought an increase in speed and capacity and a decrease in price. The World war 2, led to team up and bring up the best computing devices from the three countries, Alan Turing’s team from UK with the sole purpose of deciphering the German messages lands up developing a powerful general-purpose machine “Colossus”. The first operational programmable computer was the Z-3, the invention of Konrad Zuse in Germany in 1941. Zuse also invented floating-point numbers and the first high-level programming language, Plankalkül. The first electronic computer, the ABC, was assembled by John Atanasoff and his student Clifford Berry between 1940 and 1942 at Iowa State University. Atanasoff’s research received little support or recognition; it was the ENIAC, developed as part of a secret military project at the University of Pennsylvania by a team including John Mauchly and John Eckert, that proved to be the most influential forerunner of modern computers. Charles Babbage (1792–1871) designed two machines, neither of which he completed. The Difference Engine was intended to compute mathematical tables for engineering and scientific projects. It was finally built and shown to work in 1991 at the Science Museum in London (Swade, 2000). Babbage’s Analytical Engine was far more ambitious: it included addressable memory, stored programs, and conditional jumps and was the first artifact capable of universal computation. Babbage’s colleague Ada Lovelace was perhaps the world’s first programmer. She wrote programs for the unfinished Analytical Engine and even speculated that the machine could play chess or compose music. AI also owes a debt to the software side of computer science, which has supplied the operating systems, programming languages, and tools needed to write modern programs.

Control theory and cybernetics

 

How can artifacts operate under their own control?

 

Control systems formulates an important aspect of Engineering, a control system manages, commands, directs, or regulates the behavior of other devices or systems using control loops. It can range from a single home heating controller using a thermostat controlling a domestic boiler to large industrial control systems which are used for controlling processes or machines. Other examples of self-regulating feedback control systems include the steam engine governor, created by James Watt (1736–1819), Intelligence could be created using homeostatic devices containing appropriate feedback loops to achieve stable adaptive behavior. This roughly matches our view of AI: designing systems that behave optimally.

 

Linguistics

 

How does language relate to thought?

 

Modern linguistics and AI, then, were “born” at about the same time, and grew up together, intersecting in a hybrid field called computational linguistics or natural language processing. Understanding language requires an understanding of the subject matter and context, not just an understanding of the structure of sentences.

 

Recommended to watch:

THE HISTORY OF ARTIFICIAL INTELLIGENCE

 

The history of AI has had cycles of success, misplaced optimism, and resulting cutbacks in enthusiasm and funding. There have also been cycles of introducing new creative approaches and systematically refining the best ones. AI has advanced more rapidly in the past decade because of greater use of the scientific method in experimenting with and comparing approaches. Recent progress in understanding the theoretical basis for intelligence has gone hand in hand with improvements in the capabilities of real systems. The subfields of AI have become more integrated, and AI has found common ground with other disciplines.

The first work that is now generally recognized as AI was done by Warren McCulloch and Walter Pitts (1943). They drew on three sources: knowledge of the basic physiology and function of neurons in the brain; a formal analysis of propositional logic due to Russell and Whitehead; and Turing’s theory of computation. They proposed a model of artificial neurons in which each neuron is characterized as being “on” or “off,” with a switch to “on” occurring

in response to stimulation by enough neighboring neurons. Two undergraduate students at Harvard, Marvin Minsky, and Dean Edmonds, built the first neural network computer in 1950. The SNARC, as it was called, used 3000 vacuum tubes and a surplus automatic pilot mechanism from a B-24 bomber to simulate a network of 40 neurons.

Alan Turing's vision was perhaps the most influential. He gave lectures on the topics as early as 1947 at the London Mathematical Society and articulated a persuasive agenda in his 1950 article "Computing Machinery and Intelligence". Therein he introduced the Turin test, machine learning, genetic algorithms, and reinforcement learning. 

Princeton was home to another influential figure in AI, John McCarthy. McCarthy moved to Stanford and then to Dartmouth College, which was to become the official birthplace of the field. McCarthy convinced Minsky, Claude Shannon, and Nathaniel Rochester to help him bring together U.S. researchers interested in automata theory, neural nets, and the study of intelligence. They organized a two-month workshop at Dartmouth in the summer of 1956.

McCarthy defined the high-level language Lisp, which was to become the dominant AI programming language for the next 30 years.

 

Early work building on the neural networks of McCulloch and Pitts also flourished. The work of Winograd and Cowan (1963) showed how many elements could collectively represent an individual concept, with a corresponding increase in robustness and parallelism. Hebb’s learning methods were enhanced by Bernie Widrow (Widrow and Hoff, 1960; Widrow, 1962), who called his networks adalines, and by Frank Rosenblatt (1962) with his perceptrons. The perceptron convergence theorem (Block et al., 1962) says that the learning algorithm can adjust the connection strengths of a perceptron to match any input data, provided such a match exists.

 

However, with the start there were also many difficulties, first kind of difficulty arose because most early programs knew nothing of their subject matter; they succeeded by means of simple syntactic manipulations. The second kind of difficulty was the intractability of many of the problems that AI was attempting to solve. Most of the early AI programs solved problems by trying out different combinations of steps until the solution was found. This strategy worked initially because microworlds contained very few objects and hence very few possible actions and very short solution sequences. A third difficulty arose because of some fundamental limitations on the basic structures being used to generate intelligent behavior. For example, Minsky and Papert’s book Perceptrons (1969) proved that, although perceptrons (a simple form of neural network) could be shown to learn anything they could represent, they could represent very little. In particular, a two-input perceptron (restricted to be simpler than the form Rosenblatt originally studied) could not be trained to recognize when its two inputs were different. Although their results did not apply to more complex, multilayer networks, research funding for neural net research soon dwindled to almost nothing. Ironically, the new back-propagation learning algorithms for multilayer networks that were to cause an enormous resurgence in neural-net research in the late 1980s were actually discovered first in 1969 (Bryson and Ho, 1969).

The picture of problem solving that had arisen during the first decade of AI research was of a general-purpose search mechanism trying to string together elementary reasoning steps to find complete solutions. Such approaches have been called weak methods because, although general, they do not scale up to large or difficult problem instances. The alternative to weak methods is to use more powerful, domain-specific knowledge that allows larger reasoning steps and can more easily handle typically occurring cases in arrow areas of expertise. One might say that to solve a hard problem, you must almost know the answer already. 

In the mid-1980s at least four different groups reinvented the back-propagation learning algorithm first found in 1969 by Bryson and Ho. The algorithm was applied to many learning problems in computer science and psychology, and the widespread dissemination of the results in the collection Parallel Distributed Processing (Rumelhart and McClelland, 1986) caused great excitement.

Recent years have seen a revolution in both the content and the methodology of work in artificial intelligence. 

 

In the early period of AI it seemed plausible that new forms of symbolic computation, e.g., frames and semantic networks, made much of classical theory obsolete. This led to a form of isolationism in which AI became largely separated from the rest of computer science. This isolationism is currently being abandoned. There is a recognition that machine learning should not be isolated from information theory, that uncertain reasoning should not be isolated from stochastic modeling, that search should not be isolated from classical optimization and control, and that automated reasoning should not be isolated from formal methods and static analysis. - David McAllester (1998)

 

The field of speech recognition illustrates the pattern. In the 1970s, a wide variety of different architectures and approaches were tried. Many of these were rather ad hoc and fragile and were demonstrated on only a few specially selected examples. In recent years, approaches based on hidden Markov models (HMMs) have come to dominate the area.

Neural networks also fit this trend. Much of the work on neural nets in the 1980s was done to scope out what could be done and to learn how neural nets differ from “traditional” techniques. Using improved methodology and theoretical frameworks, the field arrived at an understanding in which neural nets can now be compared with corresponding techniques from statistics, pattern recognition, and machine learning, and the most promising technique can be applied to each application. As a result of these developments, so-called data mining technology has spawned a vigorous new industry

The Bayesian network formalism was invented to allow efficient representation of, and rigorous reasoning with, uncertain knowledge. This approach largely overcomes many problems of the probabilistic reasoning systems of the 1960s and 1970s; it now dominates AI research on uncertain reasoning and expert systems. The approach allows for learning from experience, and it combines the best of classical AI and neural nets. Work by Judea Pearl (1982a) and by Eric Horvitz and David Heckerman (Horvitz and Heckerman, 1986; Horvitz et al., 1986) promoted the idea of normative expert systems: ones that act rationally according to the laws of decision theory and do not try to imitate the thought steps of human experts.

Similar gentle revolutions have occurred in robotics, computer vision, and knowledge representation. A better understanding of the problems and their complexity properties, combined with increased mathematical sophistication, has led to workable research agendas and robust methods.

One of the most important environments for intelligent agents is the Internet. AI systems have become so common in Web-based applications that the “-bot” suffix has entered everyday language. Moreover, AI technologies underlie many Internet tools, such as search engines, recommender systems, and Web site aggregators. Recent progress in the control of self-driving cars has derived from a mixture of approaches ranging from better sensors, control-theoretic integration of sensing, localization, and mapping, as well as a degree of high-level planning. 

But all the influential founders of AI believe (John McCarthy (2007), Marvin Minsky (2007), Nils Nilsson (1995, 2005) and Patrick Winston (Beal and Winston, 2009)) it should be focused on the roots or sole purpose i.e. they call it Human Level AI (HLAI) where machines could think, learn and create, instead of focusing on single applications such as the Self- Driving car or Chatbot or speech recognition. Also, with the availability of data and a larger number of datasets. The hardware, the silicon industry boom supports on a greater reach which enables the AI to get to be more progressive.

Reference: Artificial Intelligence, a modern approach (by Russell and Norvig)