Artificial Intelligence (AI)

Addition to the lecture 11

What is AI?

It is the science and engineering of making intelligent machines, especially intelligent computer programs. It is related to the similar task of using computers to understand human intelligence, but AI does not have to confine itself to methods that are biologically observable

Applications of AI game playing speech recognition understanding natural language computer vision expert systems heuristic classification

http://www-formal.stanford.edu/jmc/whatisai/node3.html

Knowledge-based expert system

Artificial neural network (ANN) Decision tree Support vector machines (SVMs) …

The knowledge representation process normally involves encoding information from verbal descriptions, rules of thumb, images, books, maps, charts, tables, graphs, equations, etc. Hopefully, the knowledge base contains sufficient high-quality rules to solve the problem under investigation. Rules are normally expressed in the form of one or more “IF condition THEN action” statements. The condition portion of a rule statement is usually a fact, e.g., the pixel under investigation must reflect > 45% of the incident near-infrared energy. When certain rules are applied, various operations may take place such as adding a newly derived derivative fact to the database or firing another rule. Rules can be implicit (slope is high) or explicit (e.g., slope > 70%). It is possible to chain together rules, e.g., IF c THEN d; IF d THEN e; therefore IF c THEN e. It is also possible to attach confidences (e.g., 80% confident) to facts and rules.

The knowledge representation process normally involves encoding information from verbal descriptions, rules of thumb, images, books, maps, charts, tables, graphs, equations, etc. Hopefully, the knowledge base contains sufficient high-quality rules to solve the problem under investigation. Rules are normally expressed in the form of one or more “IF condition THEN action” statements. The condition portion of a rule statement is usually a fact, e.g., the pixel under investigation must reflect > 45% of the incident near-infrared energy. When certain rules are applied, various operations may take place such as adding a newly derived derivative fact to the database or firing another rule. Rules can be implicit (slope is high) or explicit (e.g., slope > 70%). It is possible to chain together rules, e.g., IF c THEN d; IF d THEN e; therefore IF c THEN e. It is also possible to attach confidences (e.g., 80% confident) to facts and rules.

Knowledge representation process

For example, a typical rule used by the MYCIN expert system is

IF the stain of the organism is gram-negative     AND the morphology of the organism is rod         AND the aerobicity of the organism is anaerobic             THEN there is strong suggestive evidence (0.8) that the class of the organism is Enterobacter iaceae.

For example, a typical rule used by the MYCIN expert system is

IF the stain of the organism is gram-negative     AND the morphology of the organism is rod         AND the aerobicity of the organism is anaerobic             THEN there is strong suggestive evidence (0.8) that the class of the organism is Enterobacter iaceae.

Following the same format, a typical remote sensing rule might be: IF blue reflectance is (Condition) < 15%     AND green reflectance is (Condition) < 25% AND red reflectance is (Condition) < 15% AND near-infrared reflectance is (Condition) > 45% THEN there is strong suggestive evidence (0.8) that the pixel is vegetated.

Following the same format, a typical remote sensing rule might be: IF blue reflectance is (Condition) < 15%     AND green reflectance is (Condition) < 25% AND red reflectance is (Condition) < 15% AND near-infrared reflectance is (Condition) > 45% THEN there is strong suggestive evidence (0.8) that the pixel is vegetated.

1. ANN

The motivation for the development of neural network technology stemmed from the desire to develop an artificial system that could perform "intelligent" tasks similar to those performed by the human brain (thousands of different inputs-neurons, output to many other neurons), with Simple processing elements A high degree of interconnection Simple scalar messages Adaptive interaction between elements

ANN usually has one input layer, one output layer, and no or some hidden layers between. Neurons in one layer are connected to all neurons in the next layer for passing information

Neural networks process information in a similar way the human brain does. The network is composed of a large number of highly interconnected processing elements (neurones) working in parallel to solve a specific problem. Neural networks learn by example. They cannot be programmed to perform a specific task. The examples must be selected carefully otherwise useful time is wasted or even worse the network might be functioning incorrectly. The disadvantage is that because the network finds out how to solve the problem by itself, its operation can be unpredictable.

How do ANN work?

Train the Network Input training sites to the network Network computes an output Network output compared to desired output Network weights are modified to reduce error

Use the network Input new data to the network Network computes outputs based on its training

An example of a complicated ANN

2. Decision tree

"A decision tree takes as input an object or situation described by a set of properties, and outputs a yes/no decision. Decision trees therefore represent Boolean functions. Functions with a larger range of outputs can also be represented...."

A decision tree is a type of multistage classifier that can be applied to a single image or a stack of images. It is made up of a series of binary decisions that are used to determine the correct category for each pixel. The decisions can be based on any available characteristic of the dataset. For example, you may have an elevation image and two different multispectral images collected at different times, and any of those images can contribute to decisions within the same tree. No single decision in the tree performs the complete segmentation of the image into classes. Instead, each decision divides the data into one of two possible classes or groups of classes.

Image segmentation (eCognition) + decision tree (such as see5 at

http://www.rulequest.com/see5-info.html)

Cont’

ETM PanchromaticETM PanchromaticETM PanchromaticETM Panchromatic

Predicted White FirPredicted White FirPredicted White FirPredicted White Fir

Expert’s ModelExpert’s ModelExpert’s ModelExpert’s Model

Hierarchical Decision Tree ClassifierHierarchical Decision Tree ClassifierHierarchical Decision Tree ClassifierHierarchical Decision Tree Classifier

ETM PanchromaticETM PanchromaticETM PanchromaticETM Panchromatic

C5.0 ModelC5.0 ModelC5.0 ModelC5.0 Model

Predicted White FirPredicted White FirPredicted White FirPredicted White Fir

Hierarchical Decision Tree Hierarchical Decision Tree Classifier Based on Classifier Based on Inductive Machine Inductive Machine

Learning Production RulesLearning Production Rules

Hierarchical Decision Tree Hierarchical Decision Tree Classifier Based on Classifier Based on Inductive Machine Inductive Machine

Learning Production RulesLearning Production Rules

Machine Learning-derived Classification MapMachine Learning-derived Classification MapMachine Learning-derived Classification MapMachine Learning-derived Classification Map

Thomas, et al. 2003, PERS

Cont’

ENVI’s decision tree tool is designed to implement decision rules, such as the rules derived by any number of excellent statistical software packages that provide powerful and flexible decision tree generators. Two examples that are used commonly in the remote sensing community include CART by Salford Systems and S-PLUS by Insightful. The logic contained in the decision rules derived by these software packages can be used to build a decision tree classifier with ENVI’s interactive decision tree tool.

Even if you have not used one of these packages to derive any decision rules, you may find ENVI’s new decision tree tool to be a useful way to explore your data, or to find areas in your data that fulfill certain criteria.

3. Support vector machines (SVMs)

Is a new generation learning system based on recent advances in statistical learning theory

SVMs deliver state-of-the-art performance in real-world applications such as text categorisation, hand-written character recognition, image classification, biosequences analysis, etc.

SVMs’s first introduction in the early 1990s lead to a recent explosion of applications and deepening theoretical analysis, that has now established SVMs along with neural networks as one of the standard tools for machine learning and data mining

Want to learn more?

http://svmlight.joachims.org/ http://svm.dcs.rhbnc.ac.uk/ http://www.csie.ntu.edu.tw/~cjlin/libsvm/ http://theoval.sys.uea.ac.uk/~gcc/svm/toolbox/ http://www.cs.wisc.edu/dmi/lsvm/ http://vision.ai.uiuc.edu/mhyang/svm.html