metody i modeli analiza dannykh olap i data mining 3642712
Post on 17-Dec-2015
15 views
DESCRIPTION
Data mining models & algorythmsTRANSCRIPT
-
. . . . . . . .
071900 654700
-
-
2004
-
681.3.06(075.8) 32.973.26-018.273 26
. ., . ., . ., . .
26 : OLAP Data Mining. .: -, 2004. 336 .: .
ISBN 5-94157-522-
: , (OLAP) (Data Mining) . : , . . Data Mining. -, Data Mining, Xelopes, - .
681.3.06(075.8) 32.973.26-018.273
:
.
.
.
02429 24.07.00. 31.08.04.
70 1001/16. . . . . 27,09.
2500 . "-", 190005, -, ., 29.
, 77.99.02.953..001537.03.02 13.03.2002 . .
" ""
199034, -, 9 , 12
ISBN 5-94157-522- . ., . ., . .,
. ., 2004 , "-", 2004
-
........................................................................................... 9
Data Mining ............................................................ 11
1. ............................................. 13
1.1. ........................................................ 13 1.2. .................................................................................. 16 1.3. OLTP- .................. 21 ................................................................................................................................ 26
2. .............................................................................. 27
2.1. ................................................................................. 27 2.2. .......................................................................................................... 34 2.3. ........................................................................................................... 39 2.4. ..................................................................................... 45 ................................................................................................................................ 45
3. OLAP- .................................................................................... 49
3.1. .................................................................................... 49 3.2. OLAP- ........................................................................................ 53 3.3. ......................................................... 54
3.3.1. .......................................................................................54 3.3.2. ............................................................................55 3.3.3. FASMI ...............................................................................................................57
3.4. OLAP- ......................................................................................... 58 3.4.1. MOLAP ......................................................................................................................59 3.4.2. ROLAP .......................................................................................................................62 3.4.3. HOLAP .......................................................................................................................65
................................................................................................................................ 66
4. ..................................................... 67
4.1. Data Mining ................................................................................. 67 4.2. Data Mining ..................................................................................................... 68
-
4
4.2.1. Data Mining ........................................................................68 4.2.2. .......................................................................70 4.2.3. ...................................................................72 4.2.4. ..............................................................................................74
4.3. Data Mining ................................................................... 76 4.3.1. - ..............................................................................................76 4.3.2. ....................................................................................................................76 4.3.3. ..................................................................................................77 4.3.4. ................................................................................77 4.3.5. ..................................................................................................................78 4.3.6. .......................................................................................................79 4.3.7. .....................................................................................................80 4.3.8. ...................................................................................80
4.4. Data Mining ................................................................................................... 80 4.4.1. (predictive) ...................................................................80 4.4.2. (descriptive) .........................................................................81
4.5. Data Mining ................................................................................................... 83 4.5.1. ........................................................................................................83 4.5.2. .......................................................................................................83 4.5.3. .........................................................................................86 4.5.4. .......................................................................................................88
4.6. ................................................................................... 89 4.6.1. ........................................................................................89 4.6.2. ................................................................................91
................................................................................................................................ 93
5. ................................................................ 95
5.1. ...................................................................................................... 95 5.2. ........................................................................................ 96
5.2.1. .........................................................................................96 5.2.2. ......................................................................................................97 5.2.3. .......................................................................................99
5.3. ........................................................... 99 5.3.1. 1-..............................................................................99 5.3.2. Naive Bayes .................................................................................................. 101
5.4. .................................................................. 104 5.4.1. " " .......................................................................... 104 5.4.2. ............................................................................................... 112
5.5. ..................................................... 117 5.5.1. .............................................................................................................. 117 5.5.2. . ............................................ 119 5.5.2. .............................................................................................. 120 5.5.3. Support Vector Machines (SVM) ............................................................................ 121
5.6. ......................................................................................................... 124 .............................................................................................................................. 128
6. ............................................................ 129
6.1. .................................................................................................... 129 6.1.1. ........................................................................... 129
-
5
6.1.2. ....................................................................................... 132 6.1.3. ..................................... 135
6.2. ...................................................................................... 137 6.3. ................................................................................................................. 141
6.3.1. Apriori ..................................................................................................... 141 6.3.2. Apriori ......................................................................... 146
.............................................................................................................................. 147
7. .................................................................................... 149
7.1. .......................................................................... 149 7.1.1. ........................................................................... 152 7.1.2. , , ................................................................................. 154
7.2. ...................................................................................... 156 7.3. ......................................................................... 158
7.3.1. .................................................................................. 158 7.3.2. .................................................................................... 159
............................................................................ 159 .................................................................................. 161
7.3.3. ................................................................................ 162 k-means (Hard-c-means) ................................................................. 162 Fuzzy -Means ................................................................................ 166 - ...................................................... 168
7.4. .................................. 174 7.4.1. ................................................................................................... 174
............................................................... 174 ........................................................................................... 179 - .......................................................................... 181
7.4.2. - .................................................................. 182 - ......................................................................... 190 - ......................................................................................... 191 - ................................................................................. 192
.............................................................................................................................. 204
8. Data Mining ..................................................................... 209
8.1. .................................................................................................. 209 8.2. CWM .......................................................................................................... 209
8.2.1. CWM ................................................................................ 209 8.2.2. CWM ...................................................................................... 211 8.2.3. Data Mining .................................................................................................. 214
8.3. CRISP ........................................................................................................ 218 8.3.1. CRISP ................................................................................ 218 8.3.2. CRISP .................................................................................. 218 8.3.3. CRISP .......................................................................... 220
-
6
8.4. PMML ........................................................................................................ 225 8.5. Data Mining................................................................................. 233
8.5.1. SQL/MM................................................................................................. 233 8.5.2. OLE DB Data Mining ..................................................................... 235 8.5.3. JDMAPI .................................................................................................. 237
.............................................................................................................................. 237
9. Xelopes ............................................................................ 241
9.1. ........................................................................................... 241 9.2. Model ...................................................................................................... 244
9.2.1. Xelopes................................................................................... 244 9.2.2. Model ............................................................................................ 246 9.2.3. ...................................................................................... 247
9.3. Settings .................................................................................................... 248 9.3.1. Settings ........................................................................................... 248 9.3.2. Settings .......................................................................................... 250
9.4. Attribute ................................................................................................... 250 9.4.1. Attribute ......................................................................................... 250 9.4.2. ....................................................................................... 251
9.5. Algorithms ............................................................................................... 252 9.5.1. .................................................................................................. 252 9.5.2. MiningAlgorithm ............................................................................................. 253 9.5.3. MiningAlgorithm ..................................................................... 254 9.5.4. ....................................................................................... 256 9.5.5. ............................................................................................................... 256
9.6. DataAccess ............................................................................................... 256 9.6.1. .................................................................................................. 257 9.6.2. MiningInputStream ......................................................................................... 258 9.6.3. Mining- ....................................................................................... 258 9.6.4. , MiningInputStream ................................................ 258
9.7. Transformation ........................................................................................ 259 9.8. Xelopes ....................................................... 261
9.8.1. .................................................................................................. 261 9.8.2. ................................................ 264 9.8.3. ............................................................................ 266 9.8.4. .......................................................................... 268
.............................................................................................................................. 271
1. .......................................................... 273
1.1. ........................................ 273 1.2. ................................................................................................. 277 1.3. ................................................................................... 279 1.4. ............................................................................... 280 1.5. ..................................................................................................... 286 1.6. .......................................................................... 293
-
7
2. ...... 299
2.1. .... 299 2.2. .................................. 304
2.2.1. ....................................................................... 304 2.2.2. ......... 305 2.2.3. ................ 307
2.3. .......................................................... 310 2.4. , ............ 311 2.5. ........................................................................... 314
2.5.1. ............................................................................................... 314 2.5.2. .............................................................................................. 316
................................................................... 316 .................................................................... 317 ................................................................ 318
2.5.3. ........................................................................ 318 2.6. ............................................................................................. 319
3. - .................................. 327
........................................................................................... 331
..................................................................................... 335
-
8
-
, . ,
.
, , , -, , , , ,
.
, , , .
: , .
. - -
. , . 7.4 . . .
, -
. , - , -
. , - , " ". , ,
.
- , .
-
10
, -
, -, .
:
- Data Mining
.
. Prudsys
.
-
Data Mining
2002 , Berkeley,
(5,000,000,000,000,000,000) . , - 23 . , ,
, . 2003 France Telecom (DSS system) 30,000 -
, a Alexa Internet Archive 500,000 .
, (Knowledge Dis-covery in Data workshop), 1989 ,
(1,000,000) . - KDD-2003 -
(1 = 1,000 , 1 = 1,000 ).
- - .
- Data Mining.
Data Mining ( Knowledge Discovery In Data
) , - . Data Mining - , , -
.
Data Mining 1989 2003
. Data ining . , , e, , . -
-
Data Mining 12
, , ,
. . Data Mining - , - (Google)
(search engines) .
, ,
, , , , . . Data Mining -
(link analysis), , , a .
M Data Mining. 1970- 2- -
, , .
- , OLAP, -
Data Mining.
Data Mining . - Data Mining, , . -
Data Mining.
Xelopes Prudsys, - Data Mining. -
, .
, Data Mining .
-, KDnuggets
, , , 2004
-
1
1.1.
. , (. 1.1), -, , . - . - - . , , , , .
, , , . , . , - , (). (DSS, Decision Support System).
, . , - , :
;
;
.
-
1 14
. 1.1.
, , , -
, ,
.
, -
, -.
,
, .
,
. -
,
.
.
.
, -
, . .
-
. , -
-
. ,
(, ,
-
15
. .) , -
. ,
, ,
. , -
,
-
. , -
.
"" :
-
. ;
- - , . -
- ;
- ,
, / ( - ) .
, (. 1.2).
. 1.2.
-
1 16
.
. , OLTP (On-line transaction processing), () -
. ().
. .
. :
- SQL (Structured
Query Language);
. - OLAP (On-line analytical processing),
;
. - Data Mining (" ").
1.2.
, -
-. ().
! ,
, .
, , .
, - , .
- , . -
, , , - 60- .
. . - .
-
17
( ) Information Management System (IMS) IBM. 1968 .
, . Data Base Task Group (DBTG) Conference on Data Systems Languages (CODASYL). DBTG 1971 .
, - , - . Integrated Database Management System (IDMS) Cullinet Soft-ware, Inc.
. "-" relatio . . . , 1970 . 12 .
1. . , -. , - , , . - , -, . , .
2. - , ( 2, 3). , . . , , . - , . - . - , .
3. NULL , NULL. 0.
-
1 18
4. : . , . - : (, ), , . , . . - .
5. . - SQL.
6. - , . (View). . - ( , ).
7. , - . - . SQL.
8. , , ( , .).
9. , , ( ). - , .
10. . :
;
-, .
-
19
, , , . .
, -, . -
: . - , -
.
11. -
, .
12. ,
. -,
. - .
- , . - , ()
. .
, . ().
, , , - . :
1- , , , ( );
2- , 1- , ( -
);
3- , 2- ,
( - ) . .
, . ,
, , . ,
BLOB
-
1 20
(Binary Large OBjects). -
, . - , ,
.
-
- .
. - . , , , , -
, .
, - ( ) -
. - .
! , -
. .
, , -
, -. , -
, , . , ,
, , . (ROLLBACK). , -
, (COMMIT).
-
. -
, , - ( ),
. , , , , -
.
-. ,
-
21
. -
- .
. , .
. - OLTP-,
.
OLTP- - , - -
, , . .
.
, , (, Executive Information Systems). ,
, , - , - , -
. , - , ,
. , .
1.3. OLTP-
OLTP- - .
, , , . -, OLTP-,
. - , OLTP . -
. 1.1.
-
1 22
1 . 1
OLTP-
-
,
-
-
-
-
()
-
,
,
,
-
-
-
-
-
, 100 %
-
, OLTP
.
-
23
OLTP-
, , ,
. ,
,
(,
. .).
, -
, . .
, ,
, ,
,
. .
OLTP-, , , -
( ). -
" " -
.
.
OLTP-,
, .
- .
(,
, ,
).
.
.
, ,
( ).
, -
OLTP-, .
, .
. -
, (view), -
. -
, -
. (-
) .
-
1 24
OLTP- - . , , .
, - . , - , -
. OLTP- - . ,
. , , - (-, ).
, - , OLTP-
- . -, OLTP
, - . OLTP- (,
). , , - .
OLTP- - - . -
, (, , -
. .). - , - .
OLTP-, , , -
. , (, ) , . . -
. , OLTP, - -
. . -
.
, OLTP-, ,
-
25
. -
. - .
- , , . . , . .
, , - . . ,
. . 1.3 Oracle OLTP, . 1.4 , .
. 1.3. OLTP
. 1.4.
OLTP- - , . . . -
, , .
OLTP- , -
, . - , -
.
OLTP- .
-
1 26
, , .
: ,
. -, - .
, --
. , -- , (OLAP).
Data Mining.
: , - . -. -
.
-
, . , -
-.
, ,
() ,
.
, -
(OLTP-). , -
- .
OLTP-
- . OLTP-
.
OLTP -
. OLTP- -
.
-
2
2.1.
OLTP-
, ,
. 1.1, , - ().
-
. , , 1988 ., . 1992 . -
. "- ".
,
. -
, OLTP- . -
( , , . .) -
, , , , -
( ). -
() .
.
! -, -
, , , -
.
-
2 28
.
. ,
(, - , , . .). , -
, . ,
.
, -
(, - -,
, ). -
- .
, , - . ,
, -
, -. , .
. -
. -
. , ,
.
. , -
, , -. , , -
. , ,
. ,
. - . -
, -
. (. 2.1).
. , ,
-
29
, 1 % !
.
. , .
, , , , , . , , . .
, .
( -) . - .
. 2.1.
, . () . , . - (. 2.2). -:
, ;
, .
Data Mining 1. 1.1. 1.2. 1.3. OLTP-
2. 2.1. 2.2. 2.3. 2.4.
3. OLAP-3.1. 3.2. OLAP-3.3. 3.3.1. 3.3.2. 3.3.3. FASMI
3.4. OLAP-3.4.1. MOLAP3.4.2. ROLAP3.4.3. HOLAP
4. 4.1. Data Mining4.2. Data Mining4.2.1. Data Mining4.2.2. 4.2.3. 4.2.4.
4.3. Data Mining4.3.1. -4.3.2. 4.3.3. 4.3.4. 4.3.5. 4.3.6. 4.3.7. 4.3.8.
4.4. Data Mining4.4.1. (predictive) 4.4.2. (descriptive)
4.5. Data Mining4.5.1. 4.5.2. 4.5.3. 4.5.4.
4.6. 4.6.1. 4.6.2.
5. 5.1. 5.2. 5.2.1. 5.2.2. 5.2.3.
5.3. 5.3.1. 15.3.2. Naive Bayes
5.4. 5.4.1. " "5.4.2.
5.5. 5.5.1. 5.5.2. . 5.5.2. 5.5.3. Support Vector Machines (SVM)
5.6.
6. 6.1. 6.1.1. 6.1.2. 6.1.3.
6.2. 6.3. 6.3.1. Apriori6.3.2. Apriori
7. 7.1. 7.1.1. 7.1.2. , ,
7.2. 7.3. 7.3.1. 7.3.2.
7.3.3. kmeans (Hard-c-means) Fuzzy Means -
7.4. 7.4.1. -
7.4.2.
8. Data Mining8.1. 8.2. CWM8.2.1. CWM8.2.2. CWM8.2.3. Data Mining
8.3. CRISP8.3.1. CRISP8.3.2. CRISP8.3.3. CRISP
8.4. PMML8.5. Data Mining8.5.1. SQL/MM8.5.2. OLE DB Data Mining8.5.3. JDMAPI
9. Xelopes9.1. 9.2. Model9.2.1. Xelopes9.2.2. Model9.2.3.
9.3. Settings9.3.1. Settings9.3.2. Settings
9.4. Attribute9.4.1. Attribute9.4.2.
9.5. Algorithms9.5.1. 9.5.2. MiningAlgorithm9.5.3. MiningAlgorithm9.5.4. 9.5.5.
9.6. DataAccess9.6.1. 9.6.2. MiningInputStream9.6.3. Mining-9.6.4. , MiningInputStream
9.7. Transformation9.8. Xelopes9.8.1. 9.8.2. 9.8.3. 9.8.4.
1. 1.1. 1.2. 1.3. 1.4. 1.5. 1.6.
2. 2.1. 2.2. 2.2.1. 2.2.2. 2.2.3.
2.3. 2.4. , 2.5. 2.5.1. 2.5.2.
2.5.3.
2.6.
3. -