OPTIMAALSE PROJETEERIMISE MATEMAATILISED MEETODIDMER 9020

Aine maht6 EAP, E S

statsionaarõpe:     nädalatunnid 4.0     loenguid 2.0     praktikume 0.0     harjutusi 2.0

Aine eesmärkideks on: süvendada ettevalmistust toodete ja tootmisprotsesside modelleerimise ja optimeerimise

alal. Tutvustada valdkonna arengut anda uusimaid teadmisi ja oskusi iseseisvaks teadus- ja arendustööks, sh. probleemide,

teadusuuringute ja projektide eesmärkide püstitamiseks, teadusuuringute ja arendustöö läbiviimiseks vajalike teooriate, meetodite ja tarkvarasüsteemide tundmises, matemaatiliste ja eksperimentaalsete meetodite ja tarkvarasüsteemide rakendamises

AineprogrammSissejuhatus

1. Matemaatilised mudelid tehniliste süsteemide ja –protsesside modelleerimiseks ja vahendid nende loomiseks/kirjeldamiseks. Eksperimentaalsete mudelite kasutamine . Inseneri statistika (Engineering statistics)

a. Vastavusfunktsioonid (asendusmudelid) ja nende kasutamineb. Statistika ja projekteerimine. Eksperimentidest määratud mudelite

kasutamine tehniliste süsteemide ja protsesside kirjeldamisel. Katsete planeerimine. Engineering statistics and design. Statistical Design of Experiments. Use of statistical models in design.

c. Kaasaegsed meetodid asendusmudelite koostamiseks. Närvivõrkude metoodika kasutamine.

d. Mudeli täpsuse hindamine. Riskide arvestamine, töökindlus, ohutus. Statistical tests of Hypotheses. Analysis of Variance. Risk, reliability, and Safety

2. Insenerilahenduste optimeerimine, üldalused (Basics of Optimisation methods and search of solutions)

a. Optimeerimisülesannete tüübid (Types of Optimization Problem)b. Optimaalse valiku ülesanded. Mitmekriteriaalsed Pareto-optimaalsed lahendid.

(Multicriterial optimal decision theory. Pareto optimality, Use of malticriterial decision theory in engineering design).

c. Klassikalised optimeerimisülesanded. (Optimization by Differential Calculus. Lagrange multipliers. Examples of use the classical optimization method in engineering).

d. Matemaatilise planeerimise meetodid. Otsene ja duaalne planeerimise ülesanne. Duaalse ülesande kasutamine. Optimeerimisülesennete lahendite tundlikkuse analüüs. (Mathematical programming methods and their use for engineering design, process planning and manufacturing resource planning. Direct and Dual Optimization tasks. Sensitivity analysis).

e. Geneetilised optimeerimise algoritmid..

3. Tehniliste süsteemide ja protsesside simuleerimine. Simuleerimine kasutades juhuslike arvude generaatoreid ja SIMULINK’i

4. Toodete ja tootmisprotsesside optimaalne projekteerimine arendused. Optimeerimise kasutamise näited.

Iseseisev-praktiline töö: Praktiline töö koosneb järgnevast viiest tööst:

1. Katsetulemuste statistiline hindamine: peamiste statistiliste hinnangute leidmine katsetulemustele, kahe erineva katseseeria vahelise seose hindamine(korrelatsioon) , katseeeria varieeruvuse erinevuse hindamine.

2. Regressioonanalüüsi mudeli koostamine: mudeli parameetrite hindamine katsetulemuste alusel, saadud mudeli parameetrite usaldadusvahemike hindamine. Valitud mudeli sobivuse analüüs.

3. Närvivõrkude mudeli leidmine 4. Lineaarse (mittelineaarse) optimeerimisülesande lahendamine,

optimeerimise tulemuste analüüs5. Mudeli statistiline simuleerimine arvutil , simuleerimstulemuste

analüüs.

Õppekirjandus 1. Übi, E. Ekstreemumülesanded majanduses ja tehnikas, Külim, 2002, 176 lk. 2. Ravindran, A; K.M. Ragsdell, G.V. Reklaitis. Engineering optimization : methods and applications. Wiley, 2006, 667 lk. 3. Antoniou, A. Practical optimization : algorithms

1. SissejuhatusTehniliste süsteemide optimerimise mudelid on oma sisult kirjeldavad ja normatiivsed (normative or prescriptive), nende eesmärgiks on leida süsteemi parameetrite parimad väärtused eeldusel , et teada on süsteemi või protsessi kirjeldav mudel. Usaldusväärse analüütilise (teoreetilise) mudeli puudumisel tulebsee luua, kasutades katseid, vaatlusi jms, ning asendades analüütilise mudeli asendusmudeliga (surrogaatmudeliga) hinnates nn “vastavuspinda (vastavus funktsiooni)” (response surface) . Arvestades püstitatud ülesannet võib kursust esitada järgmise skeemiga.

Insenerilahendused arvutipõhistes süsteemides (Definition of Engineering tasks) Inseneriülesanded sisaldavad endas info teisendamist ja neid võib jagada:

Simuleerimise (informatsiooni (andmete) interpreteerimise) Analüüsi ülesanneteks Diagnostika ülesanneteks Sünteesi ülesanneteks

Erinevate ülesannete tüüpide puhul kasutatakse erinevaid lähenemisi. Simuleerimist kasutataks etteantud struktuuriga info/objekti analüüsiks eesmärgiga hinnata objektide käitumist sõltuvalt parameetritestAnalüüsil on teada mõjuvad parameetrid ja struktuur , analüüsitakse objekti käitumist parameetrite muutumisel.Diagnostika on simuleerimise vastasülesanne, püütakse leida parameetrid , millised tagavad nõutava väljundi/käitumiseSüntees on pöördülesanne analüüsile, kus püütakse lähtudes väljundist määrata sobiv struktuur ja parameetrid.

1.1 Vastavusfunkstsiooni hindamise metoodika. Response Surface Methodology (RSM) There is a difference between data and information. To extract information from data you have to make assumptions about the system that generated the data. Using these assumptions and physical theory you may be able to develop a mathematical model of the system. Generally, even rigorously formulated models have some unknown parameters..

Identifying of those unknown constants and fitting an appropriate response surface model from experimental data requires knowledge of Design of Experiments, regression modelling techniques, and optimization methods. The response surface equations give the response in terms of the several independent

Katsed, arvutikatsed,Protsesside

jälgimine, jms

Asendusmudel;Surrogaatmudel

(response surface)

Optimeerimine

Katsete plaanimine (DOE)

Simuleerimine, analüüs

Tehniliste süsteemide optimeerimine (skeem)

variables of the problem. If the response is plotted as a function of etc., we obtain a response surface. A powerful statistical procedure, that employs factorial analysis and regression analysis has been developed for the determination of the optimum operating condition on a response surface.

Response surface methodology (RSM) has two objectives:1. To determine with one experiment where to move in the next experiment so as to continually seek out the optimal point on the response surface.2. To determine the equation of the response surface near the optimal point.

Response surface methodology (RSM) uses a two-step procedure aimed at rapid movement from the current position into the region of the optimum. This is followed by the characterization of the response surface in the vicinity of the optimum by a mathematical model. The basic tools used in RSM are two-level factorial designs and the method of least squares (regression) model and its simpler polynomial forms.

1.2. Modelleerimine. ModellingA model is a representation or pattern of an idea or problem. That is, a model is a way to describe or present a problem in a way that aids in understanding or solving the problem. Models serve several purposes: The Purpose of Modelling1. To make an idea concrete. This is done by representing it mathematically, pictorially or symbolically. 2. To reveal possible relationships between ideas. Relationships of hierarchy, support, dependence, cause, effect, etc. can be revealed by constructing a model. We have to be careful, then, how much we let our models control our thinking. 3. To simplify the complex design problem to make it manageable or understandable. Almost all models are simplifications because reality is so complex. 4. The main purpose of modelling, which often includes all of the above three purposes, is to present a problem in a way that allows us to understand it and solve it.. Types of ModelsA. Visual. Draw a picture of it. If the problem is or contains something physical, draw a picture of the real thing--the door, road, machine, bathroom, etc. If the problem is not physical, draw a symbolic picture of it, either with lines and boxes or by representing aspects of the problem as different items--like cars and roads representing information transfer in a company. B. Physical. The physical model takes the advantages of a visual model one step further by producing a three dimensional visual model. C. Mathematical. Many problems are best solved mathematically.

1.3. Projekteerimisülesannete keerukus. Complexity theory for design Ülesande keerukus hindamisel lähtutakse kolme tüüpi keerukusest: Arvutuslik keerukus (arvutus mahukus) Kirjeldamise keerukus Mõistmise (äratundmise keerukus)

Complexity theory is part of the theory of computation dealing with the resources required during computation to solve a given problem. The most common resources are time (how many steps does it take to solve a problem) and space (how much memory does it take to

solve a problem). Other resources can also be considered, such as how many parallel processors are needed to solve a problem in parallel. Complexity theory differs from computability theory, which deals with whether a problem can be solved at all, regardless of the resources required.

The time complexity of a problem is the number of steps that it takes to solve an instance of the problem, as a function of the size of the input, using the most efficient algorithm. To understand this intuitively, consider the example of an instance that is n bits long that can be solved in n² steps. In this example we say the problem has a time complexity of n². Of course, the exact number of steps will depend on exactly what machine or language is being used. To avoid that problem, we generally use Big O notation . If a problem has time complexity O(n²) on one typical computer, then it will also have complexity O(n²) on most other computers, so this notation allows us to generalize away from the details of a particular computer.Big O notation is a type of symbolism used in complexity theory, computer science, and mathematics to describe the asymptotic behavior of functions. More exactly, it is used to describe an asymptotic upper bound for the magnitude of a function in terms of another, usually simpler, function.

Keerukuse hindamiseks kasutatakse järgmist skaalat (referents-funktsioone ( Big O notation) : Logaritmiline keerukus, O(log n) Lineaarne keerukus , O(n) Polünomaalne keerukus, O( n ) Eksponentsiaalne keerukus Faktoriaalne keerukus Dopelt-eksponentsiaalne keerukus, .

Logaritmile keerukuse korral ei sõltu aeg ülesande mahust (mõõtmetest). Arvutusmahtude seisukohalt on vastuvõetavad peamiselt logaritmilised ja lineaarsed keerukused.

2. Eksperimentidest määratud mudelite kasutamine tehniliste süsteemide ja protsesside optimeerimisel. Matemaatilise statistika kasutamine. Development of models. statistical decision theory.

2.1 Use of software tools (MATLAB and Excel) for statistical analysis.Statistics Toolbox extends MATLAB® to support a wide range of common statistical tasks. The Excel Data Analysis Tools represent also the wide list of statistical analysis tools. The following tasks are our special area of interest.Probability Distributions. Toolbox supports computations involving over 30 different common probability distributions, plus custom distributions which you can define. For each distribution, a selection of relevant functions is available, including density functions, cumulative distribution functions, parameter estimation functions, and random number generators. The toolbox also supports nonparametric methods for density estimation.Linear Models. In the area of linear regression, Statistics Toolbox has functions tocompute parameter estimates, predicted values, and confidence intervals forsimple and multiple regression, stepwise regression, ridge regression, andregression using response surface models. In the area of analysis of variance(ANOVA).

Nonlinear Models. For nonlinear regression models, Statistics Toolbox provides additional parameter estimation functions and tools for interactive prediction and visualization of multidimensional nonlinear fits. The toolbox also includes functions that create classification and regression trees to approximate regression relationships. Multivariate Statistics supports methods for the visualization and analysis of multidimensional data, including principal components analysis, factoranalysis, one-way multivariate analysis of variance, cluster analysis, andclassical multidimensional scaling.Statistical Process Control. In the area of process control and quality management, Statistics Toolbox provides functions for creating a variety of control charts, performing processcapability studies, and evaluating Design for Six Sigma (DFSS) methodologies. Design of Experiments. Statistics Toolbox provides tools for generating and augmenting fulland fractional factorial designs, response surface designs, and D-optimal designs. The toolbox also provides functions for the optimal assignment of units with fixed covariates.

2.1.1. Juhuslike suuruste hindamine. Probability Distributions. Descriptive Statistics . Estimation the parameters of Distribution.A typical data sample is distributed over a range of values, with some values occurring more frequently than others. Some of the variability may be the result of measurement error or sampling effects. For large random samples, however, the distribution of the data typically reflects the variability of the source population and can be used to model the data-producing process. Statistics computed from data samples also vary from sample to sample.Modelling distributions of statistics is important for drawing inferences from statistical summaries of data. Probability distributions are theoretical distributions, based on assumptions about a source population. They assign probability to the event that a random variable, such as a data value or a statistic, takes on a specific, discrete value, or falls within a specified range of continuous values. Choosing a model often means choosing a parametric family of probability distributions and then adjusting the parameters to fit the data. The choice of an appropriate distribution family may be based on a priori knowledge, such as matching the mechanism of a data-producing process to the theoretical assumptions underlying a particular family, or a posteriori knowledge, such as information provided by probability plots and distribution tests.Parameters can then be found that achieve the maximum likelihood of producing the data.In following we give an example of use the Excel Data Analysis Tool for estimation the probabilistic parameters: Descriptive Statistics function.

Normaalse jaotusega juhuslike arvude genereerimise näide Sigma =1,mean =10 Sigma =0,1,mean =10

9,699767841 10,0422376210,24425731 10,0089277111,19835022 9,844905233

7,81641236 10,1121602511,09502253 10,08380721

9,30979584 9,8729892928,153089109 10,047397529,226492946 10,05071079,432075128 10,1448106610,13485305 9,95751125

Juhuslike suuruste jaotust iseloomustavate näitajate hindamine

(Descriptive Statistics)

Column1 Column1

Mean 9,631 Mean 10,01655Standard Error 0,350 Standard Error 0,031003Median 9,566 Median 10,04482Mode #N/A Mode #N/AStandard Deviation 1,106 Standard Deviation 0,098042Sample Variance 1,224 Sample Variance 0,009612Kurtosis -0,482 Kurtosis -0,36557Skewness -0,201 Skewness -0,71427Range 3,382 Range 0,299905Minimum 7,816 Minimum 9,844905Maximum 11,198 Maximum 10,14481Sum 96,310 Sum 100,1655Count 10,000 Count 10Largest(1) 11,198 Largest(1) 10,14481Smallest(1) 7,816 Smallest(1) 9,844905Confidence Level(95,0%) for mean 0,792 Confidence Level(95,0%) for mean 0,070135

STANDARD ERROR The standard error σ is a method of measurement or estimation of the standard deviation of the sampling distribution associated with the estimation method. The term may also be used to refer to an estimate of that standard deviation, derived from a particular sample used to compute the estimate.

Let X be a random variable with mean value μ:

Here the operator E denotes the average or expected value of X. Then the standard deviation of X is the quantity

Sample variance In probability theory and statistics, the variance is used as a measure of how far a set of numbers are spread out from each other. It is one of several descriptors of a probability distribution, describing how far the numbers lie from the mean (expected value).

Kurtosis

In probability theory and statistics, kurtosis (from the Greek word κυρτός, kyrtos or kurtos, meaning bulging) is a measure of the "peakedness" of the probability distribution of a real-valued random variable, although some sources are insistent that heavy tails, and not peakedness, is what is really being measured by kurtosis. Higher kurtosis means more of the variance is the result of infrequent extreme deviations, as opposed to frequent modestly sized deviations.

The fourth standardized moment is sometimes used as the definition of kurtosis.

It is defined as

where μ4 is the fourth moment about the mean and σ is the standard deviation.

Skewness In probability theory and statistics, skewness is a measure of the asymmetry of the probability distribution of a real-valued random variable. The skewness value can be positive or negative, or even undefined. Qualitatively, a negative skew indicates that the tail on the left side of the probability density function is longer than the right side and the bulk of the values (possibly including the median) lie to the right of the mean. A positive skew indicates that the tail on the right side is longer than the left side and the bulk of the values lie to the left of the mean. A zero value indicates that the values are relatively evenly distributed on both sides of the mean, typically but not necessarily implying a symmetric distribution.

2.1.2. Korrelation. Seosed juhuslike arvude vahel. Correlation. In statistics, dependence refers to any statistical relationship between two random variables or two sets of data. Correlation refers to any of a broad class of statistical relationships involving dependence.

Figure Correlation examples.

Näide korrelatsiooni hindamisest Example of estimating the correlation using Excel Data analysis

x y z3 3 8

1 2 75 4 46 7 27 6 38 5 64 4 45 4 22 3 25 4 7

Korrelatsiooni hindamise näide Excelis

  Column 1 Column 2 Column 3Column 1 1Column 2 0,825593 1Column 3 -0,22052 -0,48651 1

Kodutöö 1. Katsetulemuste statistiline hindamine Kasutatades Excel Data analysis vahendeid hinnata juhuslike suuruste jaotust iseloomustavaid näitajaid (Descriptive Statistics) lahendades järgmisi ülesandeid:

1. peamiste statistiliste hinnangute leidmine katsetulemustele. Hindamise aluseks on vähemalt 2 erinevat vabalt valitud katseseeriat. Statistilised hinnangud tuleb leida mõlemale seeriale.

2. kahe erineva katseseeria vahelise seose hindamine(korrelatsioon, kovariatsioon )

3. katseeeria varieeruvuste erinevuse hindamine, kasutatdes F kriteeriumit võrreleda, kas rinevate kats ete tulemuste vareeruvuste erinevus on juhuslik. So hinnatakse katsetulemuste dispersioone

vastavalt abil hinnatakse suhet = ja

võrreldakseseda F funkstiooni tabelväärtusega etteantud usaldatavusega (tavaliselt 5% )(F tabelväärtuyste leidmisel on vabadusastmetekas vastavalt katsete arvud ) .Kui siis loetakse , et kehtib nn ” nullhüpotees” ja erinevus varieeruvuste vahel erinevates katsetulemustes on juhuslik.

Ülesande lahendamiseks võib kasutada ka Excel Data Analysis funktsiooniF-Test two –Sample for Variance.

Näide analüüsist funktsiooniga : F-Test Two-Sample for Variances

 Variable 

1 Variable 2

Mean 4,642857 4,071429Variance 6,708791 3,917582Observations 14 14df 13 13F 1,712482P(F<=f) one-tail 0,172124F Critical one-tail 2,576927

Küsimused seoses kodutööga 1 Seletada ja võrrelda erinevaid juhuslike suuruste jaotust

iseloomustavaid näitajaid Anda hinnang ja erinevate katsetulemuste omavahelisele seosele Hinnata erinevate katsetulemuste (sh varieeruvuse) erinevust.

2.1.3. Lineaarse regressiooni mudelid. Linear regression Models.

Linear regression is widely used in science to describe relationships between variables. It ranks as one of the most important tools used. Researchers usually include several variables in their regression analysis in an effort to remove factors that might produce spurious correlations. However, it is never possible to include all possible confounding variables in a study employing regression. For this reason, randomized experiments are considered to be more trustworthy than a regression analysis.Linear models represent the relationship between a continuous response variable and one or more predictor variables (either continuous or categorical) in the form Where y is an n-by-1 vector of observations of the response variable. X is the n-by-p design matrix determined by the predictors.

is a p-by-1 vector of unknown parameters to be estimated. is an n-by-1 vector of independent, identically distributed random disturbances.

Näide: AlgandmedKuu Teenus Nõudlus

1 1 102 1 123 1 114 1 135 1 156 1 1310 1 131 2 1002 2 1013 2 1024 2 1055 2 1026 2 1017 2 102

Regressiooni analüüsi tulemuste näideSUMMARY OUTPUT

Regression StatisticsMultiple R 1,000R Square 0,999Adjusted R Square 0,999Standard Error 1,485Observations 14,000ANOVA

  df SS MS FSignificance

FRegression 2,000 27997,457 13998,728 6347,977 0,000Residual 11,000 24,257 2,205Total 13,000 28021,714      

  CoefficientsStandard

Error t Stat P-value Lower 95% Upper 95%Intercept -78,350 1,487 -52,682 0,000 -81,624 -75,077X Variable 1 0,278 0,164 1,692 0,119 -0,084 0,640X Variable 2 89,548 0,797 112,373 0,000 87,794 91,302RESIDUAL OUTPUTObservation Predicted Y Residuals

1 11,476 -1,4762 11,753 0,2473 12,031 -1,0314 12,309 0,6915 12,587 2,4136 12,865 0,1357 13,977 -0,9778 101,023 -1,0239 101,301 -0,301

10 101,579 0,42111 101,857 3,14312 102,135 -0,13513 102,413 -1,41314 102,691 -0,691

Linearization TransformationsThe computational difficulties associated with nonlinear regression analysis sometimes can be avoided by using simple transformations that convert a problem that is nonlinear into one that can be handled by simple linear regression analysis. The most common transformations are given in following Table. The reader needs to be aware that logarithmic transformation can introduce bias in the prediction of the response variable.

TABLE Some common linearization transformation, W = c + dV

Nonlinear equation Linearized equation Linearized variablesW V

y = a + bx (linear) y = a + bx y xy = (logarithmic) ln y=ln a + b ln x ln y ln x

y = bxae (exponential) ln y= ln a+bx ln y x(exponential) x

(square root) y=a+bx yy=a+b/x (inverse) y=a+bx y 1/x

Nonlinear regressionNonlinear regression in statistics is the problem of fitting a model

to multidimensional x,y data, where f is a nonlinear function of x with parameters θ.In general, there is no algebraic expression for the best-fitting parameters, as there is in linear regression. Usually numerical optimization algorithms are applied to determine the best-fitting parameters.

2.1.4. Regressioonanalüüs. Vähimruutude meetod. Regression analysis. Method of Least squaresRegression analysis is any statistical method where the mean of one or more random variables is predicted conditioned on other (measured) random variables. Regression analysis is the statistical view of curve fitting: choosing a curve that best fits given data points.Sometimes there are only two variables, one of which is called X and can be regarded as non-random, because it can be measured without substantial error and its values can even be chosen at will. For this reason it is called the independent or controlled variable. The other variable called Y, is a random variable called the dependent variable, because its values depend on X. In regression we are interested in the variation of Y on X.This dependence is called the regression of Y on X.

Regression is usually posed as an optimization problem as we are attempting to find a solution where the error is at a minimum. The most common error measure that is used is the least squares: this corresponds to a Gaussian likelihood of generating observed data given the (hidden) random variable. Regression can be expressed as a maximum likelihood method of estimating the parameters of a model. The earliest form of linear regression was the method of least squares, which was published by Legendre in 1805, and by Gauss in 1809. The term "least squares" is from Legendre's term, moindres carrés. However, Gauss claimed that he had known the method since 1795.

2.1.5 Analysis of varianceIn statistics, analysis of variance (ANOVA) is a collection of statistical models and their associated procedures which compare means by splitting the overall observed variance into different parts.

ANOVA is a particular form of statistical hypothesis testing used in the analysis of experimental data. A test result (calculated from the null hypothesis and the sample) is called statistically significant if it is deemed unlikely to have occurred by chance, assuming the truth of the null hypothesis. A statistically significant result (when a probability (p-value) is less than a threshold (significance level)) justifies the rejection of the null hypothesis.

In the typical application of ANOVA, the null hypothesis is that all groups are simply random samples of the same population. Rejecting the null hypothesis implies that different treatments result in altered effects.

.

ANOVA uses traditional standardized terminology. The definitional equation of sample

variance is , where the divisor is called the degrees of freedom (DF), the summation is called the sum of squares (SS), the result is called the mean square (MS) and the squared terms are deviations from the sample mean. For regression analysis ANOVA is based on partitioning of the sum of squares and estimates 3 sample variances:

a total variance based on all the observation deviations, an error variance based on all the observation deviations from their appropriate

treatment means and a treatment variance. the treatment variance based on the deviations of treatment means from the grand

mean, the result being multiplied by the number of observations in each treatment to account for the difference between the variance of observations and the variance of means.

The fundamental technique is a partitioning of the total sum of squares into components related to the effects in the model used. For example, we show the model for a simplified ANOVA with one type of treatment at different levels. (If the treatment levels are quantitative and the effects are linear, a linear regression analysis may be appropriate.)

SSTotal = SSError + SSTreatments.

The number of degrees of freedom (abbreviated df) can be partitioned in a similar way and specifies the chi-square distribution which describes the associated sums of squares.

dfTotal = dfError + dfTreatments.

Analyses of variance lead to F-tests of statistical significance using Fisher's F-distribution.

Kodutöö 2. Regressioonanalüüsi mudeli koostamine.Regression analysis based on Microsoft Excel Data analysis tools.

Microsoft Excel provides a set of Data Analysis Tools (called the Analysis ToolPak) that could be used for development complex statistical or engineering analysis.The Analysis Toolpak includes the tool for regression analysis. The Regression analysis tool perform linear regression analysis by using the “least square“ method to fit line through a set of observations.

In following figures the inputs data, setting the parameters for regression analysis task and output the results are represented.

The Regression tool in Excel’s Data Analysis determines the coefficients (ai) that yield the smallest residual sum of squares, which is equivalent to the greatest correlation coefficient squared, R2, for Equation (1). This is known as linear regression analysis.

y = a0 + a1*x1 + a2*x2 + a3*x3 + …. (1)

where y is the dependent variable (response), a0 is the intercept, and x1, x2, x3 etc. are the independent variables (factors). It is assumed that you have n observations of y versus different values of xi.

Note that the xi can be functions of the actual experimental variables, including products of different variables. Following are the meanings of the results produced by Excel:

R Square = (Multiple R)2 = R2 = 1 - Residual SS / Total SS = Regress SS / Total SS

(roughly the fraction of the variation in y that is explained by equation 1).

R = correlation coefficient

Adjusted R Square = 1 - (Total df / Residual df)(Residual SS / Total SS)

Standard Error = (Residual MS)0.5

Note that “Error” does not mean there’s a mistake or an experimental error. It’s just a

definition related to residuals. This word is used a lot in statistical analysis, and is

misunderstood by those who don’t know statistics.

ANOVA = ANalysis Of VAriance

Regression df = regression degrees of freedom = number of independent variables

(factors) in equation 1.

Regression SS = Total SS - Residual SS

Regression MS = Regression SS / Regression df

Regression F = Regression MS / Residual MS

Significance F = FDIST(Regression F, Regression df, Residual df) = Probability that

equation (1) does NOT explain the variation in y, i.e. that any fit is purely by chance.

This is based on the F probability distribution. If the Significance F is not less than 0.1

(10%) you do not have a meaningful correlation.

Residual df = residual degrees of freedom = Total df - Regression df = n - 1 - number of

independent variables (xi)

Residual SS = sum of squares of the differences between the values of y predicted by

equation 1 and the actual values of y. If the data exactly fit equation 1, then Residual SS

would be 0 and R2 would be 1.

Residual MS = mean square error = Residual SS / Residual df

Total df = total degrees of freedom = n – 1

Total SS = the sum of the squares of the differences between values of y and the average y

= (n-1)*(standard deviation of y)2

Coefficients = values of ai which minimize the Residual SS (maximize R2). The Intercept

Coefficient is a0 in equation 1.

Standard error = (Residual MS using only the Coefficient for that row)0.5

t Stat = Coefficient for that variable / Standard error for that variable

P-value = TDIST(|t Stat|, Residual df, 2) = the Student’s t distribution two-tailed

probability

If one divides this by 2, it is the probability that the true value of the coefficient has the

opposite sign to that found. You want this probability to be small in order to be sure that

this variable really influences y, certainly less than 0.1 (10%). If near 50%, try another fit

with this variable left out.

There is a 95% probability that the true value of the coefficient lies between the Lower

95% and Upper 95% values. The probability is 2.5% that it lies below the lower value,

and 2.5% that it lies above. The narrower this range the better. If the lower value is

negative and the upper value positive, try correlating the data with this variable left out. If

the resulting R2 and Significance F are either improved or little changed, then the data do

not support inclusion of that variable in your correlating equation.

Regressioonanalüüsi näide.

1. Lähteandmed ja pöördumine Microsoft Excel Data Analysis Tools (the Analysis

ToolPak) poole.

Figure Representation the data of experiments and Setting of Regression Analysis

2. Regressioonanalüüsi tulemuste näide

Figure Example of output data (results of Regression analysis) using Excel

Analüüsi tulemusena saaadi lineaarne mudel kujul (As the result, the linear model is received in form):

y=a0 + a1 x1 + a2 x2 +a3 x1*x2=15 + 0,725 x1 + 0,625 x2 +0,275 x1*x2.

Mudeli parameetrite täpsust hinnati usaldusvahemikega vastavalt. The confidence intervals for parameters of regression model are (with a 95% degree of confidence):

14,455<= a0 <= 15,5450,058 <= a1 <= 1,392-0,042 <= a2 <= 1,292-0,392 <= a3 <= 0,942.

Küsimused seoses kodutööga 2 Valida katsetulemuste kirjeldamiseks sobiv mudel. Esitada

regresioonanalüüsi tulemusena saadud mudel Hinnata valitud mudeli sobivust katsetulemuste kirjeldamiseks.

Millisel viisil oleks otstarbekas mudelit arendada? Hinnata leitud mudeli (tema parameetrite) täpsust

2.2. Katsete planeerimine. Design of Experiments (DOF).

There is a world of difference between data and information. To extract information from data you have to make assumptions about the system that generated the data. Using these assumptions and physical theory you may be able to develop a mathematical model of the system. Generally, even rigorously formulated models have some unknown constants.The goal of experimentation is to acquire data that enable you to estimate these constants. But why do you need to experiment at all? You could instrument the system you want to study and just let it run. Sooner or later you would have all the data you could use. In fact, this is a fairly common approach. There are three characteristics of historical data that pose problems for statistical modelling: Suppose you observe a change in the operating variables of a system followed by a change in the outputs of the system. That does not necessarily mean that the change in the system caused the change in the outputs.A common assumption in statistical modelling is that the observations are independent of each other. This is not the way a system in normal operation works.Controlling a system in operation often means changing system variables in tandem. But if two variables change together, it is impossible to separate their effects mathematically. Designed experiments directly address these problems. The overwhelming advantage of a designed experiment is that you actively manipulate the system you are studying. With Design of Experiments (DOE) you may generate fewer data points than by using passive instrumentation, but the quality of the information you get will be higher.

A strategy for design of experiments was first introduced in the early 1920s when a scientist at a small agricultural research station in England, Sir Ronald Fisher, showed how one could conduct valid experiments in the presence of many naturally fluctuating conditions such as temperature, soil condition, and rainfall. In the past decade, the application of DOE has gained acceptance as an essential tool for improving the quality of goods and services. This recognition is partially due to the work of Genichi Taguchi, a Japanese quality expert, who promoted the use of DOE in designing robust products--those relatively insensitive to environmental fluctuations. The well-planned experimentation can provide data that will enable device manufacturers to identify the causes of performance variations and to eliminate or reduce such variations by controlling key process parameters, thereby improving product quality. Properly designed and executed experiments will generate more-precise data while using substantially fewer experimental runs than alternative approaches. They will lead to results that can be interpreted using relatively simple statistical techniques, in contrast to the information gathered in observational studies, which can be exceedingly difficult to interpret.

Design of Experiments provides essential methods and strategies for persons having responsibility for improving current processes or products or charged with developing new processes or products. Process or product development engineers or managers, manufacturing engineers and managers, research scientists, engineers, or managers, and technical personnel in quality assurance or engineering will benefit personally from this technique. These individuals will be better prepared to plan, collect, and interpret data from experiments designed to provide real-world knowledge for gaining a competitive edge. Objectives

To understand the role that designed experiments can play in process improvement. To develop a background for pursuing further studies in experimental design.

Continuing studies in designed experiments enables participants to increase their value to the organization.

To learn from faculty who have themselves conducted experiments in industry. This course teaches practical application, not theory.

2.2.1. Experiment Planning Strategy. ConceptsStudies, often called process characterization, can be done by varying the key elements of the process (i.e., equipment, materials, and input parameters, and so forth) and determining which sources of variation have the most impact on process performance. The process should also be challenged to discover how outputs change as process variables fluctuate within allowable limits. This testing is essential to learning what steps must be taken to protect the process if worst-case conditions for input variables ever occur during actual manufacturing operations. Once again, an effective method for studying various combinations of variables is DOE. In particular, simple two-level factorial and fractional factorial designs are useful techniques for worst-case-scenario studies. In 1950, Gertrude Mary Cox and William Cochran published the book Experimental Design which became the major reference work on the design of experiments for statisticians for years afterwards.

StatisticsStatistics is a type of data analysis which practice includes the planning, summarizing, and interpreting of experiments (observations) of a system possibly followed by predicting or forecasting of future events based on a mathematical model of the system being observed. Statistics is a branch of applied mathematics specifically in the area of statistical theory which uses probability theory in the mathematical models. The basic tenet of statistics is that a population can be represented by a sample of the population when the sample is sufficiently large and when the sample is composed of a random selection of units from the population. Statistical theory provides methods for determining how large a sample is needed to provide for statistically significant results.

RandomizationRandomization is a core principle in the statistical theory of design of experiments. Its use was extensively promoted by R.A. Fisher in his book Statistical Methods for Research Workers. Randomization involves randomly allocating the experimental units across the treatment groups. Randomization is not haphazard; it serves a purpose in both frequentist and Bayesian statistics. By randomizing the order in which the test combinations are run, researchers can eliminate the effects of unknown trending variables on the results of the experiment. A frequentist would say that randomization reduces bias by equalising other factors that have not been explictly accounted for in the experimental design. Considerations of bias are of little concern to Bayesians, who recommend randomization because it produces ignorable designs. In design of experiments, frequentists prefer Completely Randomized Designs. Other experimental designs are used when a full randomization is not possible. Randomization is also required in the generation of random numbers for scientific research and cryptography. Random number generators are sometimes used for these purposes.

BlockingIn the statistical theory of the design of experiments, blocking is the arranging the experimental units in groups (blocks) which are similar to one another. This reduces sources of variability and thus leads to greater precision. Suppose we have invented a process which we believe makes the soles of shoes last longer, and we wish to conduct a field trial. Blocking can be used to prevent experimental results being influenced by variations from batch to batch, machine to machine, day to day, or shift to shift

Orthogonality

In mathematics, orthogonal is synonymous with perpendicular when used as a simple adjective that is not part of any longer phrase with a standard definition.. It comes from the Greek "ortho", meaning "right", and "gonia", meaning "angle". Two streets that cross each other at a right angle are orthogonal to each other. Two vectors in an inner product space are orthogonal if their inner product is zero. If the vectors are x and y this is written . The word normal is sometimes also used for this concept by mathematicians, although that word is rather overburdened.Two subspaces are called orthogonal if each vector in one is orthogonal to each vector in the other. Note however that this does not correspond with the geometric concept of perpendicular planes. The largest subspace that is orthogonal to a given subspace is its orthogonal complement.Orthogonality is a system design property which enables the making of complex designs feasible and compact. The aim of an orthogonal design is to guarantee that operations within one of its components neither create nor propagate side-effects to other components. For example a car has orthogonal components and controls, e.g. accelerating the vehicle does not influence anything else but the components involved in the acceleration. On the other hand, a car with non orthogonal design might have, for example, the acceleration influencing the radio tuning or the display of time. Consequently, this usage is seen to be derived from the use of orthogonal in mathematics; one may project a vector onto a subspace, by projecting it each member of a set of basis vectors separately and adding the projections if and only if the basis vectors are mutually orthogonal.Orthogonality guarantees that modifying the technical effect produced by a component of a system neither creates or propagates side effects to other components of the system. The emergent behaviour of a system consisting of components should be controlled strictly by formal definitions of its logic and not by side effects resulting from poor integration, i.e. non-orthogonal design of modules and interfaces. Orthogonality reduces the test and development time, because it's easier to verify designs that neither cause side effects nor depend on them.

ExperimentsIn the scientific method, an experiment is a set of actions and observations, performed to verify or falsify a hypothesis or research a causal relationship between phenomena. The experiment is a cornerstone in empirical approach to knowledge.

Design of experiments attempts to balance the requirements and limitations of the field of science in which one works so that the experiment can provide the best conclusion about the hypothesis being tested.In some sciences, such as physics and chemistry, it is relatively easy to meet the requirements that all measurements be made objectively, and that all conditions can be kept controlled across experimental trials. On the other hand, in other cases such as biology, and medicine, it is often hard to ensure that the conditions of an experiment be performed consistently; and in the social sciences, it may even be difficult to determine a method for measuring the outcomes of an experiment in an objective manner.For this reason, sciences such as physics are often referred to as "hard sciences", while others such as sociology are referred to as "soft sciences"; in an attempt to capture the idea that objective measurements are often far easier in the former, and far more difficult in the latter.As a result of these considerations, experimental design in the "hard" sciences tends to focus on the elimination of extraneous effects (type of flour, impurities in the water); while experimental design in the "soft" sciences focuses more on the problems of external validity, often through the use of statistical methods. Occasionally events occur naturally from which

scientific evidence be drawn, natural experiments. In such cases the problem of the scientist is to evaluate the natural "design".Controlled experiments. Many hypotheses in sciences such as physics can establish causality by noting that, until some phenomenon occurs, nothing happens; then when the phenomenon occurs, a second phenomenon is observed. But often in science, this situation is difficult to obtain.A controlled experiment generally compares the results obtained from an experimental sample against a control sample, which is practically identical to the experimental sample except for the one aspect whose effect is being tested. In many laboratory experiments it is good practice to have several replicate samples for the test being performed and have both a positive control and a negative control. The results from replicate samples can often be averaged, or if one of the replicates is obviously inconsistent with the results from the other samples, it can be discarded as being the result of some an experimental error (some step of the test procedure may have been mistakenly omitted for that sample). Most often, tests are done in duplicate or triplicate. A positive control is a procedure that is very similar to the actual experimental test but which is known from previous experience to give a positive result. A negative control is known to give a negative result. The positive control confirms that the basic conditions of the experiment were able to produce a positive result, even if none of the actual experimental samples produce a positive result. The negative control demonstrates the base-line result obtained when a test does not produce a measurable positive result; often the value of the negative control is treated as a "background" value to be subtracted from the test sample results. Controlled experiments can be particularly useful when it is difficult to exactly control all the conditions in an experiment. The experiment begins by creating two or more sample groups that are probabilistically equivalent, which means that measurements of traits should be similar among the groups and that the groups should respond in the same manner if given the same treatment. This equivalency is determined by statistical methods that take into account the amount of variation between individuals and the number of individuals in each group. In fields such as microbiology and chemistry, where there is very little variation between individuals and the group size is easily in the millions, these statistical methods are often bypassed and simply splitting a solution into equal parts is assumed to produce identical sample groups.Once equivalent groups have been formed, the experimenter tries to treat them identically except for the one variable that he or she wishes to isolate.

ErrorsAn error has different meanings in different domains. Current meanings in some of those domains are described below. The Latin word error meant "wandering" or "straying".Statistics. An error is a difference between a computed, estimated, or measured value and the true, specified, or theoretically correct value.Experimental science. An error is a bound on the precision and accuracy of the result of a measurement. These can be classified into two types: statistical error (see above) and systematic error. Statistical error is caused by random (and therefore inherently unpredictable) fluctuations in the measurement apparatus, whereas systematic error is caused by an unknown but nonrandom fluctuation. If the cause of the systematic error can be identified, then it can usually be eliminated.An error is a difference between desired and actual performance. Engineers often seek to design systems in such a way as to mitigate or preferably avoid the effects of error. One type of error is human error. Human factors engineering is often applied to designs in an attempt to minimize this type of error by making systems more forgiving or error tolerant. Errors in a

system can also be latent design errors that may go unnoticed for years, until the right set of circumstances arises that cause them to become active.

2.3. Traditional vs. Factorial DesignsOne traditional method of experimentation is to evaluate only one variable (or factor) at a time-all of the variables are held constant during test runs except the one being studied. This type of experiment reveals the effect of the chosen variable under set conditions; it does not show what would happen if the other variables also changed. It was Fisher's idea that it was much better to vary all the factors at once using a factorial design, in which experiments are run for all combinations of levels for all of the factors. With such a study design, testing will reveal what the effect of one variable would be when the other factors are changing. The advantage of the factorial design is also its efficiency. Montgomery has shown that this relative efficiency of the factorial experiments increases as the number of variables increases. In other words, the effort saved by such internal replication becomes even more dramatic as more factors are added to an experiment.

Two level Factorial DesignA factorial experiment is a statistical study in which each observation is categorised according to more than one factor. Such an experiment allows studying the effect of each factor on the response variable, while requiring fewer observations than by conducting separate experiments for each factor independently. It also allows studying the effect of the interaction between factors on the response variable.A two-factor, two-level factorial design is normally set up by building a table using minus signs to show the low levels of the factors and plus signs to show the high levels of the factors. Table and figure shows a factorial design for the application example. The simplest factorial experiment is the 22 factorial experiment, so named because it considers two levels for each of two factors, producing 22 = 4 factorial points. Suppose an engineer wishes to study the total power used by each of two different motors, A and B, running at each of two different speeds, 2000 or 3000 RPM. The factorial experiment would consist of four points: motor A at 2000 RPM, motor B at 2000 RPM, motor A at 3000 RPM, and motor B at 3000 RPM.

Run no. Level of Factor Values of parameters Response

1 +1 +1 +1 3000 3000 15,82 -1 +1 -1 2000 3000 14,73 +1 -1 -1 3000 2000 14,54 -1 -1 +1 2000 2000 14,5

The first column in the table of plan of experiments shows the run number for the four possible runs. The next two columns show the level of each main factor, A and B, in each run, and the fourth column shows the resulting level of the interaction between these factors, which is found by multiplying their coded levels (-1 or +1). Columns 5 and 6 show the actual values assigned to the low and high variable levels in the design. Test runs using each of these four combinations constitute the experiment. The last column contains the responses from the experiment. Filling in this column requires the hard work of running each experiment and then recording the result.

To use the standard plans the following coding of parameters is needed:

This coding corresponds to the following linear transformation of the initial parameters :

To save space, the points in a factorial experiment are often abbreviated as −1 −1 ; +1 −1 , −1 +1 , and +1 +1 . The factorial points can also be abbreviated by (1), a, b, and ab, where the presence of a letter indicates that the specified factor is at its high (or second) level and the absence of a letter indicates that the specified factor is at its low (or first) level. So "a" indicates that factor A is at its high setting, while all other factors are at their low (or first) setting. (1) is used to indicate that all factors are at their lowest (or first) values.Ideally, a factorial experiment should be replicated. Replication allows a researcher to estimate experimental error. If the magnitude of experimental error is unknown (which happens if it cannot be estimated), then a researcher cannot determine whether any effects or interactions are statistically significant. If a design cannot be replicated, it may be possible to get some estimate of experimental error by invoking the effect sparsity principle and/or the main effects principle. However, these principles usually don't apply unless several factors are being considered in the same experiment. The experimental runs in a factorial experiment should also be randomized. Randomization attempts to reduce the impact that bias could have on the experimental results.In some studies there may be more than two important variables. The three-factor, two-level design is shown in Figure . With three two-level factors, eight experiments will be required, and there will be four replicates of each level of each factor, further increasing the precision of the result. There will be three two-factor interactions and a three-factor interaction to evaluate. Usually, interactions involving three or more factors are not important and can be disregarded.

Sõltuvalt eesmärgist võib kasutada erienevaid katseplaane: keskpuntiga kahetasandiga kompsiitplaan, kolmetasandeline katseplaan keskpuntiga jm.

Joonis Keskpunktiga komposiitne plaan (Central composite designs)

Joonis Kolmetasandeline Box-Behnken’i katseplaan ( A Box-Behnken design for three factors. /Douglas C. Montgromery. Design and Analysis of Experiments, 2001/Joonis Tsentraalne komposiit-katseplaan (A face-centered central composite design for k=3. /Douglas C. Montgromery. Design and Analysis of Experiments, 2001/

Calculation of EffectsCalculation of the Main Effects. With a factorial design, the average main effect of changing thrombin level from low to high can be calculated as the average response at the high level minus the average response at the low level:

The fact that these effects have a positive value indicates that the response (i.e., the coagulation rate) increases as the variables increase. The larger the magnitude of the effect, the more critical the variable.

Estimate of the Interaction. A factorial design makes it possible not only to determine the main effects of each variable, but also to estimate the interaction (i.e., synergistic effect) between the two factors, a calculation that is impossible with the one- factor-at-a-time experiment design. Interaction is a kind of action which occurs as two or more objects have an effect upon one another. The idea of a two-way effect is essential in the concept of interaction instead of a one-way causal effect. Combinations of many simple interactions can lead to surprising emergent phenomena. It has different tailored meanings in various sciences.The interaction effect is the average difference between the effect of thrombin at the high level of ion concentration and the effect of thrombin at the low level of ion concentration, or As in a two-factor experiment, the average effect of each factor can be calculated by subtracting the average response at the low level from the average response at the high level.

Adding Centre Points Designs with factors that are set at two levels implicitly assume that the effect of the factors on the dependent variable of interest (e.g., fabric Strength) is linear. It is impossible to test whether or not there is a non-linear (e.g., quadratic) component in the relationship between a factor A and a dependent variable y, if A is only evaluated at two points (.i.e., at the low and high settings). If one suspects that the relationship between the factors in the design and the dependent variable is rather curve-linear, then one should include one or more runs where all (continuous) factors are set at their midpoint. Such runs are called centre-point runs, since they are, in a sense, in the centre of the design (see graph).

Central Composite DesignsAs pointed out before, in order to estimate the second order, quadratic, or non-linear component of the relationship between a factor and the dependent variable, one needs at least 3 levels for the respective factors. What does the information function look like for a simple 3-by-3 factorial design, for the second-order quadratic model?

The first four runs in this design are the previous 2-by-2 factorial design points (or square points or cube points); runs 5 through 8 are the so-called star points or axial points, and runs 9 and 10 are centre points.

The information function for this design for the second-order (quadratic) model is rotatable, that is, it is constant on the circles around the origin.

The two design characteristics discussed so far -- orthogonality and rotatability -- depend on the number of centre points in the design and on the so-called axial distance (alpha), which is the distance of the star points from the centre of the design (i.e., 1.414 in the design shown above). It can be shown that a design is rotatable if:

= ( nc )¼

where nc stands for the number of cube points in the design (i.e., points in the factorial portion of the design).

A central composite design is orthogonal, if one chooses the axial distance so that:

= {[( nc + ns + n0 )½ - nc½]² * nc/4}¼

wherenc is the number of cube points in the designns is the number of star points in the designn0 is the number of centre points in the design

To make a design both (approximately) orthogonal and rotatable, one would first choose the axial distance for rotatability, and then add centre points, so that:

n0 4*nc½ + 4 - 2k

where k stands for the number of factors in the design.

Finally, if blocking is involved give the following formula for computing the axial distance to achieve orthogonal blocking, and in most cases also reasonable information function contours, that is, contours that are close to spherical:

= [k*(l+ns0/ns)/(1+nc0/nc)]½

wherens0 is the number of centre points in the star portion of the designns is the number of non-centre star points in the designnc0 is the number of centre points in the cube portion of the designnc is the number of non-centre cube points in the design

2.4. Fractional Factorial DesignsOne disadvantage of two-level factorial designs is that the number of treatment combinations in a factorial design increases rapidly with an increase in the number of factors. Thus, if n = 4,

=16, but if n = 8, =256. Since engineering experimentation can easily involve 6 to 10 factors, the number of experiments required can rapidly become prohibitive in cost.Consider a factorial experiment. There are:

6 main effects 15 first-order interactions 20 second-order interactions 15 third-order interactions 6 fourth-order interactions 1 fifth-order interaction

Since we usually are not interested in higher-order interactions, we are collecting a great deal of extraneous information if we perform all 64 treatment combinations. This permits us to design an experiment that is some fraction of the total factorial design.

In this case, a fractional factorial design may be used.Fortunately, because three-factor and higher-order interactions are rarely important, such intensive efforts are seldom required. For most purposes, it is only necessary to evaluate the main effects of each variable and the two-factor interactions, which can be done with only a fraction of the runs in a full factorial

design. If there are some two-factor interactions that are known to be impossible, one can further reduce the number of runs. For the earlier example of eight factors, one can create an efficient design that may require as little as 16 runs. However, the number of experimental runs required for three-level factorial designs can be much greater than for their two-level counterparts. Factorial designs are therefore rather unattractive if a researcher wishes to consider multiple levels. It may not be necessary to consider more than two levels, however, if the factors are continuous in nature.When the factors are continuous, two-level factorial designs assume that the effects are linear. If a quadratic effect is expected for a factor, a more complicated experiment should be used, such as the central composite design. Optimization of factors that could have quadratic effects is the primary goal of response surface methodology.

2.5. Neural NetworksAn artificial neural network (ANN), often just called a "neural network" (NN), is a mathematical model or computational model based on biological neural networks. It consists of an interconnected group of artificial neurons and processes information using a connectionist approach to computation. In most cases an ANN is an adaptive system that changes its structure based on external or internal information that flows through the network during the learning phase.In more practical terms neural networks are non-linear statistical data modeling tools. They can be used to model complex relationships between inputs and outputs or to find patterns in data.

Neural networks are composed of simple elements operating in parallel. These elements are inspired by biological nervous systems. As in nature, the network function is determined largely by the connections between elements. We can train a neural network to perform a particular function by adjusting the values of the connections (weights) between elements.Commonly neural networks are adjusted, or trained, so that a particular input leads to a specific target output. Such a situation is shown below. There, the network is adjusted, based on a comparison of the output and the target, until the network output matches the target. Typically many such input/target pairs are used, in this supervised learning, to train a network.Batch training of a network proceeds by making weight and bias changes based on an entire set (batch) of input vectors. Incremental training changes the weights and biases of a network as needed after presentation of each individual input vector. Incremental training is sometimes referred to as “on line” or “adaptive” training.Neural networks have been trained to perform complex functions in various fields of application including pattern recognition, identification, classification, speech, vision and control systems. Today neural networks can be trained to solve problems that are difficult for conventional computers or human beings. Throughout the MATLAB toolbox emphasis is placed on neural network paradigms that build up to or are themselves used in engineering, and other practical applications.

The supervised training methods are commonly used, but other networks can be obtained from unsupervised training techniques or from direct design methods. Unsupervised networks can be used, for instance, to identify groups of data. Certain kinds of linear networks and The field of neural networks has a history of some five decades but has found solid application only in the past fifteen years, and the field is still developing rapidly. Thus, it is distinctly different from the fields of control systems or optimization where the terminology, basic mathematics, and design procedures have been firmly established and applied for many years. Neural Network Toolbox of MATLAB will be a useful tool for industry, education and research, a tool that will help users find what works and what doesn’t, and a tool that will help develop and extend the field of neural networks. 2.5.1. Neural Network ApplicationsIndustrial•Neural networks are being trained to predict the output of industrial processes. They then replace complex and costly equipment used for this purpose in the past.Manufacturing•Manufacturing process control, product design and analysis, process and machine diagnosis, real-time particle identification, visual quality inspection systems, beer testing, welding quality analysis, paper quality prediction, computer-chip quality analysis, analysis of grinding operations, chemical product design analysis, machine maintenance analysis, project bidding, planning and management, dynamic modelling of chemical process systemOil and Gas•ExplorationRobotics•Trajectory control, forklift robot, manipulator controllers, vision systems•Truck brake diagnosis systems, vehicle scheduling, routing systems

2.5.2. Neuron ModelSimple NeuronA neuron with a single scalar input and no bias appears on the left below.The scalar input p is transmitted through a connection that multiplies its strength by the scalar weight w, to form the product wp, again a scalar. Here the weighted input wp is the only argument of the transfer function f, which produces the scalar output a. The neuron on the right has a scalar bias, b. You may view the bias as simply being added to the product wp as shown by the summing junction or as shifting the function f to the left by an amount b. The bias is much like a weight, except that it has a constant input of 1. The transfer function net input n, again a scalar, is the sum of the weighted input wp and the bias b. This sum is the argument of the transfer function f.Here f is a transfer function, typically a step function or a sigmoid function, which takes the argument n and produces the output a. Examples of

various transfer functions are given in the next section. Note that w and b are both adjustable scalar parameters of the neuron. The central idea of neural networks is that such parameters can be adjusted so that the network exhibits some desired or interesting behavior. Thus, we can train the network to do a particular job by adjusting the weight or bias parameters, or perhaps the network itself will adjust these parameters to achieve some desired end.

Transfer Functions Many transfer functions are used to build a model. Three of the most commonly used functions are shown below.The hard-limit transfer function shown above limits the output of the neuron to either 0, if the net input argument n is less than 0; or 1, if n is greater than or equal to 0.

The linear transfer function is shown below.

A A

Neurons of this type are used as linear approximators in “Linear Filters”.

The sigmoid transfer function shown below takes the input, which may haveany value between plus and minus infinity, and squashes the output into therange 0 to 1. This transfer function is commonly used in backpropagation networks, in partbecause it is differentiable.

Neuron with Vector InputA neuron with a single R-element input vector is shown below. Here the individual element inputs are multiplied by weights and the weighted values are fed to the summing junction. Their sum is simply Wp, the dot product of the (single row) matrix W and the vector p.p1, p2,... pRw1 1 , w1 2 , ... w1 R

The neuron has a bias b, which is summed with the weighted inputs to form the net input n. This sum, n, is the argument of the transfer function f.

The figure of a single neuron shown above contains a lot of detail. When we consider networks with many neurons and perhaps layers of many neurons, there is so much detail that the main thoughts tend to be lost. Thus, the authors have devised an abbreviated notation for an individual neuron. This notation, which will be used later in circuits of multiple neurons, is illustrated in the diagram shown below.

Here the input vector p is represented by the solid dark vertical bar at the left. The dimensions of p are shown below the symbol p in the figure as Rx1. (Note that we will use a capital letter, such as R in the previous sentence, when referring to the size of a vector.) Thus, p is a vector of R input elements. These inputs post multiply the single row, R column matrix W. As before, a constant 1 enters the neuron as an input and is multiplied by a scalar bias b. The net input to the transfer function f is n, the sum of the bias b and the product Wp.This sum is passed to the transfer function f to get the neuron’s output a, which in this case is a scalar. Note that if we had more than one neuron, the network output would be a vector.A layer of a network is defined in the figure shown above. A layer includes the combination of the weights, the multiplication and summing operation (here realized as a vector product Wp), the bias b, and the transfer function f. The array of inputs, vector p, is not included in or called a layer.As discussed previously, when a specific transfer function is to be used in a figure, the symbol for that transfer function will replace the f shown above. Here are some examples.

Network ArchitecturesTwo or more of the neurons shown earlier can be combined in a layer, and a particular network could contain one or more such layers. First consider a single layer of neurons.A Layer of NeuronsA one-layer network with R input elements and S neurons follows.

In this network, each element of the input vector p is connected to each neuron input through the weight matrix W. The ith neuron has a summer that gathers its weighted inputs and bias to form its own scalar output n(i). The various n(i) taken together form an S-element net input vector n. Finally, the neuron layer outputs form a column vector a. We show the expression for a at the bottom of the figure.

Multiple Layers of NeuronsA network can have several layers. Each layer has a weight matrix W, a bias vector b, and an output vector a. To distinguish between the weight matrices, output vectors, etc., for each of these layers in our figures, we append the number of the layer as a superscript to the variable of interest. You can see the use of this layer notation in the three-layer network shown below, and in the equations at the bottom of the figure.

Kodutöö3 Närvivõrkude mudeli loomine.Näide. Example of use the Neural Network Toolbox of MATLAB

To the problem of selection different technology options the ANN model together with corresponding training were used.

Using the selection parameters down (Table 2.12.1), the ANN trained for each technology (like vacuum forming processes, acrylic trimming technologies and reinforcement) was used. For illustrating the data of observations are presented the Table 2.12.1. . There are three options {0 - Not usable, 1- Reverse draw forming with two heaters, 2- Straight vacuum forming} for finding out vacuum forming technology.

Table 2.12.1. Selection parameters for vacuum forming processesParameter and mark Description

Dimensions (L and B): L x B; 280x430,680x760 mm up to 2000x1000 mmMax depth of draw (H): H; 183, 220, 300 mm up to 800 mmMax material thickness (D): D; 3.2 mm, 4 mm, 6 mm, 7 mmUndercuts (UC): yes/no … …Draft angle (α): α; α >5 ˚Surface quality (Q): low, medium, highBatch size (N): N; 1 <= N <= 10000 (0<= log N<= 4) … … Wall thickness after forming (h): h; 0.7 < h < 3 mmHeating temperature (Th): Th; 180˚C <= Th <= 220˚CCooling time (E): C; 3 < C < 7 minHeating zones (Z): Z; 1 < Z < 4

Cooling points (P): P; 2 <= P <= 5

Table 2.12.2. Vacuum forming training modeSample Vacuum

forming Geom Log (nP) Dim Thick SQ PT UC I

1 1 1 2 1 0 2 1 2 22 2 2 2 2 1 2 2 2 1… … … … … … … … … …20 2 1 2 2 1 2 1 2 1

Where: Geom is the geometric complexity; Log(nP) is the number of parts; Dim is the dimension of vacuum forming bench table; Thick is maximal material thickness; SQ is surface quality; PT is part texture; UC is undercuts; I is investments.

The acceptability of model was estimated by the accuracy of model on training data (it was accepted that training goal, the total error estimate must be less than 0.000001). In that task a mathematical model for a neuron could be represented as:

Where: vi is the output activation of the unit l; W is a numeric weight; e is the bias vector; g is the activation function of the unit (there was used two log-sigmoid functions as activation

functions).

The proposed models have the following parameters:

99.26

6.55-6.48

12.025.50-4.90-

26.37-

8.325.23-....0.4009.011.30-10.69....2.7260.01.330.56-....3.6984.60.82-4.55....6.9339.2

11.81-4.28....6.1715.52-2.02-1.86-....5.04-2.26-

18.160.3357.3899.2303.1447.3

12 lggv i

In Figure 2.12.1 is brought out the training curve, which got using the system MatLAB. The training curve shows the reduction in error over several epoch of training for selection of reverse draw vacuum forming or straight forming.

),)(( 211,111,22 eelwgwgvi

Figure 2.12.1. Training curve showing the reduction in error over several epoch of training for selection of reverse draw vacuum forming or straight forming

Programmi näide (vaja uus näide):% Reinforcement: yes, no% Lähteandmed p=[ 3 3 1 2 3 2 1 3 3 2 2 1 2 1 1 2 3 2 1 2; … 2 1 1 0 2 2 2 0 2 0 1 1 1 0 2 2 1 2 2 0;... 2 3 2 2 1 2 3 3 1 1 1 0 0 2 3 2 3 1 1 2; … 2 0 1 2 2 0 1 0 1 1 2 2 2 0 0 2 0 2 1 0;... 2 3 2 1 1 2 3 3 1 2 2 3 2 1 2 3 2 1 2 3; … 2 2 2 1 1 1 1 2 2 1 1 2 2 2 1 2 1 2 1 1 ] t=[1 0 0 0 0 1 0 0 0 0 0 0 0 0 1 1 0 1 0 0 ]% Närvivõrgu mudeli kirjeldamine net =newff(minmax(p),[4,1],{'logsig','logsig'},'trainlm'); % Muudame treeningu parameetreid net.trainParam.show=100; net.trainParam.lr=0.05; net.trainParam.epochs=3000; net.trainParam.goal=0.00001; [net,tr]=train(net,p,t);% Kontrollime lahendit a=round(sim(net,p))% Väljastame parameetrid bias1=net.b{1} bias2=net.b{2} weights1=net.IW{1,1} weights2=net.LW{2,1}

p = Columns 1 through 12 3 3 1 2 3 2 1 3 3 2 2 1 2 1 1 0 2 2 2 0 2 0 1 1 2 3 2 2 1 2 3 3 1 1 1 0 2 0 1 2 2 0 1 0 1 1 2 2 2 3 2 1 1 2 3 3 1 2 2 3 2 2 2 1 1 1 1 2 2 1 1 2

Columns 13 through 20 2 1 1 2 3 2 1 2 1 0 2 2 1 2 2 0 0 2 3 2 3 1 1 2 2 0 0 2 0 2 1 0 2 1 2 3 2 1 2 3 2 2 1 2 1 2 1 1

t = Columns 1 through 12 1 0 0 0 0 1 0 0 0 0 0 0 Columns 13 through 20 0 0 1 1 0 1 0 0

TRAINLM-calcjx, Epoch 0/3000, MSE 0.21925/1e-005, Gradient 0.0881321/1e-010TRAINLM-calcjx, Epoch 12/3000, MSE 2.20172e-006/1e-005, Gradient 3.15234e-006/1e-010TRAINLM, Performance goal met.

a = Columns 1 through 12 1 0 0 0 0 1 0 0 0 0 0 0 Columns 13 through 20 0 0 1 1 0 1 0 0

bias1 = -15.4059 1.9312 -5.2136 -9.0533bias2 = -4.4747weights1 = -2.3906 0.4411 1.9699 7.0688 -2.0334 5.1471 3.8734 9.7168 2.5332 -5.6808 -3.6823 -4.1839 3.6735 -4.5363 -5.3135 4.8883 -3.1428 3.4034 1.3977 -4.8421 0.4030 8.2915 4.4874 2.7907weights2 = 14.3645 10.4440 -1.4794 -12.7180

Küsimused seoses kodutööga 31. Esitada ja põhjendada valitud mudelit2. Hinnata valitud mudeli sobivust katseandmete kirjeldamiseks.