fall 2005 lecture notes #6 eecs 595 / ling 541 / si 661 natural language processing

of 36 /36
Fall 2005 Lecture Notes #6 EECS 595 / LING 541 / SI 661 Natural Language Processing

Author: marilyn-barrett

Post on 31-Dec-2015




0 download

Embed Size (px)


  • Fall 2005Lecture Notes #6EECS 595 / LING 541 / SI 661Natural Language Processing

  • Lexicalized andprobabilistic parsing

  • Probabilistic CFGG (N, , P, S)Non-terminals (N)Terminals ()Productions (P) augmented with probabilities: A [p]

  • Disambiguation as a probability problemP(T,S) = P(T) P(S|T) = P(T)P(Tl) = .15 * .40 * .05 * .05 * .35 * .75 * .40 * .40 * .40 * .30 * .40 * .50 = 1.5 x 10-6P(Tr) = .15 * .40 * .40 * .05 * .05 * .75 * .40 * .40 * .40 * .30 * .40 * .50 = 1.7 x 10-6

  • Probabilistic parsingProbabilistic Earley algorithmTop-down parser with a dynamic programming tableCocke-Younger-Kasami (CYK) algorithmBottom-up parser with a dynamic programming tableProbabilities come from a Treebank.

  • Probabilistic CYK

  • Dependency grammarsLexical dependencies between head wordsTop-level predicate of a sentence is the rootUseful for free word order languagesAlso simpler to parse

  • DependenciesJohn likes tabby catsNNPVBSJJNNSNPVPNPS

  • Representing Meaning

  • IntroductionMeaning representation languages: capturing the meaning of linguistic utterances using formal notationExample: deciding what to order at a restaurant by reading a menuExample: answering a question on an examSemantic analysis: mapping between language and real lifeI have a car: x,y: Having(x) ^ Haver(speaker,x) ^ HadThing(y,x) ^ Car(y)

  • VerifiabilityExample: Does LeDog serve vegetarian food?Knowledge base (KB)Sample entry in KB: Serves(LeDog,Vegetarian Food)Convert question to logical form and verify its truth value against the knowledge base

  • UnambiguousnessExample: I want to eat some place near UM. (multiple interpretations)Interpretation is importantPreferred interpretationsVagueness: I want to eat Italian food. - what particular food?

  • Canonical formDoes LeDog have vegetarian dishes?Do they have vegetarian food at LeDog?Are vegetarian dishes served at LeDog?Does LeDog serve vegetarian fare?Having vs. servingFood vs. fare vs. dishes (each is ambiguous but one sense of each matches the others)word sense disambiguation

  • Inference and variables; expressivenessInference and variables:Id like to find a restaurant that serves vegetarian food.Serves (x,VegetarianFood)Systems ability to draw valid conclusions based on the meaning representations of inputs and its store of background knowledge.Expressiveness:system must be able to handle a wide range of subject matter

  • Predicate-argument structureI want Italian food. NP want NPI want to spend less than five dollars. NP want Inf-VPI want it to be close by here. NP want NP Inf-VPThematic roles: e.g. entity doing the wanting vs. entity that is wanted (linking surface arguments with the semantic=case roles)Syntactic selection restrictions: I found to fly to Dallas.Semantic selection restrictions: The risotto wanted to spend less than ten dollars.Make a reservation for this evening for a table for two persons at eight: Reservation (Hearer,Today,8PM,2)

  • First-order predicate calculus (FOPC)Formula AtomicFormula | Formula Connective Formula | Quantifier Variable Formula | Formula | (Formula)AtomicFormula Predicate (Term)Term Function (Term) | Constant | VariableConnective | | Quantifier | Constant A | VegetarianFood | LeDogVariable x | y | Predicate Serves | Near | Function LocationOf | CuisineOf |

  • ExampleI only have five dollars and I dont have a lot of time.Have(Speaker,FiveDollars) Have(Speaker,LotOfTime)variables:Have(x,FiveDollars) Have(x,LotOfTime)Note: grammar is recursive

  • Semantics of FOPCFOPC sentences can be assigned a value of true or false.LeDog is near UM.Near(LocationOf(LeDog),LocationOf(UM))

  • Variables and quantifiersA restaurant that serves Mexican food near UM. x: Restaurant(x) Serves(x,MexicanFood) Near(LocationOf(x),LocationOf(UM))All vegetarian restaurants serve vegetarian food. x: VegetarianRestaurant(x) Serves (x,VegetarianFood)If this sentence is true, it is also true for any substitution of x. However, if the condition is false, the sentence is always true.

  • InferenceModus ponens:

    Example: VegetarianRestaurant(Joes) x: VegetarianRestaurant(x) Serves(x,VegetarianFood) Serves(Joes,VegetarianFood)

  • Uses of modus ponensForward chaining: as individual facts are added to the database, all derived inferences are generatedBackward chaining: starts from queries. Example: the Prolog programming languagefather(X, Y) :- parent(X, Y), male(X). parent(john, bill). parent(jane, bill). female(jane). male (john). ?- father(M, bill).

  • Examples from Russell&Norvig (1)7.2. p.213

    Not all students take both History and Biology.Only one student failed History.Only one student failed both History and Biology.The best history in History was better than the best score in Biology.Every person who dislikes all vegetarians is smart.No person likes a smart vegetarian.There is a woman who likes all men who are vegetarian.There is a barber who shaves all men in town who don't shave themselves.No person likes a professor unless a professor is smart.Politicians can fool some people all of the time or all people some of the time but they cannot fool all people all of the time.

  • Categories & EventsCategories:VegetarianRestaurant (Joes) categories are relations and not objectsMostPopular(Joes,VegetarianRestaurant) not FOPC!ISA (Joes,VegetarianRestaurant) reification (turn all concepts into objects)AKO (VegetarianRestaurant,Restaurant)Events:Reservation (Hearer,Joes,Today,8PM,2)Problems:Determining the correct number of rolesRepresenting facts about the roles associated with an eventEnsuring that all the correct inferences can be drawnEnsuring that no incorrect inferences can be drawn

  • MUC-4 Example

  • Subcategorization framesI ateI ate a turkey sandwichI ate a turkey sandwich at my deskI ate at my deskI ate lunchI ate a turkey sandwich for lunchI ate a turkey sandwich for lunch at my desk- no fixed arity (problem for FOPC)

  • One possible solutionEating1 (Speaker)Eating2 (Speaker, TurkeySandwich)Eating3 (Speaker, TurkeySandwich, Desk)Eating4 (Speaker, Desk)Eating5 (Speaker, Lunch)Eating6 (Speaker, TurkeySandwich, Lunch)Eating7 (Speaker, TurkeySandwich, Lunch, Desk)Meaning postulates are used to tie semantics of predicates: w,x,y,z: Eating7(w,x,y,z) Eating6(w,x,y)Scalability issues again!

  • Another solution- Say that everything is a special case of Eating7 with some arguments unspecified: w,x,y Eating (Speaker,w,x,y)- Two problems again:Too many commitments (e.g., no eating except at meals: lunch, dinner, etc.)No way to individuate events: w,x Eating (Speaker,w,x,Desk) w,y Eating (Speaker,w,Lunch,y) cannot combine into w Eating (Speaker,w,Lunch,Desk)

  • Reification w: Isa(w,Eating) Eater(w,Speaker) Eaten(w,TurkeySandwich) equivalent to sentence 5.Reification:No need to specify fixed number of arguments for a given surface predicateNo more roles are postulated than mentioned in the inputNo need for meaning postulates to specify logical connections among closely related examples

  • Representing timeI arrived in New YorkI am arriving in New YorkI will arrive in New York

    w: Isa(w,Arriving) Arriver(w,Speaker) Destination(w,NewYork)

  • Representing time i,e,w,t: Isa(w,Arriving) Arriver(w,Speaker) Destination(w,NewYork) IntervalOf(w,i) EndPoint(I,e) Precedes (e,Now) i,e,w,t: Isa(w,Arriving) Arriver(w,Speaker) Destination(w,NewYork) IntervalOf(w,i) MemberOf(i,Now) i,e,w,t: Isa(w,Arriving) Arriver(w,Speaker) Destination(w,NewYork) IntervalOf(w,i) StartPoint(i,s) Precedes (Now,s)

  • Representing timeWe fly from San Francisco to Boston at 10.Flight 1390 will be at the gate an hour now.Use of tensesFlight 1902 arrived late.Flight 1902 had arrived late.similar tensesWhen Marys flight departed, I ate lunchWhen Marys flight departed, I had eaten lunchreference point

  • AspectStative: I know my departure gateActivity: John is flying no particular end pointAccomplishment: Sally booked her flight natural end point and result in a particular stateAchievement: She found her gateFiguring out statives: * I am needing the cheapest fare. * I am wanting to go today. * Need the cheapest fare!

  • Representing beliefsWant, believe, imagine, know - all introduce hypothetical worldsI believe that Mary ate British food.Reified example: u,v: Isa(u,Believing) Isa(v,Eating) Believer (u,Speaker) BelievedProp(u,v) Eater(v,Mary) Eaten(v,BritishFood)However this implies also: u,v: Isa(v,Eating) Eater(v,Mary) Eaten(v,BritishFood)Modal operators:Believing(Speaker,Eating(Mary,BritishFood)) - not FOPC! predicates in FOPC hold between objects, not between relations.Believes(Speaker, v: ISA(v,Eating) Eater(v,Mary) Eaten(v,BritishFood))

  • Modal operatorsBeliefsKnowledgeAssertionsIssues: If you are interested in baseball, the Red Sox are playing tonight.

  • Examples from Russell&Norvig (2)7.3. p.214

    One more outburst like that and you'll be in comptempt of court.Annie Hall is on TV tonight if you are interested.Either the Red Sox win or I am out ten dollars.The special this morning is ham and eggs.Maybe I will come to the party and maybe I won't.Well, I like Sandy and I don't like Sandy.