a fine-grained analysis of user-generated content to support decision making

AFine‐GrainedAnalysisofUser‐GeneratedContenttoSupportDecisionMaking

MarcirioSilveiraChavesh/p://mchaves.wikidot.com

Informa<onSystemsResearchGroup

BusinessandInforma<onTechnologyResearchCentre(BITREC)Ins<tuteforScien<ficandTechnologicalResearchofUniversidadeAtlân<ca(ISTR)

Workshop

User‐GeneratedContent(UGC)•  Asknownas

–  User‐GeneratedData–  User‐CreatedContent–  User‐ContributedData–  Consumer‐GeneratedMedia

–  …

•  Canbeexpressedthrought–  Opinions–  Reviews–  Comments–  Posts

Apr‐18‐12 MarcirioChaves‐marcirioc@uatlan<ca.pt 2

• Notes:• Alltheexamplesdescribedinthisworkshoparerealdata.• Somepapersmen<onedhereareunderreview.• Colorlegend:

• Examples• Posi<vefeature• Nega<vefeature

ExampleofUGC•  AnopinionpostedinFacebookDec‐10‐2011,12:30pm

– “wouldhighlyrecommendInfinityMotorcycles,Southamptonforallmotorbikinggear.Veryreasonablepeople.Earliertheygavemeafullmoneybackforaunused(a\erexplainingwhyitwasunused)ladiesmotorbikejacket(nodefectswhasoever)andtodaythezipperonmynewjacketwasbrokenandtheygavemeabrandnewone(noques<onsasked,noreceiptbusinessandnofusscreated).FiveStarservice.”

– Thisuserhad226friends.

Apr‐18‐12 3MarcirioChaves‐marcirioc@uatlan<ca.pt

Somesta<s<csaboutUGC•  Morethan50%ofallinternetvisitsarenowtoUGC/socialmediasites.

•  Morethan75%of<mespentontheinternetis"social”.

•  Facebooknowcapturesasmuch<mespentontheinternetasGoogle,Yahoo,andAOL.

•  Morethan80%ofconsumersareinfluencedbySocialMarkeJng.

Source: http://www.bbrisco.com/2010/05/social.html


MainObjec<vesofthisWorkshop•  In‐depthanalysisofUGC

•  UseUGCtosupportdecisionmaking

•  StudyadomainontologytosupportAr<ficialIntelligencetasks

•  Addressapproachesforsen<mentanalysis

•  Fromtheorytoprac<ce:Hands‐onSessionApr‐18‐12 MarcirioChaves‐marcirioc@uatlan<ca.pt 5

OutlinePart1

•  WorkshopContext

•  User‐GeneratedContent(UGC)

•  Characterisa<onofUGC•  KnowledgeEngineering‐

OntologyDevelopment

•  Hands‐onSession(IndividualTask):DealingwithUGC

Part2

•  Sen<mentAnalysis/OpinionMining

•  PolarityRecognizerinPortuguese(PIRPO)

•  Informa<onVisualisa<on


ContextWorkshop

AframeworkforCustomerKnowledgeManagementbasedonSocialSeman<cWeb.

Chaves,MarcirioSilveira;Trojahn,CássiaandPedron,Cris<aneDrebes.AFrameworkforCustomerKnowledgeManagementbasedonSocialSeman<cWeb:AHotelSectorApproach.In:CustomerRela<onshipManagementandtheSocialandSeman<cWeb:EnablingCliensConexus.Colomo‐Palacios,R.;Varajão,J.andSoto‐Acosta,P.(Eds.).p.141‐157,Hershey,PA:IGIGlobal,2012.ISBN:978‐161‐35‐0044‐6


AnFine‐grainedAnalysisofUGC•  OverallopinionaboutatopicisonlyapartoftheinformaJonofinterest.

•  Document‐levelsenJmentclassificaJonfailstodetectsen<mentaboutindividualaspectsofthetopic.Inreality,forexample,thoughonecouldbegenerallyhappyabouthiscar,hemightbedissaJsfiedbytheenginenoise.

•  Tothemanufacturers,theseindividualweaknessesandstrengthsareequallyimportanttoknow,orevenmorevaluablethantheoverallsa<sfac<onlevelofcustomers.(Tangetal.2009)


UGC

AnopinionissimplyaposiJveornegaJvesenJment,view,aPtude,emoJon,or

appraisalaboutanenJtyoranaspectoftheenJty(HuandLiu,2004;Liu,2006)fromanopinionholder(Bethardetal.,2004;Kimand

Hovy,2004;Wiebeetal.,2005).


Characterisa<onofUGC•  Opinion’sCharacterisa<on–  Iuseandextendthedefini<onproposedby(Dingetal.,2008;Liu,2010;Mar<nandWhite,2005)toanalysethesentencesofreviews.

– Letthereviewber.

–  Inthemostgeneralcase,rischaracterisedasasetofthefollowingelements{O,F,SO,H,S,A,R,I,SG},where:


Characterisa<onofUGC•  Opinion’sCharacterisa<on– O:Object– F:Feature– SO:Seman<c‐Orienta<on– H:Holder– S:Source– A:A%tude– SG:Sugges.on– R:Recommenda.on–  I:Inten.on


Characterisa<onofUGC1  ‐Object(O)– Anobjectisaproduct(e.g.movieandbook)oraservice(e.g.hotelandrestaurant)underreviewwhichiscomposedbyfeatures.– ObjectsarealsocalledenJJes.

2‐Feature(F)– Afeatureisacomponentorpartofanobject.•  actorandphotographyarefeaturesonamovie.•  poolandstaffarefeaturesonahotel.

– FeaturesarealsocalledaXributesorfacets.– Afeaturecanbemen<onedexplicitlyorimplicitlyinareview(Dingetal.2008).


Characterisa<onofUGC2.1‐ExplicitFeature(F)–  Ifafeaturefappearsinreviewr,itiscalledanexplicitfeatureinr.

– Thehotelislocatedverynearthecentercity.•  loca<onisanexplicitfeature.

2.2‐ImplicitFeature(F):–  Iffdoesnotappearinrbutisimplied,itiscalledanimplicitfeatureinr.

– Hotelisfarfrompublictransporta<on.•  loca<onisanimplicitfeature.


Characterisa<onofUGC3‐Sentence‐OrientaJon(SO)– Areviewconsistsofasequenceofsentencesr=⟨s1,s2,…,sm⟩(Dingetal.,2008).

– Asentencecanbeevaluatedasthefollowingperspec<ves:


Characterisa<onofUGC3.1ObjecJvity– Anobjec<vesentencecontainsormenJonfacts.•  Thishotelisfarfromtheairport,ca.15km.

– Asubjec<vesentencedoesnotmenJonanyfact.•  Theparkingcouldbefree.

3.2Polarity–  ItdescribestheorientaJonpresentinasentence(i.e.posiJve,negaJve,neutralandirrelevant).


Characterisa<onofUGC3.3Intensity(strengthofthepolarity)–  Itreferstothestrengthoftheprivatestatethatisbeingexpressed,inotherwords,howstrongisanemo<onoraconvic<onofbelief(Wilson,2008).

–  Itdescribeshowintenseitwastheexperienceusingaproductorservice:•  veryposiJve,posiJve,neutral,negaJveandverynegaJve.

•  Verykindlystaff.referstoaveryposi<veimpressiononthestaffservice.


Characterisa<onofUGC4‐OpinionHolder(H)–  Theholderofapar<cularopinionisthepersonortheorganisaJonthatholdstheopinion(Dingetal.,2008).

– Aholderisiden<fiedwithdemographiccharacterisJcs(e.g.name,cityandcountry).

–  Sitessuchastripadvisor.comandbooking.comclassifyholdersastypesincluding:•  familieswitholderchildren

•  familieswithyoungchildren•  maturecouples

•  groupsoffriends•  solotravellers•  youngcouples


Characterisa<onofUGC5–Source– Aninforma<onsourceisawebsitewhichprovidesasetofreviews.•  tripadvisor.com

•  booking.com•  amazon.com

•  A:A%tude

•  SG:Sugges.on•  R:Recommenda.on

•  I:Inten.onApr‐18‐12 18MarcirioChaves‐marcirioc@uatlan<ca.pt

OutlinePart1




OntologyDevelopment


Part2





Limita<onsforrepresen<ngknowledgeintheaccommoda<onsector


language?

Morelimita<ons•  Actually,webagentsareunabletoanswerques<onssuchas:– WhatarethehotelswithlongerindoorswimmingpoolJmetableinRoma?

– WhatarethehotelswiththecheapestbreakfastinLisbon?

– WhatarethecheapesthotelswithfamilysuiteroomwithseaviewinBarcelona?


KnowledgeEngineering•  OntologyasasupporttoevaluateUGC– Setofconceptstoaspecificdomain

– Humanandmachinereadable– Supporttofine‐grainedanalysisoftheinstances(e.g.reviews)

– Hontology(Hstandsforhotel,hostalandhostel)•  Arobust,coherentandmul<lingualrepresenta<onoftheaccommoda<onsector.


ContextWorkshop




KnowledgeEngineering•  DevelopmentMethodology

–  Iden<fyexis<ngontologiesonrelateddomains–  Selectthemainconceptsandproper<es–  Organizeconceptsandproper<eshierarchicallyintocategories–  Translatetheontology(manual)–  Expandconceptsandproper<esbasedoncomments–  Translatethenewconceptsandproper<es(manual)–  Generatetheontologyinseveralformats


Chaves,M.S.andTrojahn,C.TowardsaMulJlingualOntologyforOntology‐drivenContentMininginSocialWebSites.Proc.oftheISWC2010Workshops,VolumeI,1stInternaJonalWorkshoponCross‐CulturalandCross‐LingualAspectsoftheSemanJcWeb.Shanghai,China,November7th,2010.

KnowledgeEngineering•  Hontology– AmulJlingualontologyfortheaccommodaJonsector.

•  DemoProtégé

Chaves,M.S.;Freitas,L.A.andVieira,R.(2012).Hontology:AmulJlingualontologyfortheaccommodaJonsector.4thInternaJonalConferenceonKnowledgeEngineeringandOntologyDevelopment,Barcelona,Spain,4‐7October.(SubmiXed)


KnowledgeEngineering

PreliminaryHontologySta<s<cs


Metrics ValueNumberofConcepts 285NumberofObjectProper<es 10NumberofDataProper<es 31

ConceptAxiomsSubconceptaxioms 270Equivalentconceptsaxioms 4Disjointconceptsaxioms 93

ObjectPropertyAxiomsFunc<onalobjectpropertyaxioms 6Objectpropertydomainaxioms 11Objectpropertyrangeaxioms 8

DataPropertyAxiomsFunc<onaldatapropertyaxioms 12Objectdatadomainaxioms 17Objectdatarangeaxioms 1

Hands‐onSession•  Theaimofthishands‐onsessionistoallowyouthinking

in‐depthaboutUGConthecontextoftheaccommoda<onsector.

•  Youaregoingtoreceiveasetof4or5reviewsaboutaccommoda<onsandshouldevaluateeachoneaccordingtothefollowingparameters:–  Featurespresentinthereview(seetheconceptsofHontology)

–  Intensity(StrengthofthePolarity):(verynega<ve,nega<ve,neutral,posi<ve,veryposi<ve)

•  Notes:–  Evaluateonefeatureperline.–  Please,[email protected]:UB:GX

–  X=numberofthegroup.


OutlinePart1




OntologyDevelopment


Part2





Sen<mentAnalysis•  AnalysisandautomaJcextracJonofSemanJcOrientaJon

•  SemanJcorientaJonreferstothepolarityandstrengthofwords,phrases,ortexts.

•  Approaches–  Lexicon‐based

•  Dic<onariesofwordsannotatedwiththeword´sseman<corienta<on,orpolarity.

•  AmanuallybuiltdicJonaryprovidesasolidfoundaJonforalexicon‐basedapproach(Taboadaet.al.,2011).

–  StaJsJcalorMachine‐learning•  Supervisedclassifica<on


Sen<mentAnalysis•  Lexicon‐basedApproach– Sen<ment‐bearingwords:alistofnouns,verbs,adjecJvesandadverbs(Chesleyetal.,2006)• useverbsandadjec<vestoclassifyEnglishopinionatedblogtexts.

– ListofconjuncJonsandconnecJves(Liu,2010).– Useofauxiliaryverbstogetfeaturesandopinion‐orientedwordsaboutproductsfromtexts(Khanetal.,2010).


Sen<mentAnalysis•  Seedwords– areasmallsetofwordswithstrongnegaJveorposiJveassocia<ons,suchasexcellentorabysmal.

–  Inprinciple,aposi<veadjec<veshouldoccurmorefrequentlyalongsidetheposi<veseedwords,andthuswillobtainaposi<vescore,whereasnega<veadjec<veswilloccurmosto\eninthevicinityofnega<veseedwords,thusobtaininganega<vescore(Taboadaet.al.2011).•  Thisrestauranthasabadandexpensivefood.


Sen<mentAnalysis•  Part‐of‐Speech(PoS)–  Inordertoevaluateasentenceinareview,weshouldconsidertheparts‐of‐speechmen<onedsuchasadjecJves,adverbsandverbs.

– Adjec<vesareclassifiedas:•  posi<ve(good,excellentandclean),•  nega<ve(awful,boringandterrible),•  neutral(regularandindifferent)and•  dual,whichcanexpressposi<veandnega<veopinion(small,long).

–  Insomeapproachesnounsarerepresentedbyconceptsofadomainontologyandmappedasfeatures.


Sen<mentAnalysis•  ConjuncJonandConnecJve(CC)– Connec<vesarewordsthathelpiden<fyingaddiJonaladjecJveopinionwordsandtheirorientaJons.

– Oneoftheconstraintsisaboutconjunc<on(i.e.and),whichsaysthatconjoinedadjec<vesusuallyhavethesameorienta<on(Liu,2010).•  Thisroomisbeau<fulandspacious.

–  ifbeau<fulisknowntobeposi<ve,itcanbeinferredthatspaciousisalsoposi<ve.

– HeurisJc:•  PeopleusuallyexpressthesameopiniononbothsidesofaconjuncJon.


Sen<mentAnalysis•  ConjuncJonandConnecJve(CC)– Rulesorconstraintsarealsodesignedforotherconnec<ves(e.g.or,but,either‐or,andneither‐nor).•  Thishotelisbeau<fulbutdifficulttogetthere.

–  Theoccurrencea\ertheconnec<vebutisanindicatorofanega<veopinion.


Sen<mentAnalysis•  StrengthofthePolaJryorIntensityorIntensificaJon– Amplifiers(very,alot)increasetheseman<cintensityofaneighboringlexicalitem;

– AXenuators/Downtoners(ali/le,slightly)decreaseit.

•  SomeapproacheshaveimplementedintensifiersusingsimpleaddiJonandsubtracJon–  ifaposi<veadjec<vehasanSOvalueof2:•  anamplifiedadjec<vewouldhaveanSOvalueof3,and•  adowntonedadjec<veanSOvalueof1.


Sen<mentAnalysis•  NegaJon– Theobviousapproachtonega<onissimplytoreversethepolarityofthelexicalitemnexttoanegator,changinggood(+3)intonotgood(−3).

– Not,none,nobody,never,andnothing,andotherwords,suchaswithoutorlack.


PolarityRecognizerinPortuguese(PIRPO)•  PolarityRecognizerinPortuguesetoclassifysenJmentin

onlinereviews.

•  PIRPOwasbuiltfromthegroundtoPortugueseforrecognisingthepolarityoftheuseropiniononaccommoda<onreviews.

•  Eachreviewisanalysedaccordingtoconceptsfromadomainontology.

•  Wedecomposethereviewinsentencesinordertoassignapolaritytoeachconceptoftheontologyinthesentence.

Chaves,M.S.,Freitas,L.,Souza,M.andVieira,R.PIRPO:AnAlgorithmtodealwithPolarityinPortugueseOnlineReviewsfromtheAccommodaJonSector.17thInternaJonalconferenceonApplicaJonsofNaturalLanguageProcessingtoInformaJonSystems(NLDB),Groningen,TheNetherlands,26‐28June2012.


PIRPOInforma<onArchitecture


PIRPO•  Reviews– Fulldataset:1500reviewsfromJanuary2010toApril2011inPortuguese,EnglishandSpanish,fromwhich180inPortuguese.

•  OntologyConcepts– TheconceptsusedtoclassifythereviewsareprovidedbyHontology,whichinitscurrentversion,has110concepts.


PIRPO•  ListofadjecJves:Itiscomposedbysen<ment‐bearingwords.– ThislistofpolaradjecJvesinPortuguese•  contains30.322entries.•  iscomposedbythenameoftheadjecJveandapolaritywhichcanassignoneofthreevalues:+1,‐1and0.

•  ThesevaluescorrespondingtotheposiJve,negaJveandneutralsensesoftheadjec<ve.

– PIRPOusesthislisttocalculatethesemanJcorientaJonoftheconceptsfoundinthesentences.


PIRPOAlgorithm


PIRPOMeasureEvalua<on


•  Precision

•  Recall

•  F‐score(harmonicmeanofprecisionandrecall)

€

P ={relevantConcepts}∩{retrievedConcepts}

{retrievedConcepts}

€

R ={relevantConcepts}∩{retrievedConcepts}

{relevantConcepts}

€

F = 2 × P × RP + R

PIRPOPreliminaryResults


PIRPO:DiscussionontheResults•  PIRPOreachedabe/errecallforconceptswithposi<vepolarity,whilemixedpolarityhadahigherprecision.

•  ThelowF‐scorecanbemainlyduetothealgorithmhasassignedapolaritytoaspecificconceptoftheontology,whilethehumanclassifiedthereviewasawhole.


OutlinePart1




OntologyDevelopment


Part2

•  KnowledgeEngineering‐ModellingUGC





ContextWorkshop




Informa<onVisualisa<on•  Whatisthevisualmodelofthepoten<alend‐user?

•  Howshouldweproperlymapandrender:–  themostvaluedaccommoda<onfeatures?

–  thepercep<onofthequalityofferedbythehotel?–  thecorrela<onbetweentheguest’sprofileandthemostlyrelevantfeatures?

–  theintensityoftheposi<vityornega<vityofthefeatures?

•  Doestheuseofadvancedvisualtechniques(suchastreeoriented)tomaptheresultswillhelptheaccommoda<onmanagersandgueststohaveabe/erinsightofthedata?


ExploringInforma<onVisualisa<on•  Inthenextfigures

– ThecolorwasusedtomapthepolarityandthestrengthofthepolarityvaluesontheCO.

– ThesizewasusedtomapthefrequencythattheCOismen<onedinthereviews.


ExploringInforma<onVisualisa<on

Resultoftheapplica<onofBubbleTreevisualisaJonoftherela<onamongconceptsoftheontology,polarity(le\)and

strengthofthepolarity(right).

•  Carvalho,E.;Chaves,M.S.,2012.ExploringUser‐GeneratedDataVisualizaJonintheAccommodaJonSector.16thInternaJonalConferenceInformaJonVisualisaJon,IEEE.(SubmiXed)


ExploringInforma<onVisualisa<on


ResultsusingTreemapvisualisaJonoftherela<onamongtypeofcustomer,conceptsoftheontologyandpolarity.

Ques<onnaire(inSpanish)•  Youaregoingtoreceiveaques<onnaireaboutinforma<onvisualisa<onusingUGCinthecontextoftheaccommoda<onsector.

•  Please,clickhereh/p://kwiksurveys.com?u=Infovisestoanswerit.


FinalRemarks•  In‐depthanalysisofUGCcanbeusedasinputtoimprovedecisionmaking.

•  Itis<metothinkaboutnewmodelstostoreUGCdata.

•  ItisnecessarythebuildingfromthegroundofnewalgorithmstodealwithUGCforlanguagesotherthanEnglish.

•  InformaJonvisualisaJonofUGCisinitsinfancystate.


MainReferences•  S.Bethard,H.Yu,A.Thornton,V.Hatzivassiloglou,andD.Jurafsky,2004.Automa<cextrac<onofopinionproposi<onsand

theirholders.inProceedingsoftheAAAISpringSymposiumonExploringA%tudeandAffectinText.•  Chesley,P.;Vincent,B.;Xu,L.andSrihariR.,2006.Usingverbsandadjec<vestoautoma<callyclassifyblogsen<ment.in

AAAISymposiumonComputa<onalApproachestoAnalysingWeblogs(AAAI‐CAAW),27–29.

•  Ding,X.,Liu,B.,andYu,P.S.,2008.Aholis<clexicon‐basedapproachtoopinionmining.ProceedingsoftheConferenceonWebSearchandWebDataMining(WSDM).

•  M.HuandB.Liu,2004.Miningopinionfeaturesincustomerreviews.InProceedingsofAAAI,pp.755–760.

•  S.‐M.KimandE.Hovy,2004.Determiningthesen<mentofopinions.InProceedingsoftheInterna.onalConferenceonComputa.onalLinguis.cs(COLING),2004.

•  Liu,Bing,2010.Sen<mentAnalysisandSubjec<vity.InHandbookofNaturalLanguageProcessing,SecondEdi<on,Eds:N.IndurkhyaandF.J.Damerau),CRCPress,TaylorandFrancisGroup,BocaRaton,FL.Chapter28.

•  Mar<n,J.R.andWhite,P.R.R.,2005.TheLanguageofEvalua<on,AppraisalinEnglish,PalgraveMacmillan,London&NewYork.

•  Taboada,M.,Brooke,J.,Tofiloski,M.,Voll,K.D.,Stede,M.,2011.Lexicon‐basedmethodsforsen<mentanalysis.Computa<onalLinguis<cs37(2),267–307.

•  Tang,H.,Tan,S.,Cheng,X.,2009.Asurveyonsen<mentdetec<onofreviews.ExpertSystemswithApplica<ons36(7),10760–10773.

•  Whitelaw,C.;Garg,N.andArgamon,S.,2005.Usingappraisalgroupsforsen<mentanalysis.InProceedingsofthe14thACMinterna<onalconferenceonInforma<onandknowledgemanagement(CIKM'05).ACM,NewYork,NY,USA,625‐631.

•  Wilson,T.,2008.Fine‐GrainedSubjec<vityAnalysis.PhDDisserta<on,IntelligentSystemsProgram,UniversityofPi/sburgh.

•  Wilson,T.,Wiebe,J.,Hoffmann,P.,2009.Recognizingcontextualpolarity:Anexplora<onoffeaturesforphrase‐levelsen<mentanalysis.Computa<onalLinguis<cs35,399–433.

•  Y.Wu,F.Wei,S.Liu,N.Au,W.Cui,H.Zhou,andH.Qu,2010.OpinionSeer:Interac<veVisualisa<onofHotelCustomerFeedback.IEEETransac<onsonVisualiza<onandComputerGraphics,6,1109‐1118.Nov‐Dec.


Open‐sourcesen<ment‐analysistools• PythonNLTK(NaturalLanguageToolkit)–  h/p://www.nltk.organdh/p://text‐processing.com/demo/sen<ment

• R,TM(textmining)moduleh/p://cran.r‐project.org/web/packages/tm/index.html

• RapidMinerh/p://rapid‐i.com/content/view/184/196/

• GATE,theGeneralArchitectureforTextEngineeringh/p://gate.ac.uk/sen<ment

• UIMA‐plug‐inannotatorsforsen<ment—ApacheUIMAistheUnstructuredInforma<onManagementArchitecture,h/p://uima.apache.org/

•  SenJmentclassifiersfortheWEKAdata‐miningworkbench,h/p://www.cs.waikato.ac.nz/ml/weka/.

•  StanfordNLPtools‐h/p://www‐nlp.stanford.edu/so\waremaximum‐entropyclassifica<onapproachforsen<ment.


Thankyouverymuchforyoura/en<on!!

Ques<ons


a fine-grained analysis of user-generated content to support decision making

Education