umap.eu (ieb2011)
DESCRIPTION
IEB2011 jardunaldietarako prestatutako aurkezpena, Aitzol Naberan eta Josu Azpillagak, Umap.eu-ren gaineanTRANSCRIPT
![Page 1: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/1.jpg)
IEB2011 | eu.umap.eu
1
eu.umap.eu
Josu Azpillaga
![Page 2: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/2.jpg)
IEB2011 | eu.umap.eu
eu.umap.eu
● Ideia. Zer da umap.eu● Tresna. Nola dabil.● Emaitzak. Estatistika batzuk.● Aurrera begira.
![Page 3: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/3.jpg)
IEB2011 | eu.umap.eu
Zer da umap.eu?
![Page 4: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/4.jpg)
IEB2011 | eu.umap.eu
Zer da umap.eu?
![Page 5: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/5.jpg)
IEB2011 | eu.umap.eu
![Page 6: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/6.jpg)
IEB2011 | eu.umap.eu
eu.umap.eu tresna... proba bat
![Page 7: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/7.jpg)
IEB2011 | eu.umap.eu
eu.umap.eu. tresna
● Tuitak jasotzea, denbora errealean● Erabiltzaile euskaldun berriak harrapatzen● Hizkuntza detekzioa● Albisteak, gakohitzak
![Page 8: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/8.jpg)
IEB2011 | eu.umap.eu
Tuitak jasotzea, denbora errealean
● Twitter. Streaming API– http://dev.twitter.com/pages/streaming_api
● Aukerak:– Location– Track (keywords)– Language and Country > users– Limit: eskaera berezia (50000)
● X (10) minutuz behin, eskaera berritu
![Page 9: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/9.jpg)
IEB2011 | eu.umap.eu
class Client: def __init__(self): self.buffer = u'' self.conn = pycurl.Curl() self.conn.setopt(pycurl.USERPWD, "%s:%s" % (USER, PASS)) self.conn.setopt(pycurl.URL, STREAM_URL) self.conn.setopt(pycurl.POSTFIELDS, urllib.urlencode(get_post_values())) self.conn.setopt(pycurl.WRITEFUNCTION, self.on_receive) try: self.conn.perform() except pycurl.error, e:
print "Error code: ", e[0] print "Error message: ", e[1]
def on_error(self, data): import pdb;pdb.set_trace() print data def on_receive(self, data): print data self.buffer += data if data.endswith("\r\n") and self.buffer.strip(): content = json.loads(self.buffer) a = create_status_from_dic(content) self.buffer = ""
![Page 10: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/10.jpg)
IEB2011 | eu.umap.eu
Erabiltzaile euskaldun berriak harrapatzen
● Ehhhh.... algoritmo...
● Hasteko erabiltzaile multzo batekin, nahikoa
● Garbiketa ere egiten da
● Norbanakoak bai; robotak?
● Eu... ondo; ca... ejem.
![Page 11: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/11.jpg)
IEB2011 | eu.umap.eu
Hizkuntza detekzioa
● Aukera diferenteak
● Errazena? Google API !!!
● Onena? Google API!!!– http://code.google.com/apis/language/
● Lizentziak ez du baimentzen. Orduan?
![Page 12: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/12.jpg)
IEB2011 | eu.umap.eu
Hizkuntza detekzioa
●
GuessLanguage erabiltzen dugu● LangID http://langid.net/
● Text cat http://odur.let.rug.nl/~vannoord/TextCat/
● LID http://web.me.com/dcavar/LID/
● Guess Language https://code.google.com/p/guesslanguage/
LangID Textcat LID GuessLanguage
Test 1. eu OK % 75 % 74 % 85 % 93
Test 2. eu OK % 76 % 88 % 96 % 95
![Page 13: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/13.jpg)
IEB2011 | eu.umap.eu
Albisteak, gakohitzak
● Gakohitzak
– Hitz guztiak zenbatu eta batu– Logika
● Hitz normala: 1 point● Hasha: 10 point● Tuit horrek link bat du: x1.1● Tuit hori retuit bat da: x1.1● ...● Aldatu liteke!
![Page 14: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/14.jpg)
IEB2011 | eu.umap.eu
Albisteak, gakohitzak
● Albisteak
– Link guztiak batu (ojo laburtzaileekin!)– x minutuz behin, aipatuenak kalkulatu– Aipatuen hoiek bisitatu eta informazio
relebantea ekarri● OEmbed!!!!!!!!!!● HTML parserra ;)
![Page 15: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/15.jpg)
IEB2011 | eu.umap.eu
oEmbed vs HTML parser
http://www.youtube.com/oembed?url=http%3A//www.youtube.com/watch%3Fv%3DbDOYN6gdRE&format=json
{"provider_url": "http:\/\/www.youtube.com\/", "title": "BRIARD ON SCOOTER - scooting dog",
<h1 id="watch-headline-title"> <span id="eow-title" class="" dir="ltr" title="BRIARD ON SCOOTER - scooting dog"> BRIARD ON SCOOTER - scooting dog </span> </h1> <div id="watch-headline-user-info"> <a id="watch-username" class="inline-block" rel="author" href="/user/courteous1"><strong>courteous1</strong></a>
vs
![Page 16: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/16.jpg)
IEB2011 | eu.umap.eu
eu.umap.eu tresna. kodea
● Kode guztia, horratx:– http://www.codesyntax.com/products
● cstwitterstatus● cstwitternews● cslanguagedetection
![Page 17: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/17.jpg)
IEB2011 | eu.umap.eu
Emaitzak: datu batzuk
● 2010eko urritik martxan (aurreko datuak...)● Datu orokorrak:
– Tuitak: 1.177.666 (aurrekoak beste 400.000)– eu tuitak: 151.969 (%13)– Erabiltzaileak: 2000tik gora jarraitzen. Eu
aktiboak? 1200 inguru... gutxiago seguraski.
![Page 18: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/18.jpg)
IEB2011 | eu.umap.eu
Emaitzak: datu batzuk
![Page 19: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/19.jpg)
IEB2011 | eu.umap.eu
Emaitzak: datu batzuk
● eu tuiten bat egin duten erabiltzaileak, 6 hilabetean: +%60
![Page 20: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/20.jpg)
IEB2011 | eu.umap.eu
Emaitzak: datu batzuk● 2010/11tik hona tuit kopuruaren araberako
erabiltzaileak
![Page 21: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/21.jpg)
IEB2011 | eu.umap.eu
Emaitzak: datu batzuk● 2010/11tik hona euskarazko tuit kopuruaren
araberako erabiltzaileak
![Page 22: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/22.jpg)
IEB2011 | eu.umap.eu
Emaitzak: datu batzuk● 2011 / 04. Retuitak.● Tuitak (eu): 26.982● Retuitak (eu): 4.109 (%15)● Erabiltzaile retuiteatuenak (eraginkorrenak!):
berria (225), kalaportu (126), argia (108), Sustatu (82), boligorria (75), euskalherrian (71), bildueh (57), zaldieroa (55), garanet (53), theklaneh (49), larbelaitz (48), luistxo (46), iturri (44), euskaljakintza (42), 17Korrika (41), info7irratia (40), zuzeu (40), dabidmp (36), robergutierrez (35), beaetaeneko (34), imnl (30), gaztea (30), tropela (30), teketen (29), Karrikiri (28), birasuegi (26), euskadi_irratia (25), ameslaria (25), EITBKultura (24), eitbcom (24), donostiakultura (24), ainhoaeus (24), baieuskarari (24), eguraldia (23), koldocas (23), goiena (22), zerdionk (22), hedoi_etxarte (21), jonabril (21), iPtx (21), jazpillaga (21), ueu_orokorra (20), exprai (20), GAZTEA (20), joxearanzabal (19), demonocrazy (19), IBIL2D (19), macmikel (18), kazetainfo (18), xletona (17),
![Page 23: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/23.jpg)
IEB2011 | eu.umap.eu
Emaitzak: datu batzuk● 2011 / 04. Loturak.● Tuitak (eu): 26.982● Loturadunak (eu): 14.802 (%55!!!)● Hedabide(!) aipatuenak:
eitb.com (1238), youtube.com (504), argia.com (395), berria.info (372), sustatu.com (314), goiena.net (310), zuzeu.com (245), gara.net (235), twitpic.com (230), feedburner.com (226), euskadi.net (200), twitter.com (197), ukberri.net (191), topatu.info (165), google.com (156), askatu.org (143), kulturweb.com (132), etengabe.blogsome.com (117), bizkaie.biz (116), kazeta.info (103), bildu.info (100), uberan.org (87), otamotz.com (87), aiaraldea.com (86)
![Page 24: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/24.jpg)
IEB2011 | eu.umap.eu
Emaitzak: datu batzuk● 2011 / 04. Albisteak.● Albisteak albistegi automatikoan: 508● Nondik?
eitb.com (49), argia.com (37), youtube.com (28), berria.info (26), gara.net (19), zuzeu.com (14), twitpic.com (12), sustatu.com (13), topatu.info (10), orekakomunikazioa.deusto.es (8), bildu.info (8), askatu.org (7), ezkerabertzalea.info (7), otamotz.com (6), bit.ly (6), ueu.org (6), hitzondo.net (6), diariovasco.com (5), askapena.org (5), ikasleabertzaleak.org (5), baisarean.org (5), feeds.feedburner.com (4), feedproxy.google.com (4), berriketan.info (4), imh.es (4), juankarh.blogspot.com (4), etxerat.info (4), elpais.com (3), euskaljakintza.com (3), albisteak.euskalherrian.info (3), eibar.org (3), bizkaie.biz (3), noticiasdegipuzkoa.com (3), flickr.com (3), imanolmurua.blogspot.com (2), deia.com (2), codesyntax.com (2), mukom.mondragon.edu (2),
![Page 25: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/25.jpg)
IEB2011 | eu.umap.eu
Aurrera begira
![Page 26: Umap.eu (IEB2011)](https://reader033.vdocuments.pub/reader033/viewer/2022060116/557dcca2d8b42a4c238b45b6/html5/thumbnails/26.jpg)
IEB2011 | eu.umap.eu
Eskerrik asko
@jazpillaga