1. 1. NoSQLWhy, When, and How starring CouchDB
2. 2. akabuilt rebuilding onhack atcontribute towork at
3. 3. One More thing...CoorganizingRESTFest2011w/MikeAmundsenrestfest.org
4. 4. What isWhy NoSQL ^
5. 5. NoSQL is...Not OnlySQLorSQLIvoteforthisone
6. 6. Origins of the NameCarloStrozziNoSQLappstartedin1998.RDBMSsansSQLappliedtononrelationalDBsaround2008ishno:sql(east)selectfun,profitfromreal_worldwhererelational=false;
7. 7. Origin of the SpeciesnonrelationaldatabasespredaterelationalonesitssortalikeAJAXbeendoinitforawhilegetsanamenowitscool!!1!
8. 8. Types of NoSQL DBsgraph(RDF/SemanticWeb/triples)keyvalue(justwhatitsays)document(k/v+queriability)object(biginthe1980s)multivalue(oldtech...like1960s)NewSQL?
9. 9. NewSQL?Recently(April,2011)coinedtermbyThe451GroupmostlymeansSQLdbs+betterscallabilityorNoSQLdbs+SQLlayersyeah!morekeywords!!keepsmarketinghappy...
10. 10. ...on to specics
11. 11. Graph (RDF) prettyheadystuff Web3.0?Maybe... simpleconcept,FlockDB (from Twitter) complicatedexecution (often)AllegroGraph Queriedwith: Cytoscape SPARQL Java
12. 12. Key Value Stores scalablecaches generallyno querylanguageProject Voldemortgetkey(s)Hibari returnvalue(s) BigConeed driven
13. 13. Document Databaseskey value + querying Lotus NotesAmazon SimpleDB
14. 14. object
15. 15. multivalueprettyantiquelotsoflegacyrollouts
16. 16. NewSQLanyofthepreviouslymentionedDBs+querylayers...maybeexpectanyandallSQLDBstojumponthistrainand/orslipinthenextfewyearsworthalookifyou*must*havenormalizedstoragebutwhoneedsnormalization?
17. 17. Why
18. 18. One Reason
19. 19. options
20. 20. denormalizationschemalessgraph/objectschemaclosermatchtobusinesslogicgenerallyfaster/morescalablecertainlymoredistributable
21. 21. alt.queriesMap/Reducethanks,Google(paperfrom2004)XPathand/orXQuerySPARQLSQL...orsomethingquitesimilarLinq
22. 22. licensingApacheLicense2.0thefavoriteAGPLv3thecommercialfavoriteothersincludeLGPL(v2/3),GPL(v2/3),BSD,MIT,customcommercialoropensource...heretherebedragons...
23. 23. (un)expected extras at least in CouchDBBuiltinWebServerorAppServerGeospatialboundingboxqueriesnmasterreplicationbinaryfilestoragescalesupanddown
24. 24. When
25. 25. ScenariosScalabilitycaching,shardingAnalyticsDataWarehousingUbiquitous/DistributedDatamobile,desktop,server
26. 26. ScalabilitycachestyleDBsdataservedfromRAM(mostly)Membase,ElasticCouchbase,memcached,MongoDB,Casandrahorizontalscalabilityaddmoreservers,notmoreserver
27. 27. AnalyticsHadoopHBaseCassandrathisbleedsoverintodatawarehousingquickly
28. 28. Ubiquitous/Distributed Data CouchDB serverdesktopmobile RiakMobile? gotoErlangFactoryinJune
29. 29. FALE Scenariospoweroutagesdatalossfailedpersistence,nopersistencenetworkunavailability special thanks to @coats who runs fale.ca for the FALE stamp
30. 30. Solution: CouchDB
31. 31. Howwith CouchDB
32. 32. Time to Relax
33. 33. Time to Relax ThatsDamien.HebuiltCouchDB.
34. 34. ThatstheCouchDB gangsign.Time to Relax ThatsDamien.HebuiltCouchDB.
35. 35. ThatstheCouchDB gangsign.Time to Relax Learnit! ThatsDamien.HebuiltCouchDB.
36. 36. and heres why
37. 37. CouchDB has Super Powers!SchemalessDocumentcentricReplication/SyncFailFastArchitecturestatelessAPIappendonlyfilestorage
38. 38. Document centricnaturaldatamodelstoredatalikeitexistseverywhereelse:asadocumentmap/reducevs.sqlsortingdocumentsoutofadrawervs.reassemblingthemfrombitsofdata
39. 39. Replication/SyncMVCCbasedtransactionsversioning...butonlymeantfortransactionssafelymergedatabasesdocumentsarentcompared,onlyUUIDs&revisionIDsconflictingdocumentsaremarkedandawinnerispicked
40. 40. Fail Fastappendonlydatabasefileeverythinggoesontheendofthefilequerysarecachedtheretoobouncebackfromerrorsratherthanspinwheelsindefinitely
41. 41. C.O.U.C.HCollectionOfUnreliableCommodityHardware
42. 42. CouchDB ScalingscalesupanddownserverdesktopmobilethankstonmasterreplicationBigCouchpluginforshardingHTTPAPIcanbeloadbalancedreverseproxiesandcaching
43. 43. Get CouchDBhttp://www.couchbase.com/downloads/couchbaseserver/communityhttp://iriscouch.com/bothhaveGeoCouchbuiltin!https://cloudant.com/(BigCouch/sharded)yoursystemspackageinstaller...http://wiki.apache.org/couchdb/Installation
44. 44. Time to Relax (command line)Apache CouchDB 1.0.2 (LogLevel=info) is starting.Apache CouchDB has started. Time to relax.[info] [] Apache CouchDB has started on http://127.0.0.1:5984/
45. 45. PHP & CouchDB
46. 46. HTTP ClientsAllyoureallyneedtogetstartedDoesrequireabetterunderstandingofCouchDBsAPI...butthatsAGoodThing!WillimproveyourHTTPskillz...anotherGoodThing!requiresmorework...thepartyouwontlikebutitsworthittolearnHTTP&REST
47. 47. CouchDB ClientsCouchDBsAPIisjustHTTPbut...helperlibrariescan...help:auto(en|de)codeJSONhandlebase64inginlineattachmentsmanageauthentication,cookies,OAuthtokenexchangecaching!!!_changesfeedwatching
48. 48. CouchDB ClientsSagforCouchDB(ApacheLicense2.0)PHPOnCouch(GPLv2orv3)Beyondhere,therebegiants...PHPillow(LGPL3)PHPObject_Freezer(BSD)PHPCouchDBExtension(PHPLicense3.0)Doctrine2CouchDBODM
49. 49. HTTP Clientscurlugh...messypecl_httplovely(nexttocurl),buttakessomeinstalltime,lacksexamplesZend_HTTP&PEARHTTP_Request2Mostmajorframeworkshavetheirown
50. 50. Client SuggestionsHTTPpickonethatsflexible(canhandleCOPY)CouchDBIuseSagcurrently.Caching,CookieAuth,nicename.PHPonCouchseemsgreat,butwatchthelicense(GPL)
51. 51. Todays StackInPHP:Sagsaggingcouch.com/ForHTTPAPIDemoing/Testing:Restygithub.com/micha/restyPosterforFirefoxcode.google.com/p/posterextension/
52. 52. Other Handy HTTPClientsHTTPClientforMacOSXCharlesProxy($$)httptwiddleforEmacsFiddlerforWindowsSolexforEclipse
53. 53. CouchDB HTTP APIwell be back to PHP in a bit
54. 54. JSON DocumentsallresponsesarevalidJSONJSONsupportisbuiltintoPHP5.2+pecl&purePHP(de|en)codeforolderversions
55. 55. A JSON Document{ json: key/value pairs, _id : some uuid, _rev: mvcc key, string keys: [1,2,3,four,null], schema free: {so its:flexible}}
56. 56. JSON to PHP Object$json={"json":"document","with":["an","array"]};$j=json_decode($json);//$jstdClassObject([json]=>document[with]=>Array([0]=>an[1]=>array))echo$j>with[1];
57. 57. JSON to PHP array $json = {"json":"document","with":["an", "array"]}; $j = json_decode($json, true); // $j Array ( [json] => document [with] => Array ( [0] => an [1] => array ) ) echo $j[with][1];
58. 58. I got tired of having topick -> or [] $j=newArrayObject( json_decod($json), ArrayObject::ARRAY_AS_PROPS ); print_r($j[with][0]); print_r($j>with[0]); //ansameresult!noerrors!
59. 59. HTTP / REST basics GETread PUT create or updateDELETEdeletePOSTbulk operation
60. 60. Restycommand line RESTful good times
61. 61. Setup Restyinstall(seerestypage)$resty.#GET,POST,PUT,DELETE&HEAD#arescriptsnow!$restyhttp://localhost:5984/#setRestytodefaulttoCouchDB
62. 62. Create a Database$PUT/pouch/{ok:true}$GET/pcouch/{"db_name":"pcouch","doc_count":0,"doc_del_count":0,"update_seq":0,"purge_seq":0,"compact_running":false,"disk_size":79,"instance_start_time":"1289923325819422","disk_format_version":5,"committed_update_seq":0}
63. 63. DELETE a Database$DELETE/pouch/{ok:true}#^^^becarefulwiththatone!#letsrecreateit$PUT/pouch/
64. 64. Create DocumentPUT/pouch/tek{php:tek}{"ok":true,"id":"test","rev":"189af21439a03933bc3fc8c14cbeb496e"}
65. 65. GET the DocumentGET/pouch/tek{"_id":"test","_rev":"189af21439a03933bc3fc8c14cbeb496e","php":"tek"}//wellneedthat_revvaluetoupdatethisdoc
66. 66. PUT (failure)on purpose...PUT/pouch/tek{php:tek}{"error":"conflict","reason":"Documentupdateconflict."}#weneedthat_revvaluenow
67. 67. PUT things rightPUT/pouch/tek{"_rev":"189af21439a03933bc3fc8c14cbeb496e","php":"tek"}#thatrepresentsanewline#doccouldalsocontaintheid{"ok":true,"id":"tek","rev":"2fff750985c2c2602e859fe38cd1d347e"}
68. 68. PUT binary attachments PUT/pouch/tek/photo?rev=2 fff750985c2c2602e859fe38cd1d347e Qfilename.png HContentType:image/png {"ok":true,"id":"tek", "rev":"318d519e58b569e43a6fd5e87491f0c4c"} GET/pouch/tek {"_id":"tek", "_rev":"318d519e58b569e43a6fd5e87491f0c4c", "_attachments":{"photo":{"content_type":"image/ png", "revpos":1,"length":18,"stub":true}}}
69. 69. A bit aboutattachmentsEachattachmenttoadochasitsownURL:/pouch/tek/photo/pouch/tek/schedule.pdfeachattachmentshasitsownmimetypeattachmentscanbeadded/updatedviatheirownURLsorinline
70. 70. DELETE (failure) on purpose...againDELETE/pouch/tek{"error":"conflict","reason":"Documentupdateconflict."}#youalwayshavetosendarevwhenchangingadocinanyway
71. 71. DELETEDELETE/pouch/tek?rev=318d519e58b569e43a6fd5e87491f0c4cQ#thatQtellsRestynottourlencode{"ok":true,"id":"tek","rev":"43e8c21e8e610c4ea7f8e247a45c6eb04"}#what?!?anotherrev?...butthe#documentshouldbedead?!
72. 72. GETing 404sGET/pouch/tek{"error":"not_found","reason":"deleted"}GET/pouch/tek12{"error":"not_found","reason":"missing"}#RESTifariannote:wouldbea409#ifcacheswerebuiltbetter
73. 73. That was just the very basics GET/_stats(serverstats) GET/_all_dbs(listallDBs) GET/_all_docs(listalldocs) GET/db/_changes(listrecent changes) superpowers!
74. 74. Map/Reduce Queriessearching the le draweressentiallystoredqueriesa.k.a.ViewswritteninJavaScript(orPythonorRubyorErlangorPHP?)similartoarray_map()/array_reduce(),butscalablenoadhocqueries
75. 75. Pouchput your media in the Couch starring PHP!
76. 76. i want to coverschemalessJSONdocs_securityBinaryAttachmentsserversiderenderingstuffMap/Reduce _show Javascript _list severalexamplesURLRewritingreplication _rewritesecurity vhosts validate_doc_updates
77. 77. Pouch is...aDatabaseofJSONdocsoffilemetadatawiththefileattached!afilesystemimporteraWebbasedCouchAppforbrowsing
78. 78. 0.1 Files into CouchDB gatherEXIFdata addattachment PUTtoCouchDB letusknowhowitwent
79. 79. code preamble#!/usr/bin/php