Low Latency Loggingat BrightonPHP

James Titcumb

@asgrimgithub.com/asgrim

@asgrimgithub.com/asgrim

● Started on Amiga 500● PHP for 11 years● ZCE PHP 5.3

● Run PHP Hampshire user group● Lead dev on browscap.ini● First open source GoDeploy● Development Manager at Protected.co.uk● Mostly a ZF2 developer

Hello! :)

@asgrimgithub.com/asgrim

Errors

● Something broke… :)● e.g.

○ Can’t connect to MySQL (mysqli_connect)○ No such file or directory (fopen)

● Usually PHP core● Sometimes “fatal”

What are errors?

Problems?

source: http://www.dpaddbags.com/blog/episode-119-technology-sucks/

Problems.

● error_reporting setting● Errors look ugly● “@”● Limited options● Not very “OO”

Ways Around

// Will raise E_NOTICEfopen($somefile, 'r');

Ways Around

// No error! :)if (file_exists($somefile)) { fopen($somefile, 'r');} else { // nice handling... // maybe throw exception...}

Exceptions

Jargon Buster

● throwTriggering

● tryTry to run

● catchHandle it

● finallyRun code after try/catch

● Something still broke● OO● Catchable● Fatal errors● They are classes

What are exceptions?

● Built in to PHP ● More descriptive than just “Exception”, e.g.:

○ InvalidArgumentException○ LogicException○ OutOfBoundsException○ RuntimeException○ see PHP manual for more

SPL Exceptions

Example (exception class)

class DivisionByZeroException

extends LogicException

{

}

Example (throw)class Mathematics

{ public function divide($a, $b)

{

if ($b == 0) {

$msg = sprintf(‘Tried to divide %s by zero”, $a);

throw new DivisionByZeroException($msg);

}

return ($a / $b);

}

}

Example (catch)

$maths = new Mathematics();

try

{

$result = $maths->divide(5, 0);

}

catch (DivisionByZeroException $exception)

{

$logger->warning($exception->getMessage());

}

Logging

What is “logging”?Keeping a record of all events...

exceptions, errors, warnings, info, debug

Why use logging?

● Easier to find problems● More detail● “paper trail” for code● Log where you want

What about Apache’s error_log?

source: http://up-ship.com/blog/?p=20903

Why?

● error_log is too basic● Reading / parsing● error_reporting (again)

Doin’ it right wrong… /************************************************************\

* Magic file that makes your entire project work perfectly *

\************************************************************/

@ini_set('display_errors', 0);

@error_reporting(0);

function __globalErrorHandler()

{

return true;

}

@set_error_handler('__globalErrorHandler');

@set_exception_handler('__globalErrorHandler');

@register_shutdown_function(function() {

if(error_get_last())

{

echo "Script executed successfully!";

}

});https://github.com/webarto/boostrap.php

Requirements (for everyone)

● Fire & forget● Minimum or zero latency● Highly available● Log everything:

○ Exceptions○ Errors○ Fatal Errors○ Debug & info

● PSR-3 compatible

PSR-3

● Common logging interface→ LoggerInterface

● RFC-5424 Levels(debug, info, notice, warning, error, critical, alert, emergency)

● Interoperability● Reusability

● monolog (PSR-3)● Drupal - PSR-3 Watchdog● phpconsole● log4php● RavenPHP + Sentry● FirePHP (dev environment)● Roll your own

Logging Options

How they work...

source: http://mirror-sg-wv1.gallery.hd.org/_c/natural-science/cogs-with-meshed-teeth-AJHD.jpg.html

Capturing Logging

Use “capture methods”, send to $logger

● set_exception_handler()○ Handles all uncaught exceptions

● set_error_handler()○ Handles most errors

● register_shutdown_function()○ Handles fatal errors

Sending Log Messages

● Handler/Adapter translates● However you want…● Monolog has loads:

○ syslog-compatible / error_log○ Email, HipChat○ AMQP, Sentry, Zend Monitor, Graylog2○ Redis, MongoDB, CouchDB

Capture Method

Logger (PSR-3)

Handler / Adapter

Data Storage

Typical PSR-3 Compatible Design

Monolog\ErrorHandler->handleException()

Monolog\Logger->log()

Monolog\Handler->handle()

Monolog

Low LatencyHigh Performance

● Easy to add● Fire & Forget● Minimum impact to request

What do I mean?

Slow Logging

ApplicationBrowser Log Server

HTTP request

Send log message to database

Error!

Acknowledge message

HTTP response to client

Zero Latency Logging (ideal)

ApplicationBrowser Log Server

HTTP request

Send log message to database

Error!

HTTP response to client

What about UDP?

● Yes… Zero latency, but….

● Do you even care about your logs?(UDP means log messages may get lost...)

● TCP means guaranteed network delivery

● Any non-blocking fails

Low Latency Logging (balance)

ApplicationBrowser Log Server

HTTP request

Send log message to database

Error!

HTTP response to client

So how?

Say hello to RabbitMQ

● Robust messaging for applications● Easy to use● Runs on all major operating systems● Supports a huge number of developer

platforms● Open source and commercially supported

www.rabbitmq.com

What is RabbitMQ?

RabbitMQ - Basic

“Publisher” “Consumer”

queue

source: http://www.rabbitmq.com/tutorials/tutorial-one-php.html

RabbitMQ - Exchanges

“Consumer 1”

“Consumer 2”

queue 1

queue 2 “Publisher” “Exchange”

“Publisher”

“Publisher”

source: http://www.rabbitmq.com/tutorials/tutorial-three-php.html

“Publisher”

“Publisher”

Using Queues === Fast!Add RabbitMQ to logging architecture...

Ed\Log\Handler\ErrorHandler->handleException()

Ed\Log\Logger->log()

Ed\Log\Publisher\AmqpPublisher->publish()

Logging Server

Low Latency (using AMQP)

RabbitMQ JSON payload

Fetch message

Low Latency Logging (with AMQP)

ApplicationBrowser Log Server

HTTP request

JSON via AMQP

Error!

HTTP response

RabbitMQ

Why bother?● Scalability

RabbitMQ

Application A

Application B

Application C

Log Worker

Log Worker

Log Worker

Log Worker

Single Point of Failure...

● RabbitMQ can do HA

RabbitMQNode 1

RabbitMQNode 3

RabbitMQNode 2

RabbitMQNode 4

RabbitMQNode 5

RabbitMQNode 6

Live demo(pre-recorded… meh)

Questions?

Feedback please...https://joind.in/9928

Thanks!

@asgrimgithub.com/asgrim

https://joind.in/9928