low latency logging (brightonphp - 18th nov 2013)
Post on 11-May-2015
3.158 Views
Preview:
DESCRIPTION
TRANSCRIPT
Low Latency Loggingat BrightonPHP
James Titcumb
@asgrimgithub.com/asgrim
@asgrimgithub.com/asgrim
● Started on Amiga 500● PHP for 11 years● ZCE PHP 5.3
● Run PHP Hampshire user group● Lead dev on browscap.ini● First open source GoDeploy● Development Manager at Protected.co.uk● Mostly a ZF2 developer
Hello! :)
@asgrimgithub.com/asgrim
Errors
● Something broke… :)● e.g.
○ Can’t connect to MySQL (mysqli_connect)○ No such file or directory (fopen)
● Usually PHP core● Sometimes “fatal”
What are errors?
Problems?
source: http://www.dpaddbags.com/blog/episode-119-technology-sucks/
Problems.
● error_reporting setting● Errors look ugly● “@”● Limited options● Not very “OO”
Ways Around
// Will raise E_NOTICEfopen($somefile, 'r');
Ways Around
// No error! :)if (file_exists($somefile)) { fopen($somefile, 'r');} else { // nice handling... // maybe throw exception...}
Exceptions
Jargon Buster
● throwTriggering
● tryTry to run
● catchHandle it
● finallyRun code after try/catch
● Something still broke● OO● Catchable● Fatal errors● They are classes
What are exceptions?
● Built in to PHP ● More descriptive than just “Exception”, e.g.:
○ InvalidArgumentException○ LogicException○ OutOfBoundsException○ RuntimeException○ see PHP manual for more
SPL Exceptions
Example (exception class)
class DivisionByZeroException
extends LogicException
{
}
Example (throw)class Mathematics
{ public function divide($a, $b)
{
if ($b == 0) {
$msg = sprintf(‘Tried to divide %s by zero”, $a);
throw new DivisionByZeroException($msg);
}
return ($a / $b);
}
}
Example (catch)
$maths = new Mathematics();
try
{
$result = $maths->divide(5, 0);
}
catch (DivisionByZeroException $exception)
{
$logger->warning($exception->getMessage());
}
Logging
What is “logging”?Keeping a record of all events...
exceptions, errors, warnings, info, debug
Why use logging?
● Easier to find problems● More detail● “paper trail” for code● Log where you want
What about Apache’s error_log?
source: http://up-ship.com/blog/?p=20903
Why?
● error_log is too basic● Reading / parsing● error_reporting (again)
Doin’ it right wrong… /************************************************************\
* Magic file that makes your entire project work perfectly *
\************************************************************/
@ini_set('display_errors', 0);
@error_reporting(0);
function __globalErrorHandler()
{
return true;
}
@set_error_handler('__globalErrorHandler');
@set_exception_handler('__globalErrorHandler');
@register_shutdown_function(function() {
if(error_get_last())
{
echo "Script executed successfully!";
}
});https://github.com/webarto/boostrap.php
Requirements (for everyone)
● Fire & forget● Minimum or zero latency● Highly available● Log everything:
○ Exceptions○ Errors○ Fatal Errors○ Debug & info
● PSR-3 compatible
PSR-3
● Common logging interface→ LoggerInterface
● RFC-5424 Levels(debug, info, notice, warning, error, critical, alert, emergency)
● Interoperability● Reusability
● monolog (PSR-3)● Drupal - PSR-3 Watchdog● phpconsole● log4php● RavenPHP + Sentry● FirePHP (dev environment)● Roll your own
Logging Options
How they work...
source: http://mirror-sg-wv1.gallery.hd.org/_c/natural-science/cogs-with-meshed-teeth-AJHD.jpg.html
Capturing Logging
Use “capture methods”, send to $logger
● set_exception_handler()○ Handles all uncaught exceptions
● set_error_handler()○ Handles most errors
● register_shutdown_function()○ Handles fatal errors
Sending Log Messages
● Handler/Adapter translates● However you want…● Monolog has loads:
○ syslog-compatible / error_log○ Email, HipChat○ AMQP, Sentry, Zend Monitor, Graylog2○ Redis, MongoDB, CouchDB
Capture Method
Logger (PSR-3)
Handler / Adapter
Data Storage
Typical PSR-3 Compatible Design
Monolog\ErrorHandler->handleException()
Monolog\Logger->log()
Monolog\Handler->handle()
Monolog
Low LatencyHigh Performance
● Easy to add● Fire & Forget● Minimum impact to request
What do I mean?
Slow Logging
ApplicationBrowser Log Server
HTTP request
Send log message to database
Error!
Acknowledge message
HTTP response to client
Zero Latency Logging (ideal)
ApplicationBrowser Log Server
HTTP request
Send log message to database
Error!
HTTP response to client
What about UDP?
● Yes… Zero latency, but….
● Do you even care about your logs?(UDP means log messages may get lost...)
● TCP means guaranteed network delivery
● Any non-blocking fails
Low Latency Logging (balance)
ApplicationBrowser Log Server
HTTP request
Send log message to database
Error!
HTTP response to client
So how?
Say hello to RabbitMQ
● Robust messaging for applications● Easy to use● Runs on all major operating systems● Supports a huge number of developer
platforms● Open source and commercially supported
www.rabbitmq.com
What is RabbitMQ?
RabbitMQ - Basic
“Publisher” “Consumer”
queue
source: http://www.rabbitmq.com/tutorials/tutorial-one-php.html
RabbitMQ - Exchanges
“Consumer 1”
“Consumer 2”
queue 1
queue 2 “Publisher” “Exchange”
“Publisher”
“Publisher”
source: http://www.rabbitmq.com/tutorials/tutorial-three-php.html
“Publisher”
“Publisher”
Using Queues === Fast!Add RabbitMQ to logging architecture...
Ed\Log\Handler\ErrorHandler->handleException()
Ed\Log\Logger->log()
Ed\Log\Publisher\AmqpPublisher->publish()
Logging Server
Low Latency (using AMQP)
RabbitMQ JSON payload
Fetch message
Low Latency Logging (with AMQP)
ApplicationBrowser Log Server
HTTP request
JSON via AMQP
Error!
HTTP response
RabbitMQ
Why bother?● Scalability
RabbitMQ
Application A
Application B
Application C
Log Worker
Log Worker
Log Worker
Log Worker
Single Point of Failure...
● RabbitMQ can do HA
RabbitMQNode 1
RabbitMQNode 3
RabbitMQNode 2
RabbitMQNode 4
RabbitMQNode 5
RabbitMQNode 6
Live demo(pre-recorded… meh)
Questions?
top related