cli, the other sapi

CLI, the other SAPIUsing PHP on the command line in a Linux

environmentThijs Feryn - [email protected]

mailto:[email protected]

mailto:[email protected]

• Thijs Feryn (http://blog.feryn.eu)• Support manager at COMBELL• Zend Certified PHP 5 developer• I love the LAMP stack• Working on my MySQL certification• Open-source contributor• Nerd/geek of some sort

Who am I?

http://blog.feryn.eu

http://blog.feryn.eu

Who’s COMBELL?

• Largest independent hoster in Belgium• Founded in 1999 (a decade of COMBELL)• Focus on premium/quality hosting• More than 25 000 customers• More than 100 000 domains• More than 20 000 websites• More than 1000 servers• More than 800 resellers

Summarized: authorative partner for all hosted activity

Who’s COMBELL?

• Who’s a developer?

• Who’s a PHP developer?

• Who has ever used the PHP CLI?

• Who is used to working on Linux?

• Who has experience with process forking?

Who are you?

1. SAPI ... schmapi? (definition)2. Common PHP SAPI’s (overview)3. The CLI SAPI (how to use, possibilities, when to use)4. Apache vs CLI (a comparison)5. PHP binary options6. CLI in action (some examples)7. PCNTL == $fooDoo (the magic of process control)8. Wake the daemons (putting it all together)9. Q & A (the obligatory epilogue)

What’s up?

Definition

SAPI ... schmapi?

Wikipedia says: “The Server Application Programming Interface

(SAPI) is the generic term used to designate direct module interfaces to

web server applications”

In concreto: the SAPI is the way your server interacts with PHP

Overview

• Apache/Apache 2

• CGI

• FastCGI

• ISAPI

• CLI

• GTK

Common SAPI’s

Enabling the SAPI you need

• CLI is included by default• Add the right parameter to enable the Apache

Common SAPI’s

./configure --with-apxs=/location/of/your/apache/module

SAPI when compiling from source

Definition

The CLI SAPI

Php.net says: As of version 4.3.0, PHP supports a new SAPI type (Server Application Programming Interface) named CLI which means Command Line Interface. As the name implies, this SAPI type main focus is on developing shell (or desktop as well) applications with PHP.

In Concreto: PHP scripts are no longer exclusively called via the web browser. Using CLI, PHP scripts can be executed from the command line of the server which offers a vast array of benefits

When to use

• In cron’s

• For batches

• Process control

• Linux interactivity (e.g.: pipes)

The CLI SAPI

CLI 101

• Each CLI script is called via the PHP binary

• Arguments can be passed

• Arguments can be interpreted via special variables

• Input can be passed via STDIN

• Output can be returned via STDOUT

• Errors can be returned via STDERR

• I/O via pipes is possible

The CLI SAPI

$ php cli.php

CLI 101: arguments

• Passing arguments

• Interpreting arguments via $_SERVER

• $_SERVER[‘argc’] = counts the number of arguments

• $_SERVER[‘argv’][0] = scriptname (cli.php)

• $_SERVER[‘argv’][1] = first argument (arg1)

• $_SERVER[‘argv’][2] = second argument (arg2)

The CLI SAPI

$ php cli.php arg1 arg2

CLI 101: arguments

If ‘register_argc_argv’ is enabled 2 extra local variables are registered:

• $argc: counts the number of arguments

• $argv: array which stores the arguments

• $argv[0] = scriptname (cli.php)

• $argv[1] = first argument (arg1)

• $argv[2] = second argument (arg2)

The CLI SAPI

CLI 101: arguments & getopt()

• Gets command line arguments & values based on an input variable containing all the options:

• Individual characters (= flags, do not accept values)

• Characters followed by colon (input value required)

• Characters followed by 2 colons (input value optional)

• Long options only work well in PHP 5.3

• Check out Zend_Console_Getopt, it has some nice features similar to getopt()

The CLI SAPI

$options = getopt(‘ab:c::’);

CLI 101: input

• Reading from STDIN by opening a stream

• Reading a single line from STDIN

The CLI SAPI

$handle = fopen('php://stdin', 'r');

$stdin = trim(fgets(STDIN));

CLI 101: output

• Writing to STDOUT by opening a stream

• Writing a single line to STDERR

The CLI SAPI

$handle = fopen('php://stdout', 'w');

fwrite(STDOUT,‘output’);

CLI 101: errors

• Writing to STDERR by opening a stream

• Writing a single line to STDERR

The CLI SAPI

$handle = fopen('php://stderr', 'w');

fwrite(STDERR,‘error’);

CLI 101: piping

Using STDIN, STDOUT & STDERR streams you can easily benefit from the piping mechanisms in Linux to interact with other binaries

The CLI SAPI

$ cat input.ext | php myCliScript.php > output.ext

$ php myCliScript.php | grep filterText > output.ext

CLI 101: piping

You can also combine it with arguments

The CLI SAPI

$ php myCliScript.php arg1 arg2 > output.ext

$ cat input.ext | php myCliScript.php arg1 arg2 arg3 | grep filterText > output.ext

State means everything

Apache VS CLI

Apache• HTTP is stateless a protocol

• Limited interactivity

• Sessions & cookies as workaround for these drawbacks

• Execution timeouts

• Limited packet size

State means everything

Apache VS CLI

CLI• Controllable state

• More interactivity

• No (need for) sessions

• No execution timeouts

Ins & outs: input

Apache VS CLI

Apache$_GET

$_POST

$_COOKIE

$_SESSION

$_SERVER

$_ENV

CLI$_SERVER[‘argv’]

$argv

$_ENV

getopt()

STDIN

Ins & outs: output

Apache VS CLI

ApacheHTTP output only

Limited packet size

CLISTDOUT

STDERR

CLI usage, Apache mentality

Apache VS CLI

• Don’t use sessions or cookies, just use a local variables

• Don’t “get” your input, “read” it

• No execution timeouts

• Bundled output (HTTP): in response message

• Distributed output (CLI): via STDOUT. Available when needes

CLI usage, Apache mentality

Apache VS CLI

• OVERHEAD ALERT: don’t use HTTP via crons (e.g. via lynx or wget), use the php binary.

• CLI mode doesn’t change the CWD (current working directory). Be careful with path reference

• Use “dirname(__FILE__)”

• Use “chdir()”

CLI options reference

PHP binary options

Usage: php [options] [-f] <file> [--] [args...] php [options] -r <code> [--] [args...] php [options] [-B <begin_code>] -R <code> [-E <end_code>] [--] [args...] php [options] [-B <begin_code>] -F <file> [-E <end_code>] [--] [args...] php [options] -- [args...] php [options] -a

-a Run interactively -c <path>|<file> Look for php.ini file in this directory -n No php.ini file will be used -d foo[=bar] Define INI entry foo with value 'bar' -e Generate extended information for debugger/profiler -f <file> Parse and execute <file>. -h This help -i PHP information -l Syntax check only (lint) -m Show compiled in modules -r <code> Run PHP <code> without using script tags <?..?> -B <begin_code> Run PHP <begin_code> before processing input lines -R <code> Run PHP <code> for every input line -F <file> Parse and execute <file> for every input line -E <end_code> Run PHP <end_code> after processing all input lines -H Hide any passed arguments from external tools. -s Display colour syntax highlighted source. -v Version number -w Display source with stripped comments and whitespace. -z <file> Load Zend extension <file>.

args... Arguments passed to script. Use -- args when first argument starts with - or script is read from stdin

--ini Show configuration file names

--rf <name> Show information about function <name>. --rc <name> Show information about class <name>. --re <name> Show information about extension <name>. --ri <name> Show configuration for extension <name>.

Run code

PHP binary options

$ php -r “echo \"Give me some output\";”

Give me some output

$

• Binary option: -r

• Meaning: parse & run PHP code that is passed as an argument

Example

Interactive mode

PHP binary options

$ php -a

$ php > echo "Give me some interactive output";

Give me some interactive output

$ php>

• Binary option: -a

• Meaning: running PHP interactively in a true command line environment

• Requirements: PHP compiled with readline support

Example

Define configuration

PHP binary options

$ php -d max_execution_time=20 -r “echo ini_get("max_execution_time");”20$

• Binary option: -d

• Meaning: define INI configuration settings at runtime

Example

Lint check

PHP binary options

• Binary option: -l

• Meaning: perform a “lint” check on a PHP file. Validates the syntax of a script. Errors are printed via STDOUT. Shell return codes are “0” or “1”

• Notice:

• Doesn’t work combined with the “-r” option.

• Only checks for parsing errors & cannot retrieve fatal errors (e.g. undefined functions)

List modules

PHP binary options

$ php -m

[PHP Modules]

xmltokenizerstandardsessionposixpcreoverloadmysqlmbstringctype

[Zend Modules]

• Binary option: -m

• Meaning: prints all loaded PHP & Zend modules

Example

Syntax highlighting

PHP binary options

$ echo "<?php var_dump($_SERVER);" | php -s

<code>

<?php var_dump(); 



$

• Binary option: -s

• Meaning: format PHP code into highlighted HTML code

• Notice: doesn’t work with the “-r” option.

Example

Version information

PHP binary options

$ php -v

PHP 5.2.4-2ubuntu5.3 with Suhosin-Patch 0.9.6.2 (cli) (built: Jul 23 2008 06:44:49)

Copyright (c) 1997-2007 The PHP Group

Zend Engine v2.2.0, Copyright (c) 1998-2007 Zend Technologies

with Xdebug v2.0.3, Copyright (c) 2002-2007, by Derick Rethans

$

• Binary option: -v

• Meaning: displays version information

Example

Function reflection

PHP binary options

$ php --rf var_dumpFunction [ <internal:standard> function var_dump ] { - Parameters [2] { Parameter #0 [ <required> $var ] Parameter #1 [ <optional> $... ] }}$

• Binary option: --rf

• Meaning: displays function API information

• Requirements: PHP must be compiled with “Reflection” support

Example

Class reflection

PHP binary options

• Binary option: --rc

• Meaning: displays class API information

• Requirements: PHP must be compiled with “Reflection” support

Class reflection

PHP binary options

$ php --rc stdclass

Class [ <internal> class stdClass ] { - Constants [0] { } - Static properties [0] { } - Static methods [0] { } - Properties [0] { } - Methods [0] { }}

$

Example

Read from STDIN + distributed output

CLI in action

<?php/** * Read input from STDIN and print input with line numbers. * Quit when blank rule entered. */$line = 0;do{ $line++; $input = trim(fgets(STDIN)); if(strlen($input) > 0){ echo "$line# $input".PHP_EOL; }else{ break; }}while (true); Code

Read from STDIN + distributed output

CLI in action

$ php cli.php

this1# thisis2# ismy3# myoutput4# output

$

Output

Read from STDIN + bundled output

CLI in action

<?php/** * Read input from STDIN and print input with line numbers. * Quit when blank rule entered. * Bundle output and print at the end of the script. */$line = 0;$output = '';do{ $line++; $input = trim(fgets(STDIN)); if(strlen($input) > 0){ $output.= "$line# $input".PHP_EOL; }else{ break; }}while (true);echo $output; Code

Read from STDIN + bundled output

CLI in action

$ php cli.php

thisismyoutput

1# this2# is3# my4# output

$ Output

Working with arguments

CLI in action

<?php/** * Parse all arguments starting at index 1. * If format is "--key" or "-key", parse key and assign "true" as value * If format is "--key=value" or "-key=value", parse key and extract value * Else, value is argument, key is argument index */for($i=1;$i<$argc;$i++) { if(preg_match('/^(\-+)([a-zA-Z0-9_\-]+)(=([a-zA-Z0-9_\-])+)?$/',$argv[$i],$matches)){ $key = $matches[2]; if(array_key_exists(4,$matches)){ $value = $matches[4]; } else { $value = true; } } else { $key = $i; $value = $argv[$i]; } echo "Key: $key - Value: $value".PHP_EOL;}

Code

Working with arguments

CLI in action

$ php cli.php --flag1 -flag2 --option1=value1 -option2=value2 unNamedArgument

Key: flag1 - Value: 1Key: flag2 - Value: 1Key: option1 - Value: 1Key: option2 - Value: 2Key: 5 - Value: unNamedArgument

$Output

Shell binary

CLI in action

#!/usr/bin/php<?php/** * Print the current time. * This binary directly uses the PHP binary. * When the script has execute permissions it can be called as "./cli" instead of "php cli.php" */echo "The current time is ".date('H:i:s').PHP_EOL;

Code

$ chmod +x ./cli

$

$ ./cli

The current time is 17:43:08

$

Permissions

Call script + output

Linux interaction & piping

CLI in action

$ php -r 'for($i=0;$i<10;$i++) echo $i.PHP_EOL;' | wc -l

10

$

• Pass PHP code to binary using the “run” flag

• Print 10 lines

• Pipe output from PHP script as input for the “word count” binary

• Add the “-l” flag to count the number of lines

• Output 10

Using getopt()

CLI in action

<?php$shortopts = "";$shortopts .= "f:"; // Required value$shortopts .= "v::"; // Optional value$shortopts .= "abc"; // These options do not accept values

$options = getopt($shortopts);var_dump($options);

$ php cli.php -a -v -f value

array(3) { ["a"]=> bool(false) ["v"]=> bool(false) ["f"]=>string(5) "value"}

Code

Output

The magic of process control

PCNTL == $fooDoo

Php.net says: Process Control support in PHP implements the Unix style of process creation, program execution, signal handling and process termination. Process Control should not be enabled within a web server environment and unexpected results may happen if any Process Control functions are used within a web server environment.

InstallationAdd “--enable-pcntl” configuration option when compiling PHP

Forking is not a culinary term

PCNTL == $fooDoo

• pcntl_fork() copies the program execution into a child process

• Workload can be distributed between parent & child(s) via PID checking

• Allow “multithreadish’ parallel execution by forking multiple child processes

• Use a shared resource to define the workload (e.g.: file, array, DB, IPC, ...)

Wikipedia says: In computing, when a process forks, it creates a copy of itself, which is called a "child process." The original process is then called the "parent process". More generally, a fork in a multithreading environment means that a thread of execution is duplicated, creating a child thread from the parent thread.

PCNTL voodoo/$fooDoo

PCNTL == $fooDoo

• After forking, your logic is distributed in different PHP processes

• Forking too many childs can cause performance issues

• When using infinite loops, build in sleeps to avoid performance issues

• Only use PCNTL in CLI mode

• Be absolutely sure you master & control your child processes

• Avoid zombie processes

• Be able to kill your child processes at any time

• Only kil your own processes

Be very careful when forking child processes, because there are risks involved (hence the term $fooDoo)

Forking example

PCNTL == $fooDoo

<?php$pid = pcntl_fork();if ($pid == -1) { die('could not fork');} else if ($pid) { echo "[parent] I am the parent process and my child process has PID $pid".PHP_EOL; echo "[parent] Waiting for child termination".PHP_EOL; pcntl_wait($status); echo "[parent] Exiting".PHP_EOL;} else { for($i=0;$i<10;$i++){ echo "[child] Loop $i, sleeping for a second ...".PHP_EOL; sleep(1); } echo "[child] Exiting".PHP_EOL; exit;}

<?phpfor($i=0;$i<10;$i++){ $pid = pcntl_fork(); if ($pid == -1) { die('could not fork'); } else if ($pid == 0) { echo "[child] Child $i doing stuff".PHP_EOL; sleep(1); echo "[child] Child $i exiting".PHP_EOL; exit; } else { echo "[parent] Starting parent $i".PHP_EOL; pcntl_wait($status); echo "[parent] Ending parent $i".PHP_EOL; }}

Multiple forking

PCNTL == $fooDoo

<?phpecho "[parent] Starting parent".PHP_EOL;for($i=0;$i<10;$i++){ $pid = pcntl_fork(); if ($pid == -1) { die('could not fork'); } else if ($pid == 0) { echo "[child] Child $i doing stuff".PHP_EOL; sleep($i); echo "[child] Child $i exiting".PHP_EOL; exit; } }echo "[parent] Ending parent".PHP_EOL;exit;

Multiple forking without waiting

PCNTL == $fooDoo

Signals

PCNTL == $fooDoo

Wikipedia says: A signal is a limited form of inter-process communication used in Unix, Unix-like, and other POSIX-compliant operating systems. Essentially it is an asynchronous notification sent to a process in order to notify it of an event that occurred. When a signal is sent to a process, the operating system interrupts the process' normal flow of execution. Execution can be interrupted during any non-atomic instruction. If the process has previously registered a signal handler, that routine is executed. Otherwise the default signal handler is executed.

Signals

PCNTL == $fooDoo

• Communication between 2 processes

• Process sends asynchronous notification of an event

• Signal handlers can be used to execute custom logic

• Otherwise default signal handlers

Signals: SIGTERM

PCNTL == $fooDoo

• SIGTERM is used to terminate a process in a ‘nice’ way

• Much more gentle than SIGKILL

• Allows cleanup and closure of process

• Issued via ‘kill’ (no kill -9)

Signals: SIGINT

PCNTL == $fooDoo

• SIGINT is used to terminate a process via interruption

• Stops program execution just like SIGTERM

• Issued by a terminal (via CTRL+C)

Signals: SIGCHLD

PCNTL == $fooDoo

• SIGCHLD is sent to the parent when a child process is terminated

• Common when using process forking

• SIGCHLD is by default ignored and a zombie process is created

• Only sent when the parent issues a ‘wait’ call (avoids zombies)

<?phpdeclare(ticks = 1);function sig_handler($signo){

switch ($signo) { case SIGTERM: echo "SIGTERM".PHP_EOL; exit(); break; case SIGINT: echo "SIGINT".PHP_EOL; exit(); break; }

}pcntl_signal(SIGTERM, "sig_handler");pcntl_signal(SIGINT, "sig_handler");sleep(100);

Signals: example of SIGTERM & SIGINT

PCNTL == $fooDoo

<?phpdeclare(ticks = 1);$max=5;$simultaneous=0;$childId=0;

function sig_handler($signo) { global $simultaneous,$childId; switch ($signo) { case SIGCHLD: $simultaneous--; echo "SIGCHLD received for child #$childId".PHP_EOL; echo "Decrementing to $simultaneous".PHP_EOL; }}

pcntl_signal(SIGCHLD, "sig_handler");

Signals: example of SIGCHLD

PCNTL == $fooDoo

for ($childId=0;$childId<10;$childId++){ $pid=pcntl_fork(); if ($pid == -1) { die("could not fork"); } else if ($pid) { if ( $simultaneous >= $max ){ pcntl_wait($status); } else { $simultaneous++; echo "Increment to $simultaneous".PHP_EOL; } } else { echo "Starting new child #$childId ".PHP_EOL; echo "Now we de have $simultaneous simultaneous child processes".PHP_EOL; sleep(rand(3,5)); exit; }}

Signals: example of SIGCHLD

PCNTL == $fooDoo

POSIX functions

PCNTL == $fooDoo

Wikipedia says: POSIX or "Portable Operating System Interface for Unix" is the collective name of a family of related standards specified by the IEEE to define the application programming interface (API), along with shell and utilities interfaces for software compatible with variants of the Unix operating system, although the standard can apply to any operating system.

POSIX functions

PCNTL == $fooDoo

In PHP: POSIX functions in PHP allow you to interact with the POSIX interface of the operating system. For process control functions we mainly use POSIX to retrieve PID information and terminate/kill processes

POSIX functions

PCNTL == $fooDoo

• Determine the PID of the current process

• pcntl_fork() return value for the parent process is the same as posix_getpid() for the child process

posix_getpid()

• Determine the PID of the parent of the process

• posix_getpid() value for the parent process is the same as posix_getppid() for the child process

• posix_getppid() value for the parent process is the PID of the bash session

posix_getppid()

<?phpecho "[prefork] PID POSIX: ".posix_getpid().", parent PID: ".posix_getppid().PHP_EOL;for($i=0;$i<3;$i++){ $pid = pcntl_fork(); if ($pid == -1) { die('could not fork'); } else if ($pid == 0) { echo "[child] ID: $i, PID: ".posix_getpid().", parent PID: ".posix_getppid().PHP_EOL; exit; } else { echo "[parent] PID : ".posix_getpid().", parent PID: ".posix_getppid().PHP_EOL; pcntl_wait($status); }}

POSIX functions: example of posix_getpid() & posix_getppid()

PCNTL == $fooDoo

POSIX functions

PCNTL == $fooDoo

• Send a signal to a certain process

• posix_kill($pid,0): gets PID status

• true: pid exists

• false: pid no longer exists

• posix_kill($pid,9) or posix_kill($pid,SIGKILL): kill a process

• posix_kill($pid,15) or posix_kill($pid,SIGTERM): terminate a process

posix_kill()

<?php$pid=pcntl_fork(); if ($pid == -1) { die("could not fork");} else if ($pid) { $exists = posix_kill($pid,0)?'still':'no longer'; echo "[parent] Child process $pid $exists exists".PHP_EOL; echo "[parent] Killing child process $pid".PHP_EOL; posix_kill($pid,SIGKILL); echo "[parent] Child process $pid killed".PHP_EOL; pcntl_wait($status); $exists = posix_kill($pid,0)?'still':'no longer'; echo "[parent] Child process $pid $exists exists".PHP_EOL; } else { while(true){ sleep(100); } exit;}

POSIX functions: example of posix_kill()

PCNTL == $fooDoo

#!/usr/bin/php<?phpdate_default_timezone_set('Europe/Brussels');$options = getopt('p:d:');/** * Determine pidfile */if(!array_key_exists('p',$options)){ die('No pidfile specified'.PHP_EOL);}define('PIDFILE',$options['p']);/** * Determine datadir */if(!array_key_exists('d',$options)){ die('No data directory specified'.PHP_EOL);}else{ if(!is_dir($options['d'])){ die('Data directory does not exist'.PHP_EOL); }}define('DATADIR',$options['d']);

The daemon (1)

Wake the daemons

/** * Delete pidfile on startup */if(file_exists(PIDFILE)){ unlink(PIDFILE);}/** * Delete pidfile after SIGINT or SIGTERM */function signal_handler($sig){ if(file_exists(PIDFILE)){ unlink(PIDFILE); }}/** * Set signal handler */pcntl_signal(SIGINT,'signal_handler');pcntl_signal(SIGTERM,'signal_handler');

The daemon (2)

Wake the daemons

/* * Setup daemon */$pid=pcntl_fork(); if ($pid == -1) { die("could not fork".PHP_EOL);} else if ($pid) { exit; }/** * System settings */posix_setsid();chdir("/");/** * Store PID */$handle = fopen(PIDFILE,'w+');fwrite($handle,posix_getpid());fclose($handle);

The daemon (3)

Wake the daemons

/** * Perform daemon logic */while(true){ $handle = fopen(DATADIR.'/thijs_'.date('YmdHis'),'w+'); fwrite($handle,'abc123'); fclose($handle); sleep(10);}

The daemon (4)

Wake the daemons

#!/usr/bin/php<?php/** * Count args */if($argc < 2){ die("\tUsage: php ".basename(__FILE__)." start|stop|status|restart".PHP_EOL);}/** * Init params */define('PIDFILE','/tmp/daemon.pid');define('DAEMONBIN','/home/thijs/test/pcntl/daemon.php');define('DATADIR','/tmp');function startDaemon(){ echo "Starting daemon".PHP_EOL; $start = DAEMONBIN.' -p'.PIDFILE.' -d '.DATADIR; passthru($start);}

The startup script (1)

Wake the daemons

switch ($argv[1]){ case 'start': startDaemon(); break; case 'stop': stopDaemon(); break; case 'status': statusDaemon(); break; case 'restart': stopDaemon(); sleep(2); startDaemon(); break; default: die("\tUsage: ".basename(__FILE__)." start | stop | status | restart".PHP_EOL);}


Wake the daemons

THANKS !

cli, the other sapi

Technology