Initial Commands

# Getting ready for Chef.
apt-get update
curl -L http://www.opscode.com/chef/install.sh | sudo bash
apt-get -y install git-core

# Running Chef.
git clone https://github.com/msonnabaum/drupalcon-training-chef-repo.git
cd drupalcon-training-chef-repo
chef-solo -c config/solo.rb -j config/run_list.json

# Downloading Drupal
cd /var/www
drush dl drupal

Installing Drupal

Navigate to http://${SERVER_IP}/drupal-7.28/

Install the "Standard" install profile.  Use "drupal" as the database name, "root" as the DB user, and "1234" as the DB password.

On the next installation page, set a Drupal site name, email address, username and password.  Choose whatever you'd like for this, so long as you can remember it.

ab

ab is installed as part of apache, but you can install it seperate with:

yum install ab

ab -n 100 -c 5 http://localhost/drupal-7.28/

Results of load testing with changes outlined below with Drupal 7

OPcache replaces APC for PHP 5.5 or news

Note: if you are using php 5.5 or newer there is a build in OPcache cache that is better than APC and you should use.

# yum install php_opcache 

Edit /etc/php.ini and add the following:

; Enable Zend OPcache for PHP 5.5
zend_extension=opcache.so
opcache.enable=On

Other php.ini settings for OPcache (on fedora these are installed in /etc/php.d/opcache.ini for you)

; Sets how much memory to use
opcache.memory_consumption=128

;Sets how much memory should be used by OPcache for storing internal strings 
;(e.g. classnames and the files they are contained in)
opcache.interned_strings_buffer=8

; The maximum number of files OPcache will cache
opcache.max_accelerated_files=4000

;How often (in seconds) to check file timestamps for changes to the shared
;memory storage allocation.
opcache.revalidate_freq=60

;If enabled, a fast shutdown sequence is used for the accelerated code
;The fast shutdown sequence doesn't free each allocated block, but lets
;the Zend Engine Memory Manager do the work.
opcache.fast_shutdown=1

;Enables the OPcache for the CLI version of PHP.
opcache.enable_cli=1

To view the status of opcache, install this php file and run from browser

cd /var/www/example.com/htdocs

wget https://raw.github.com/rlerdorf/opcache-status/master/opcache.php

Copy apc.php into place

# yum install php_pecl_apc 

or

# apt-get install php-apc

# sudo pecl install apc

restart apache

#service httpd restart

run a load test

# ab -n 100 -c 5 http://drupal.org/   

(remember the trailing slash or it does not go)

Report on APC memory and cache useage

Apache/PHP tuning

Set memory big enough to cover 100% of normal traffic, but smaller than big admistration tasks.

Total possible apache memory usage

= (Apache MaxClients * memory_limit) + apc.shm_size

= (10 * 128M) + 192M

= 1280M + 192M

= 1472M

Compression

Emable mod_gzip and optionally enable mod_deflate

php-fpm users substitute pm.max_children for MaxClients

MySQL Tuning

cd /opt

wget http://www.day32.com/MySQL/tuning-primer.sh

chmod u+x tuning-primer.sh

wget http://mysqltuner.pl -O mysqltuner.pl

chmod u+x mysqltuner.pl

Download devel

View the node module's schema at http://drupalcode.org/project/drupal.git/blob/refs/heads/7.x:/modules/node/node.install .

How to show a summary of a table with all its fields and indexes:

XHProf

Installing the PECL extension

Installing the Drupal module

Tuning the Views Cache

Caching the query results can result in things being possibly out of date.

You can set the query results to never cache but still cache the rendered output. The rendered output is keyed on the contents of the result set itself.

More info on XHProf magic functions is available at http://msonnabaum.github.io/xhprof-presentation/#13 .

Redis

sudo yum install redis

sudo service redis enable

sudo service redis start

sudo pecl install redis

Make sure you install Redis module version version 7.x-2.6! Versions later than that have a regression that cause it not to work with $conf['page_cache_without_database'] = TRUE .

Enabling Redis caching

Edit /var/www/drupal-7.28/sites/default/settings.php and add the following at the bottom.

Varnish

Disable Drupal's internal page cache when using varnish:

 Adobe EM with Varnish.pdf

Varnish with multiple backends

Defining multiple backends

First of all, you will need to define the different backends that Varnish will rely on. In the example above, we have 3 VMs with each a private IPv4 in the 10.0.0.0/24 range.

At the beggining of your VCL, set your backends :

backend vm1 {
    .host = "10.0.0.11";
    .port = "80";
    .connect_timeout = 6000s;
    .first_byte_timeout = 6000s;
    .between_bytes_timeout = 6000s;
}
 
backend vm2 {
    .host = "10.0.0.12";
    .port = "80";
    .connect_timeout = 6000s;
    .first_byte_timeout = 6000s;
    .between_bytes_timeout = 6000s;
}
 
backend vm3 {
    .host = "10.0.0.13";
    .port = "80";
    .connect_timeout = 6000s;
    .first_byte_timeout = 6000s;
    .between_bytes_timeout = 6000s;
}

Using the appropriate backend

To define which backend (local HTTP server) should be used by Varnish to respond HTTP requests, we will set a few custom rules in the vcl_recv section of our VCL config file :

# Default backend is set to VM1
set req.backend = vm1;
 
if (req.http.host == "www.myhost2.com") {
    set req.backend = vm2;
}
 
if (req.http.host == "www.myhost3.com") {
    set req.backend = vm3;
}

Now restart Varnish and try to connect to one of the 3 hostnames : you should be forwarded to the appropriate backend.

Varnish to cache content from several servers - 3.0.5

Varnish to cache content from several servers - 4.0.1

Varnish Architect Article

You're Doing It Wrong

Other notes

Dries Keynote

History of camera

Log stash - Howard @Tizzin

Agent – inputs, filters, outputs (like unix pipes on steroids)

It is Ruby (Java) so it takes a minute to start, and will take 10 MG to run, there is a version written in C that is called lumberjack or Log forwarder

Log files, Logstash Shipper, Redis Queue, Logstash indexer

 Logstash Inputs (search for in google)

Output to a Redis Queue

Take from Redis Queue and put into ElasticSearch

ElasticSearch

               Download and run, has good documentation

Kibana connects to elasticsearch BUT Graylog2 is better than Kibana

Monitoring, sensu sensuapp.org way better than Nagios  See my post “Provisioning Sensu with Puppet”. 

Logstash includes support for grok

Github.com/tizzo/vagrant-logstash-gra

Logstalgia to show cool screen for monitoring

Instructors

Mark Sonnabaum - Acquia

Steve Merrill - Phase2

David Strauss - Pantheon

Dan Reif - BlackMesh - Director of Emerging Technologies