actly support SSL connections… directly.

Have no fear! If you are willing to give up a little bit of your own bandwidth, you can work around this problem. We are going to do this using a simple PHP script and if you want to get really fancy, Memcached to keep the redundant API calls to a minimum.

Let’s get started.

First, let’s make sure we understand why it is a problem to include a non-secure Google Charts API into your otherwise secure page. When your users are browsing to your page using an address starting with https://, their browser is going to negotiate an TLS/SSL connection which will encrypt all the data flowing between your client and the server. If any element on the page (image, external CSS or JavaScript, etc.) is not being called from a secured connection as well, then the browser will throw up a warning. Each browser is different, but most will alert the user with a broken lock and/or a popup message. The user can usually accept this warning and proceed at their own risk, but this is not acceptable for a production website or application.

The obvious answer to this problem is to make sure that all external elements on the page are loaded over a secure connection. So – how do we do that with Google Charts? Google does not allow you to just take a traditional API call and slap a “https://” in front of it in lieu of the expected http://. Your request will be denied.

Basic Implementation

Here’s what we do:

1. Create a simple PHP script that will fetch a chart from Google Charts API
2. Change your web page to call the new PHP “fetcher” instead of Google Charts directly.
3. Make sure the PHP script is being called by your page over your https:// connection

It is pretty much that simple.

Here is the before example:

<!-- HTML snippet -->
<img src='http://chart.apis.google.com/chart?cht=p&chd=s:Uf9a&chs=200x100&chl=January|February|March|April' alt='Pie Chart'/>

Here is the after example:

<!-- HTML snippet -->
<img src='https://mywebsite.com/gchart.php?api_url=http%3A%2F%2Fchart.apis.google.com%2Fchart%3Fcht%3Dp%26chd%3Ds%3AUf9a%26chs%3D200x100%26chl%3DJanuary%7CFebruary%7CMarch%7CApril' alt='Pie Chart'/>

Notice, the same API URI is used in the “after” example, but it is URL Encoded. This just makes it safer to pass along to the PHP script.

Here is the PHP script that makes this work:

<?php

$url = urldecode($_GET['api_url']);

$image_contents = file_get_contents($url);
echo $image_contents;
exit;

A quick note about this: you may be saying – where is your closing PHP tag!?! Well, PHP doesn’t require that you have a closing PHP tag, as the PHP interpreter will automatically close any open tags. I do this on just about every file that will be used as an include of some sorts – particularly in something like this where any extra, inadvertent white space at the end of the file can alter the output adversely and cause errors in the page.

Advanced Implementation w/ Caching

The script as it is above, is fine and dandy, but terribly inefficient. You see, every time the user requests the page, your script is going out and using your bandwidth to make the request to the Google Charts API instead of the client. You are becoming the proxy for all requests to and from Google Charts. This will not fly – especially if you are in a high volume environment.

So what is the solution? I chose Memcache to store a copy of the image contents for a certain period of time before having to re-query the Charts API and re-fetch the data again.

It looks something like this:

<?php

$url = urldecode($_GET['api_url']);
$cache_key = md5($url);
$item_cache_expire = 3600;
$mc_host = '127.0.0.1';
$mc_port = 11211;

# Connect to Memcache
$memcache = new Memcache;
$memcache->connect($mc_host, $mc_port) or die ("Could not connect");

if ( $get_result = $memcache->get($cache_key) ) {
    $image_contents = $get_result->chart_image;
}
else {
    $image_contents = file_get_contents($url);

    $tmp_object = new stdClass;
    $tmp_object->chart_image = $image_contents;

    $memcache->set($cache_key, $tmp_object, false, $item_cache_expire) or die ("Failed to save data at the server");

}
echo $image_contents;
exit;

Here’s what is happening.

<?php

$url = urldecode($_GET['api_url']);
$cache_key = md5($url);
$item_cache_expire = 3600;
$mc_host = '127.0.0.1';
$mc_port = 11211;

Setup some variables we’ll need in the script. Grab the api_url value from the GET array, create an MD5 hash of the request URL (we’ll use this as our memcache item key), set the expiration time of the cached item to 3600 seconds (a.k.a. 1 hour) and finally provide the connection information to your memecached server.

# Connect to Memcache
$memcache = new Memcache;
$memcache->connect($mc_host, $mc_port) or die ("Could not connect");

Instantiate a new Memcache object using the connection info provided above.

if ( $get_result = $memcache->get($cache_key) ) {
    $image_contents = $get_result->chart_image;
}
else {
    $image_contents = file_get_contents($url);

    $tmp_object = new stdClass;
    $tmp_object->chart_image = $image_contents;

    $memcache->set($cache_key, $tmp_object, false, $item_cache_expire) or die ("Failed to save data at the server");
}

Using the MD5 hash that we created above, check to see if there is a valid cached object on the memcache server for us to retrieve. If not, go out and grab a fresh version from Google Charts API and stick it into memcache. Either way, we end up with a variable $image_contents which contains the desired chart image.

echo $image_contents;
exit;

All that is left to do now is print out the image contents and exit the script. Again, I do not use a closing PHP tag on purpose to avoid extra whitespace.

All ye, all ye, now hear this…

This is a very basic example. It should probably be categorized more as a proof of concept. It does NOT, I repeat – DOES NOT – take into account a variety of security concerns such as SQL injection or CSRF (cross-site request forgeries), etc. Notice that I didn’t even validate the user input from the GET variable. Who does that!?! So, in other words, don’t just copy and paste and then come back and blame me for getting your site hacked. Be smart about employing something like this.

BTW, if you want to know more about how to make your PHP scripts more secure, go see Chris Shifflet’s excellent blog at http://shiflett.org/.

http://code.google.com/p/phpmultithreadeddaemon/

Here’s their example of how to implement it. Everything needed to run the daemon is in the single included file “class.MTDaemon.php” – cool!

<?php

error_reporting(E_ALL);

require_once(include/class.MTDaemon.php);

class MTTest extends MTDaemon {

public function getNext($slot)
{
$this->lock();
$num = $this->getVar("num");
if ($num == null) $num = 1;
if ($num > 100) {
$this->unlock();
return null;
}
$this->unlock();

$rand = rand(0, 5);
echo "Next for slot " . $slot . " : " . $rand . "\n";
if ($rand == 0) return null;
else return $rand;
}

public function run($next, $slot)
{
$rand = rand(3, 10);
$this->lock();
$num = $this->getVar("num");
$this->setVar("num", $this->getVar("num") + 1);
$this->unlock();
echo "## Iteration #" . number_format($num) . " in " . $rand . "sec" . "\n";

sleep($rand);
return 0;
}

}

$mttest = new MTTest(2);
$mttest->handle();

?>

An example in a MySQL query would be this.

<?php
// build our sql string.
$sql = "SELECT * FROM table WHERE field=%d";
$sqlf = sprintf( $sql, $somevalue );
$db->query($sqlf);
?>

As you can see, you can designate where the substitution will take place in the $sql string. That’s easy. But what happens if you need to use MySQL’s DATE_FORMAT() function? It requires that you pass in arguments to define its output (ie. Day as a word, day as a date, month as a number, etc).

<?php
// build our sql string.
$sql = "SELECT DATE_FORMAT( %b %M %d %Y, some_date_field ) as myDate FROM table WHERE field=%d";
$sqlf = sprintf( $sql, $somevalue );
$db->query($sqlf);
?>

This will fail. sprintf() will complain because you haven’t passed in enough arguments. It is expecting 5 values as part of the call, instead of just the one that you are trying to replace (in the SQL WHERE clause).

So what’s the solution? You have to “comment-out” the % that aren’t part of your sprintf() substitution. You can do this by putting another % in front of the ‘%’ symbols in the DATE_FORMAT() function. This deems them as a literal percent-sign instead of the start of another sprintf() “variable”.

<?php
// build our sql string.
$sql = "SELECT DATE_FORMAT( %%b %%M %%d %%Y, some_date_field ) as myDate FROM table WHERE field=%d";
$sqlf = sprintf( $sql, $somevalue );
$db->query($sqlf);

Hope that helps!

Well, instead of doing this:

<?php
$sql = "SELECT COUNT(UserID) FROM configuration WHERE UserID='SomeUser'";
$result = mysqli_query($db,$sql);
if ($result && mysqli_num_rows($result)>0) {
$aResult = mysqli_fetch_array($result);
$iRecordExists = ($aResult[0]>0?1:0);
}

if ($iRecordExists>0) {
//do an update
$sql = "UPDATE configuration SET someField='someValue' WHERE UserID='SomeUser'";
mysqli_query($db,$sql);
}
else {
//do an insert
$sql = "INSERT INTO configuration SET someField='someValue', UserID='SomeUser'";
mysqli_query($db,$sql);
}
?>

You could just do this:

<?php
//insert the user's configuration field - if the record already exists - update instead
$sql = "INSERT INTO configuration SET UserID='SomeUser', someField='someValue' ON DUPLICATE KEY UPDATE someField='someValue' ";
mysqli_query($db,$sql);
?>

Simply put, the query will attempt to insert the configuration record first. If it finds that the specified UserID already has a configuration record in the table, it will simply update the existing record according to the values you include after “ON DUPLICATE KEY UPDATE”. You can include more than one field to update as well.

[Update: As Paul questioned in the comment below, the WHERE clause is not correct (in my original post). The trick is, you have to include the primary key as part of the insert statement - such as UserID in the example above.]

Anyway, I kept getting an error when I tried to add the apostrophe to my character classes in my regex. It gave me a strange error referencing REG_ERANGE. After some googling, I came across this blog post which led me to the answer. The problem is related to the placement of the dash (”-”) character in the regex.

Example 1:

if (ereg("[^a-zA-Z0-9_-.]", $userid)) {
    echo 'bad';
}
else {
    echo 'good';
}

The problem? The dash, or hyphen, being before the period. It thinks it’s a range, like you see in a-z. This may not be a bug, per se, but it’s certainly not smart enough for me.

The solution? Simply put the dash at the end of the regex.

Example 2:

if (ereg("[^a-zA-Z0-9_.-]", $userid)) {
    echo 'bad';
}
else {
    echo 'good';
}

It does the following:

1. Creates a backup of the selected db using mysqldump
2. Generates an MD5 checksum of the backup file (written to a separate file)
3. Attempts to restore the dumped file into a dummy test database
4. If errors are encountered, it grabs the error and sends an email to the designated address
5. If no errors are encountered, wraps the .sql and .sql.md5 in a timestamped, gzipped, tarball – then deletes the originals

Download the file here

#!/bin/sh
#######################################################
# LICENSE:
# (c) 2007 Brian Bell (GNU LGPL V2.1) You may
# view the full copyright text at:
# http://www.opensource.org/licenses/lgpl-license.html
#
# DESCRIPTION:
# A simple BASH script to do automate MySQL database
# backup; includes testing and MD5 hash creation.
# Emails designated address on failure.
#######################################################

## CONFIGURATION VARS
MYSQL_NAME=
MYSQL_HOST=
MYSQL_USER=
MYSQL_PASS=
MYSQL_TESTDB=
BACKUP_PATH=/path/to/backup/dir # No trailing slash
MAIL_SUBJECT=”TESTING MySQL Backup Error”
MAIL_TO=”monitor@yourdomain.com”

#######################################################
## We need to create a unique timestamp for use on the filename
TIMESTAMP=`date +%Y_%m_%d`

## Generate the base part of the filename to use in backing up
BACKUP_FILE_BASE=”${MYSQL_NAME}_${TIMESTAMP}”

echo “Backing up $MYSQL_NAME…”
/usr/bin/mysqldump –opt -c -e -Q -h $MYSQL_HOST -u $MYSQL_USER –password=$MYSQL_PASS \
–add-drop-table $MYSQL_NAME > $BACKUP_PATH/$BACKUP_FILE_BASE.sql

## MD5 the backup file
/usr/bin/md5sum -t $BACKUP_PATH/$BACKUP_FILE_BASE.sql > $BACKUP_PATH/$BACKUP_FILE_BASE.sql.md5

## Try to import the backup sql into a test db
MYSQL_RESULT=`/usr/bin/mysql -h ${MYSQL_HOST} -u ${MYSQL_USER} –password=${MYSQL_PASS} ${MYSQL_TESTDB} < \
${BACKUP_PATH}/${BACKUP_FILE_BASE}.sql >

${BACKUP_PATH}/mysql_test.log`

if [[ “$MYSQL_RESULT” =~ “ERROR” ]]
then
echo “The following error was encountered at `date` ” > ${BACKUP_PATH}/error_email.log
echo “” >> ${BACKUP_PATH}/error_email.log
echo “#####################################################################” >> ${BACKUP_PATH}/error_email.log
echo $MYSQL_RESULT >> ${BACKUP_PATH}/error_email.log

echo “SENDING ERROR EMAIL TO: ${MAIL_TO}”
/bin/mail -s “$MAIL_SUBJECT” “$MAIL_TO” < ${BACKUP_PATH}/error_email.log
else
tar czpf $BACKUP_PATH/$TIMESTAMP_$BACKUP_FILE_BASE.sql.tar.gz $BACKUP_PATH/$BACKUP_FILE_BASE.sql* –remove-files
fi