Travel Plans

After what is probably the longest period of being at home without flying anywhere (since November!) in the last 3 years, I am actually quite excited to be flying again.

After a quick visit to Zend’s Cupertino office, I will be attending two conferences in may: I’ll be visiting the last couple of days (May 22nd - 23rd) of php|tek in Chicago - in which I will not be presenting, but will probably hang around with the Zend guys and go to some lectures.

After spending the weekend with friends in New York City, I’m flying to Germany to attend DLW Europe - a first-year conference dedicated to dynamic languages (PHP, Ruby, Python, Perl etc.). There’s quite a lot of known PHPers attending and speaking, and it looks like it’s going to be a blast. Personally I will be giving a talk I call “Non-framework Zend Framework Components” (working title) - which will showcase some of the more useful components of Zend Framework for those who do not want to base their entire application on it - but rather just need to preform tasks and want to reuse the high-quality components of ZF in order to do that.

So it looks like quite a road trip: Tel Aviv -> San Francisco -> Chicago -> New York -> Karlsruhe -> Tel Aviv - by the end of it I might be able to buy a MacBook Pro using frequent flyer miles only ;)

If you’re in either conferences - come over and say hi!

Playing with CouchDB View Collations

I've been playing with CouchDB quite a lot lately. While it's still far from feature-complete, I think it's a cool project. Generally the concept of object databases is a new thing to me, growing up in the web where for most people "database" == "Relational Database".

Since CouchDB related documentation is still scarce, I decided to share a cool thing I managed to pull around today with CouchDB's view API.

For the purpose of playing with CouchDB, I decided to write a PHP-implemented CouchDB interface (yes, I know there are exiting ones) I currently dub "Sopha" and a Wiki using CouchDB and Sopha as backend (if I ever get those to a stage where they would actually work, I will probably release them as open source).

One of the design goals of my Wiki, was that previous revisions of each page are kept and can be accessed. While CouchDB keeps track of object revisions, access to old revisions is not guarantees as those eventually get deleted - So I had to implement my own revision tracking of pages. The design I went for was simple: each revision in it's own document in the DB. They all have the same "title" field - but an incrementing revision number, starting from 1 (for new pages) and going up.

This design poses one major problem: how do you easily access the single most recent revision?

I started by creating 3 different view functions for accessing wiki pages: one to access the most recent revision of a page by it's title, one to access a specific revision of a page by title and revision number, and another one - to fetch all the revisions of a specific page by it's title.

This was a mistake, and it resulted from my years of thinking relational (keys, indexes, SQL, etc.). As I worked my way through, I started understanding the power and magic of CouchDB's views and view collations. With proper collation use, it turns out it only takes one view function to rule them all:

CODE:
  1. function(doc) {
  2.   if (doc.doctype == "WikiPage") {
  3.     map([doc.title, doc.revision], doc);
  4.   }
  5. }

Using this view function, I could do the following ("titlerev" is the name of my view function, URLs are decoded for readability):

Access the specific revision 123 of a specific page titled "My Page":
GET /database/_view/wikipage/titlerev?key=["My Page", 123]

Access all the revisions of "My Page":
GET /database/_view/wikipage/titlerev?startkey=["My Page"]&endkey=["My Page", "Z"]
This will work because CouchDB orders the results according to their key - and has strictly defined ordering rules - so ["My Page"] will always come before ["My Page", 1] and ["My Page", "Z"] will always be after ["My Page", n] as long as n is a number.

Access the latest revision of "My Page":
GET /database/_view/wikipage/titlerev?startkey=["My Page", "Z"]&endkey=["My Page", 0]&decending=true&count=1
This works as well - because records are selected in reverse order from ["My Page", "Z"] to ["My Page", 0] - but only the first one is returned - so it will always be the one with highest revision number.

How cool is that?

BTW there is a good article which also explains CouchDB view collations at cmlenz's (one of the contributors to CouchDB) blog.

Security By Obscurity - “Not to Stand Up”

I am giving a security lecture at the local PHP users group tomorrow, and one of the topics is, ahm, security by obscurity. In fact, it's not really a topic - I just mention it and say that it's not really an approach to security, and should generally only be used as an extra measure and should not to be relied on.

This got me thinking about one of my old-time favorite Monty Python sketches:

The first rule of not being seen: not to stand up!

New year, new position

I spent the last 2.5 years working for Zend's Global Services department, first as a technical support engineer and later as a technical consultant. I've learned allot and met lots of very smart people - but for some time now I have a feeling it's time to move on.

Apparently, some people at Zend had the same feeling - don't worry, I wasn't fired. I didn't quit either. Starting from January 2008, I am now a part of Zend's Product Management group.

My new and awesome title is "Senior PHP Specialist" which kind of means nothing (and that's what so good about it!). My actual roles are varied and flexible and involve duties such as competitive analysis, benchmarking, prototyping, meeting with customers and PHP community folks, and generally getting more customer and community feedback and touch into Zend's product line. I'll be working with all products - mostly Zend Platform, Zend Studio and Zend Core, and will also maintain my involvement in Zend Framework, and perhaps other projects as they come.

I have to say this has the potential of a dream job: it's hands-on, and yet very dynamic. It's community related and touches commercial products. It means working closely with services people, R&D, and customers (and PHP users who are not our customers!). It involves studying emerging technologies, programming, testing software, writing and reading papers and generally keeping a finger on the pulse of the web.

Oh, and did I say my new direct manager is Zeev Suraski?

So good luck and happy 2008!

Reducing your bandwidth for benchmarking purposes

In the last few days I've been busy doing allot of web application performance benchmarks. For some of these benchmarks, it was important to simulate a real end-user's average effective bandwidth (real-world download / upload speed from the end user's perspective).

Since we used a fast, clean LAN for our benchmarks (you can't get consistent numbers otherwise), we had to find a way to somehow limit the bandwidth of our LAN so that it would resemble "real Internet" throughputs. Since it took some research and reading to find the best way to do that, I've decided to write a quick HOWTO to describe what we've done.

(more...)

Creating magic_quotes_gpc portable apps with Zend Framework

I've been working on a small ZF based application for work in the last few days, and it was running quite well on my local machine. But when time came to deploy - alas, we discovered that slashes are magically added before single quotes to all data. The dreaded magic_quotes_gpc "feature" was apparently on on that server - so instead of turning it off, I decided to make my application more portable by making it work regardless of magic_quotes_gpc's status.

Since my application was based on Zend Framework (as most of the stuff I write for work these days), I decided to do it in a reusable "Zend Way". So I created a Zend Controller Plugin to reverse the effects of magic_quotes_gpc:

PHP:
  1. <?php
  2.  
  3. /**
  4. * A Zend Controller Plugin dedicated to undoing the damage of magic_quotes_gpc
  5. * in systems where it is on.
  6. *
  7. * @author  Shahar Evron
  8. * @version $Id$
  9. */
  10.  
  11. require_once 'Zend/Controller/Plugin/Abstract.php';
  12.  
  13. class MyApp_StripMagicQuotesPlugin extends Zend_Controller_Plugin_Abstract
  14. {
  15.     /**
  16.      * Called before the action loop is started. Will internally strip all
  17.      * slashes off $request parameters
  18.      *
  19.      * @param Zend_Controller_Request_Abstract $request
  20.      */
  21.     public function dispatchLoopStartup(Zend_Controller_Request_Abstract $request)
  22.     {
  23.         $params = $request->getParams();
  24.         array_walk_recursive($params, array($this, 'stripSlashes'));
  25.         $request->setParams($params);
  26.     }
  27.  
  28.     /**
  29.      * Strip the slashes off an item in the Params array
  30.      *
  31.      * @param string $value
  32.      * @param string $key
  33.      */
  34.     protected function stripSlashes(&$value, $key)
  35.     {
  36.         $value = stripslashes($value);
  37.     }
  38. }

You place this plugin under your application library directory. Then, in your index.php bootstrap file, check if magic_quotes_gpc is enabled, at load and register the plugin:

PHP:
  1. /**
  2. * Setup controller
  3. */
  4. $controller = Zend_Controller_Front::getInstance()
  5.     ->setControllerDirectory('../application/controllers');
  6.  
  7. /**
  8. * Load the magic_quotes_gpc undoing plugin if needed
  9. */
  10.     require_once 'MyApp/StripMagicQuotesPlugin.php';
  11.     $controller->registerPlugin(new MyApp_StripMagicQuotesPlugin());
  12. }
  13.  
  14. // run!
  15. $controller->dispatch();

Now, this is only loaded and executed in systems where magic_quotes_gpc is enabled. It probably has quite allot of overhead - so a best solution in production environments would be to turn magic_quotes_gpc off (consider the fact that PHP works hard to escape the input, and then works hard to undo it... what a waste!), but if you want to produce a portable application that can be installed anywhere (including on shared hosting environments), this is a good solution.

BTW I'm not really sure if this handles cookies - so there might be a more "correct" way to do this. If you have any suggestions, I'd be happy to hear them!

XSRF and session.referer_check

I was giving Zend's on-line "security for developers" course to some customers last week, and as usual Cross-Site Request Forgeries (XSRF) was one of the hardest topics for me to explain and for other people to understand. This made me spend some time thinking about XSRF again, coming up with code demos and such to make it more clear.

Incidentally, I came across a php.ini directive today, which I've never noticed before: "session.referer_check". According to the docs, you can set the value of this directive to a substring of the domain name you're using. PHP will then check every HTTP_REFERER, if set, to contain this substring. If the referring address exists but does not contain the value of session.referer_check, the embedded session ID will be marked as invalid.

I didn't test this - but if it works as it's supposed to, it might be a good quick-and-dirty defense against XSRF - because it will not allow you to continue your session if the requested URL was referred from another domain (through an IMG tag, link, form submission and such).

There are several reasons why I think it's quick-and-dirty, to name a few:

  • It will not help you if the XSRF was launched from your own domain - for example through exploiting an XSS vulnerability in your site
  • Not all requests referred to your site from other domains are malicious - for example, I might want to have a "Digg This" or "Add to Facebook Profile" button on my site - if you click those, you'll be referred to Digg / Facebook from my site. In most cases you'd want your sessions to stay active on those sites, handling the request properly, and not treating it as XSRF
  • As the experience of magic_quotes_gpc taught us, trying to solve code-level problems in the engine level will probably end up in a half-baked solution, providing mostly the false sense of security

On the other hand, it's nice to know PHP has that option (and that it's turned off by default) - if you've just found out your site is full of XSRF vulnerabilities, and you know it will take you a while to fix them, you might just want to turn in on as a temporary solution.

Be careful though - it might break some of your functionalities. Also, there's a popular Hebrew saying that "nothing is more permanent than temporary solutions" (my own translation here..) - so FIX YOUR CODE if you can.

PHP Israel: First meeting

Yesterday we had the first meeting of the newly founded Israeli group of PHP users, or "Kvutzat Mishtamshey PHP Beysra'el" as one might say.

The meeting was held at Zend offices, and there was a pretty good turnover - ~20 non-Zenders showed up with ~10 more Zenders joining in. I gave a presentation (slides are in Hebrew but you can get them here if you read that funny language) about the past, present and future of PHP. There was also a trivia quiz (yes, I stole some ideas from a PHP Norge meeting I attended some months back) and we gave a book and a couple of T-Shirts (and a PHP stress ball) to the winners.

Boaz took some pretty good pictures as well.

It was lots of fun and good socializing which was exactly what I hoped for. The general idea is to have a meeting once a month or so - hopefully, next time I will be sitting and asking questions and not presenting.

Thanks everyone for coming, and if you happen to be in Israel, and do PHP - you should come as well!

That’s why I got so many business cards printed!

A few days ago my work Thinkpad T43's screen started flickering (not flickring, actually flickering). I immediately dismissed it as some kind of broke-my-Gentoo thing I always do, but after rebooting to Windows (yeah, it's still there - IT demanded I keep a Windows partition), I realized it's a hardware problem - something is fucked up with the backlighting or something. It got worse until a couple of days ago it started going blank for hours, and only coming back randomly for 5 minutes every time.

Now, this would all be fine if I wouldn't be in the middle of a training engagement in India - and a week from returning home. Projecting still worked - so I could go on training, but I couldn't do anything except for that - plus it doesn't really feel nice that during breaks all your students can still watch your desktop ;)

Anyway while not giving up to hardware, I found out that if I apply pressure at certain points on my screen, it goes on - that made me spend hours massaging my screen trying to get it working, but whenever I got my fingers off it it went dark again.

I almost thought of giving up and started thinking about finding an IBM service center or something (or is it Lenovo?), but that soon realized as impractical. Since doing customer engagements for Zend away from home always feels a bit like a military operation (that's an Israeli thing: Israeli men tend to compare hard / exciting / surreal situations in their lives to military related experiences - even if they don't admit it out loud...), I suddenly remembered something one of my officers used to say whenever we complained our equipment or supply is crappy: "This is all we have, and that's what we're going to win with!". All motivated, I started MacGyvering, trying to get my screen working back with a toothpick and some chicken curry (that's mostly what I have here).

Finally, I came up with this solution:
How I fixed my T43's screen

In case you're wondering, that's one of my Zend business cards I always carry and hardly use :)

Off topic: yes, I'm in some kind of photo-taking craze. Perhaps I was bit by a Japanese vampire or something.

Canon EOS Rebel / 400D, libgphoto2 and Gentoo

I had some troubles yesterday importing images from my new Canon EOS Rebel XTi to my Gentoo - I installed UFRaw and gtKam and was trying to import RAW images into GIMP. For some reason gphoto2 wasn't seeing my camera (although it was listed with lsusb) and so I couldn't import photos from it using any application.

After several attempts (and very little information on Google), I gave up and went to sleep. Today I tried again and a solution came surprisingly fast: The problem was probably that I didn't compile libgphoto with ptp2 support. What I did was:

- Add 'CAMERA="canon ptp2"' to /etc/make.conf
- Emerge libgphoto2, gphoto2 and hal again (I'm quite sure libgphoto2 was enough).

Now everything works and I'm off to play with some RAW images with GIMP.

Next Page »