I joined the HHVM team right before Christmas for a hackamonth (or, in my case, a hack-a-two-months).
Getting My Feet Wet
To prepare for what was to be my big project, I rewrote the
ini parser to better match Zend. This was released in HHVM 2.4.0. Interestingly, the parser was going to be shipped with HHVM 2.3.0; but when we tested the RC (release candidate), we noticed that we had a bunch of
ini files at Facebook that the parser failed upon. So, I had to fix those files before we could release the new one.
After warming up with the parser, I was ready to start my big project: implement
MySQLi. This has been a long requested feature for HHVM. And, this extension is required to help meet our compatibility goals.
MySQLi is not a trivial extension. As a PHP developer, I am so glad that SaraG has created HNI (HHVM Native Interface), which makes it possible to write extensions in PHP directly instead of writing them in C++. Also, SaraG created a tool called docskel.php, which uses the raw XML that forms the php.net documentation to stub out the appropriate HNI-based signatures for extensions. While the documentation can be wrong sometimes when it comes to return types and parameter types, the stub generation saves a ton of time. After the stub generation, essentially the only work left is to implement all the required functions.
For many of the MySQLi methods, it was simply a matter of essentially passing through calls to the older MySQL equivalent implementations. For example,
mysqli::real_escape_string() just calls the implementation for
MySQLi has both a procedural and an object-oriented interface. However, because we can write part of the extension in PHP, I could implement almost all of the procedural interface in PHP by just having it call the object-oriented interface. For example, the procedurual function
mysqli_real_escape_string() just calls the object-oriented function
To test the correctness of my implementation, I used the Zend test suite. There are about 300 tests in this test suite. However, these tests are not meant to be run in parallel. Many tests uses the same database table which, when running in parallel, will randomly make the test fail because some other test touched the same table at the same time. Running the tests in serial wasn’t a great option because that would make development of
MySQLi a lot slower. So, instead, I updated our Zend import script to fix the tests to use different tables, thus avoiding the conflicts when running in parallel.
Another testing issue came with what we call
ZendParamMode is our flag to indicate the calling convention that Zend uses. In short, when using
ZendParamMode, if a function is called with a parameter of the wrong type, a warning is logged and the function returns
null. The problem is that functions in
HNI that are implemented in PHP do not currently support
ZendParamMode. So even if the function was doing everything correct, except how it handled
ZendParamMode, the test would fail. This made it very hard to see what was implemented correctly and what was incorrect or missing. But, since making the function actually have correct behavior is more important than supporting
ZendParamMode, I just changed the import script to remove most of the cases where
ZendParamMode was tested.
After fixing the above issues, I was able to perform the normal “where does the test fail and fix it” methodology. This often included reading Zend’s
MySQLi source code to determine what the right behavior should be.
The most difficult part of the
MySQLi implementation was
mysqli_stmt::bind_result(). These methods take a variable number of arguments that are passed by reference. And, at the time,
HNI did not support either. First SaraG added support for variable number of arguments, and then I added support for variables as references.
mysqli_multi_query() (and its friends
mysqli_next_result()) were initially going to be a bit tricky since they were not part of Zend. However, just the opposite occurred. The reason for that is that HHVM already had implementations for
mysql_next_result(), so all I had to do was essentially do a pass through call to those functions.
We currently pass 182 of the Zend
MySQLi tests. We fail 114. However, many of those 114 tests are failures are what I call “technicality” failures. For example, when we
double, we print more digits which makes our output different from Zend.
The biggest missing pieces when it comes to passing all Zend tests are:
- Bugs that comes from the interaction between
php.ini. Reading default values from
php.iniis currently missing (but is being developed).
- Sometimes we output different error messages which confuses the test runner when it compares output.
- Zend has two different
MySQLdrivers. One is the normal
MySQLC API and one is their own
MySQLNative Driver. Because HHVM uses the
MySQLC API we, don’t support the functions that require the
MySQLNative Driver. We may be able to work around this issue for some functions, but, for now, we don’t support any of the functions that require
- What happens when something goes wrong may be different between HHVM and Zend.
ZendParamModefor PHP functions in HNI.
- Make more of the 114 failing Zend tests pass.
- The plan is to release
MySQLiin HHVM 2.5.0. Please test the implementation as much as possible before the 2.5.0 release (you can use a nightly), so we can continue to fix bugs.