Locking Down for Performance and Parity
The HHVM team, along with our wonderful open source contributors, is constantly trying to improve the runtime. We know, however, that there is work to do to run the world’s PHP code reliably and consistently.
Every 6 months or so, the HHVM team goes into lockdown. Lockdown is a Facebook tradition where a team picks a few concentrated areas and pushes hard on them. The team is huddled together. No meetings. Specific task focused. Shaving is discouraged.
The HHVM team’s latest Lockdown started on Monday. It is a 3-week effort focused entirely on two things: performance and parity.
The first graph is our performance graph. The second graph is our parity graph, and the percentages are unit tests passing per framework. Simply put, by the end of the three weeks, we want to have had each graph move up and to the right. (Note: The dip you see in the parity graph is our unit test script misbehaving. The script is being continuously modified to make our testing better and more valuable).
The stickies below are an example of the individual tasks to get us there. We have prioritized them by impact (e.g., number of unit tests that will pass upon completion of the task) and length (how many days the task will take to complete). If these stickies are moved to the “Done” column, the graphs should respond favorably.
The HHVM team really cares about maintaining a high quality runtime for PHP. At the end of the three weeks, the no shaving mantra having fully taken hold, we are hoping you see a much better HHVM in terms of being able to accurately run a broader array of PHP code, very fast. Stay tuned for the results of this lockdown.
Comments
- Bob Brunius: Is mysqli supported yet?
- puentesdiaz: Hi there! Amazing work! I hope that we can install this tool, very soon over our servers ....
- Kobbe: Great work! will typo3 and the typo3 flow framework be supported?
- Thomas Heidrich: I'm looking forward to see HHVM in top row of the Benchmarks Game</a>. :-) You guys give PHP a future. Keep up the good work.
- qianzhihe: +1
- RouteXL: Movember lockdowns are the best. Have fun! PS: No Sara mantra too?</em>
- Máximo Cuadros: Any idea when will be ready mongodb ext? Regards
- Paul Tarjan: Their unit tests were too hard to run to add to our charts, but IIRC it worked well already. Did you try it?
- Paul Tarjan: What should we do to get added there?
- Nazar Mokrynskyi: How difficult is to add __call() and __callStatic() functions?
I'm using such syntax:
h::{'section article.some-class[data-article=yes]'}()</code> And I need support of this in order to work with HHVM
- Jim: Have you tried alternate compilers at all? The Intel compiler, their threading libraries and their ipp libraries for zlib and openssl can speed up certain tasks dramatically.
- Louie: laravel, twig, drupal for the win, hope to see this three working next month.. ^__^
- Paul Tarjan: That should work already.
- Paul Tarjan: We've experimented with a few. GCC 4.8 is currently the fastest for us.
- Radu: Actually, if I were you, I would focus strictly on the ability of HHVM to run (almosy) any piece of PHP code out there. Maybe on easier installation process too. Performance should be a lower priority. What good is it to have extreme performance if the most popular frameworks crash on it ? Like owning a Ferrari and not having gas money....
- Martin: I didn't try it yet with flow is one of my next spare time projects to test typo3 flow and hhvm. But why are the unit test so difficult?
- Bostjan: Are there any plans for Kohana support?
- Samuel Demirdjian: Another +1 for Kohana support here. Kohana stands out with his native HMVC and cascading file system. The community and core developers are very disciplined and I am sure we can work this out mutually with minimal overhead from your side, pleaaase :-) Thank you for efforts and for making PHP better.
- jcoetz: Wow, next time please actually read the article...
- Nathan Malcolm: I do hope you'll be posting up a follow up article on how things went -- performance changes, new features, improved parity, etc. :)
- Josh: Would love a follow up as well!
- Joel Marcey: Stay tuned. A post-lockdown post will be coming. :)
- Josh: Hooray, thanks Joel. HHVM over the past few weeks has been exciting to see the spate of commits on Github!
- Ivan Palamarchuk: thank you, guys!)
- Marko: What about WordPress? It's not in the list, but I remember a graph which said "9x% passing". I would love to finally switch my blog to HHVM. I haven't seen any crashes so far, but I would sleep better with 100% unit tests passing.
- Nathan Malcolm: Brilliant. Looking forward to it. :)
- Joel Marcey: Marko, we have been trying to automate our unit testing to run all the frameworks simultaneously. Wordpress requires some database setup and other things that make this automation a bit difficult right now. The 21 we have listed were a bit easier to automate the unit testing. That said, Wordpress does work very well with HHVM. We are working on a blog post that discusses some results with Wordpress and FastCGI.
- Nazar Mokrynskyi: But it does not working. I get error: HipHop Fatal error: syntax error, unexpected '{' in /web/cscms.org/www/core/functions.php on line 710 Mentioned line in repository: https://github.com/nazar-pc/CleverStyle-CMS/blob/7b70604db2968fb642efc31efb3a874e831279f5/core/functions.php#L710
- Nazar Mokrynskyi: I've created an issue on GitHub: https://github.com/facebook/hhvm/issues/1292
- RouteXL: Looking forward to the resutts. Not only of the no shaving mantra, but also on the runtime. Will you be posting the post-lockdown post anytime soon?
- Paul Tarjan: Yes, as soon as we cut the new release (should be next week).
- Vassil Velichkov: Fingers crossed for a post-lockdown release and blog update before Christmas 2013 :-) Even a short blog update would be a gift :-)
- Joel Marcey: We are preparing a post-lockdown post that should indeed be released before Christmas.
- We are the 98.5% (and the 16%) « HipHop Virtual Machine: […] November 4th, the HHVM team went on a 3-week performance and parity lockdown. The lockdown officially ended on November 22th. Overall, this lockdown was a qualified success. […]
- Maxx: Really an amazing work guys. The whole PHP community is looking forward for this boon. Many of us long time PHP users want HHVM to push the current code performance to several order of magnitude higher... Keep it up!
- HHVM: The Next Six Months « HipHop Virtual Machine: […] Lockdown improvements […]
- Tomasz Grzemski: We have some experience in starting TYPO3 Unit Tests on HHVM. Check out post blog: Running TYPO3 CMS on HHVM</a>
- Tymoteusz Motylewski: For current progress on having TYPO3 CMS unit tests run on HHVM see http://blog.macopedia.co/en/post/running-typo3-cms-on-hhvm-part2 . Now 95.57% are green. I will work on pusing a pull request for HHVM to include TYPO3 in parity testing.
- Facebook’s HHVM Speeds Up Box | CloudBackupCheap.com: […] they have not interacted well with HHVM. But just under a year ago, the HHVM team’s growing focus on parity with the standard PHP runtime caught our attention. We decided to seriously re-evaluate HHVM, […]