With the recent discussion around MemSQL vs MySQL vs NoSQL solutions, I've become really curious if their is a 'fair' way to do some sort of comparative testing or benchmarks.
I'm thinking of something like Tim Bray's WideFinder project[1], something practical and straightforward that lots of different people with expertise in different data stores could approach.
1 - http://www.tbray.org/ongoing/When/200x/2007/09/20/Wide-Finder