I agree. The final measure is always whether it looks acceptable on the target machine. I guess the main reason I am interested in this is to test changes to performance-dependent projects like tween engines internally and not so much to compare different ones or get detailed numbers. It’s difficult to tell whether your optimization changes are having any effect sometimes.

So far great feedback!

Also, I wasn’t trying to pick on Grant with that statement! ;-D

Take tweening – in most cases, performance only has to be at an acceptable level. Most people do not utilize anywhere near 1500 simultaneous tweens in their projects. Once a tweening engine exceeds acceptable speed (which most engines beyond Adobe’s Tween do), decisions should be based on whether the capabilities and API of a library fit your needs. Unfortunately, many developers (particularly junior developers) tend to latch on to measurable qualities, because it provides a solid metric to justify their choice. IMHO, this mentality is also what leads to the abuse of design patterns and frameworks.

The main reason I put that very vague metric on the gTween page was to indicate an acceptable level of performance, not to provide a direct benchmark against other engines.

Benchmarking is really something that needs to be performed by developers on performance critical code on an as-needed basis. The best performance benchmark will always be whether it runs acceptably on the minimum target system.