On 03/12/2010 04:15 PM, Ulf Wendel wrote: For a transition period there's likely to be more work and the number of test failures is likely to go up. That is nothing to really worry about as long as you manage to educate users that it is not a quality defect of PHP as such but a temporary matter of an different and improved testing approach.
I'd say it's more work since some tests do not exists in some branch. As we might soon have quite closely related 3 branches, such differences are quite small. As trunk would have exactly same tests as PHP_5_3 has, the real differences would be only with PHP_5_2. So if we want such common tests, I think the time to do it is about now. Let's assume the worst case status quo of different versions of the same test in each branch. The one in an old branch has been written with the versions of the external library in mind that has been current when the branch was current. The one in a current branch has been written against the latest version of the external library. I continue to assume that users of the old PHP branch run rather old systems whereas users of a current PHP branch run on current systems. Therefore only few people may have tried to run the old test against the latest library or the other way around. It is just a feeling that your proposal will implicitly cause some combinations of PHP version x test x external library version to be tested that have not been checked before.
So you actually fear that you'd have to fix some things in older branches too? Well, I think (hope) such cases are rare and such problems actually already exist too. Of course we can always stay with the status quo, but is that really good for PHP in general? Your "one test for all" proposal is likely to unveil some "different versions of the same test" and "external library is not BC" issues. That's good. But its gonna be work...
IMO, such work is never a waste. --Jani
