[cairo] Time for a release ?

[Stuart Axon]

So to summarize the CI useful for regression is blocking on two kinds of 
test failure:

[1] Locking issues.

[2] Rendering issues.

Some tests seem to pass though, can we set those to expected-to-pass, 
and the rest expected-to-fail ?

It would be good to have an epic issue in gitlab for "Improve the CI 
pass rate" to put these issues under.


[1] Locking issues

I ran the tests in a Windows VM with Meson and quite a few timed out, it 
seems like quite a few will be the ones with locking issues.

https://gist.github.com/stuaxo/1d125dc83888a28f60c684718619a112

There were a couple of tests that popped up dialog boxes under Windows 
when they failed, which seems not idea for CI.

I'll see if I can do the same under Linux and possibly MacOS to see if 
the same ones time out.


[2]  Rendering issues:

It sounds like there are different sources for these, so maybe it's 
worth having tickets for the different kinds of failure, once a baseline 
is established.

Maybe we could make sanity checks for the different failure types (or 
designate existing tests for these) ?


Pixman:  As mentioned elsewhere, changes that affect rendering here will 
mean ref images need updating.

Ghostscript:  We could render some known postscript through this and 
check the output.


Fonts and operating systems:

Do we expect output on differing operating systems to be the same?  If 
not, maybe per-OS reference images is could work (e.g. 
Linux/MacOS/Windows would be expected to be similar, but whichever OS 
you are running on should be an exact match).


On 17/11/2020 18:38, Dave Yeo wrote:
> On 11/17/20 06:10 AM, Uli Schlachter wrote:
>>> [...] however I cannot remember the test-suite passing at
>>> any time,
>> Yup, I never saw the test suite passing either.
>
> My understanding is that the testsuite requires exactly the right 
> versions of the various fonts, ghostscript and likely other things and 
> likely only ever passed on the original developers setup.
> In the real world, I tried using the latest (well recent) Cairo with 
> Firefox and I see some fonts where the beginning of the font is faint 
> or non-existing. An F where the vertical stroke is missing for 
> example, only at the beginning of lines and mostly where Firefox fakes 
> system menus. This might be partially caused by the system font but 
> still seems to be a regression.
> Dave