<html> <head> <meta http-equiv="content-type" content="text/html; charset=UTF-8"> </head> <body> <div class="moz-cite-prefix">Am 20.11.2020 um 00:14 schrieb Bryce Harrington: </div> <blockquote type="cite" cite="mid:20201119231442.GK2602109@bryceharrington.org"> <pre class="moz-quote-pre" wrap="">On Thu, Nov 19, 2020 at 10:23:33PM +0000, Stuart Axon wrote: </pre> <blockquote type="cite"> <pre class="moz-quote-pre" wrap="">[1] Locking issues I ran the tests in a Windows VM with Meson and quite a few timed out, it seems like quite a few will be the ones with locking issues. <a class="moz-txt-link-freetext" href="https://gist.github.com/stuaxo/1d125dc83888a28f60c684718619a112">https://gist.github.com/stuaxo/1d125dc83888a28f60c684718619a112</a> There were a couple of tests that popped up dialog boxes under Windows when they failed, which seems not idea for CI. I'll see if I can do the same under Linux and possibly MacOS to see if the same ones time out. [2] Rendering issues: It sounds like there are different sources for these, so maybe it's worth having tickets for the different kinds of failure, once a baseline is established. Maybe we could make sanity checks for the different failure types (or designate existing tests for these) ? Pixman: As mentioned elsewhere, changes that affect rendering here will mean ref images need updating. </pre> </blockquote> For rendering issues in the image backend, I tended to find that a common root cause would often be a changed assumption about how drawing operations should behave. For example, algorithm changes in pixman can result in a pixel to flip on or off, or to be set to a slightly different shade of grey. Legitimate performance or quality improvements to blending or dithering algorithms can cause these kinds of changes too. Typically for those cases the rendering behavior is kind of a judgment call, so not much to do except redo the reference images to match the new behavior. </blockquote> Latest pixman version also has a regression in downscaling <a class="moz-txt-link-freetext" href="https://gitlab.freedesktop.org/pixman/pixman/-/issues/43">https://gitlab.freedesktop.org/pixman/pixman/-/issues/43</a> <blockquote type="cite" cite="mid:20201119231442.GK2602109@bryceharrington.org"> <blockquote type="cite"> <pre class="moz-quote-pre" wrap="">Fonts and operating systems: Do we expect output on differing operating systems to be the same? If not, maybe per-OS reference images is could work (e.g. Linux/MacOS/Windows would be expected to be similar, but whichever OS you are running on should be an exact match). </pre> </blockquote> <pre class="moz-quote-pre" wrap="">A foundational principle for Cairo is that rendering output should be reliably identical not only cross-platform but also cross-device (i.e. what's on your monitor matches what's printed on paper, and rendering to PNG, PDF, or SVG should all be identical.) In practice, no of course actual renderings will differ from one platform to the next. Honestly, though, it's even more complicated than that due to dependence on fonts, dependency versions, etc. So I wouldn't expect images on Ubuntu to match ones from Fedora, nor even ones rendered on Ubuntu 16.04 to match those on Ubuntu 20.04. Achieving decent apples-to-apples comparison gets to be a bit of an combinatorial explosion...</pre> </blockquote> A bit of the variance problem is rooted in cairo, though. For example <pre>CAIRO_HINT_STYLE_DEFAULT Use the default hint style for font backend and target device, since 1.0</pre> means that cairo will not set up anything, but use whatever freetype/fontconfig, CoreText or GDI will come up with. The tests do not do that. To get halfway stable results you'd have to manually configure all options to a non-default value I guess. So, as I understand now, the procedure for testing can only be: - Generate reference images - Mess with code - Run tests again This even is documented. Now, that is enlightening. </body> </html>