<html><head></head><body><div style="font-family:Helvetica Neue, Helvetica, Arial, sans-serif;font-size:13px;"><div>Hi, First of all, Tom, it's definitely a good thing you've done, and I agree that using _wfopen() on Windows is the way to go. Personally I can live with this either way. It's controllable so I can deal with it, no worries. That said, I believe this is an API break. Functions that used to only accept codepage strings now only accept UTF-8 strings. An analogy would be a function which used to take a signed integer and now starts interpreting it as an unsigned integer. Or one which used to take integers and now switched to floats. And in my opinion, adding heuristics for determining the format of the input would just open up to new problems down the line. If you have data that you don't know the format of and you pass that to a library like Cairo then you're doing it wrong. Fix your data. So if it were my code I would add new API and deprecate the old, but it's certainly not the end of the world :) All the best, Mikael</div> <div> </div><div> </div> <div id="yahoo_quoted_5678400395" class="yahoo_quoted"> <div style="font-family:'Helvetica Neue', Helvetica, Arial, sans-serif;font-size:13px;color:#26282a;"> <div> On Friday, January 5, 2018, 10:24:11 AM GMT+1, <tom.schoonjans@diamond.ac.uk> wrote: </div> <div> </div> <div> </div> <div><div dir="ltr">Hi all, I am the author of the commit that added UTF-8 support for Windows. First I would like to state that I was indeed wrong to say that it enforces UTF-8 encoded filenames on all platforms. What I should have said is: On Windows filenames are assumed to be encoded in UTF-8, while on other platforms the current filename encoding will be used (which is usually UTF-8). In my experience, this is the most commonly observed behavior in the majority of open source projects out there, especially those in the GNOME ecosystem (Glib, Gtk+, libxml…). I would also argue that this is not an API breakage, but it does require the user to take into account this new behavior when running his/her software on Windows. Mikael, your usage of g_win32_locale_filename_from_utf8 is actually a bit dangerous since it will still fail in some (admittedly exotic) cases. The implementation of this function (<a shape="rect" href="https://github.com/GNOME/glib/blob/75fa8c2afbab4f414d2eb03684d9f807bd690aef/glib/gwin32.c#L692" target="_blank">https://github.com/GNOME/glib/blob/75fa8c2afbab4f414d2eb03684d9f807bd690aef/glib/gwin32.c#L692</a>) reveals that it will first try to convert the UTF-8 filename to the current codepage encoding, which will fail if not all characters can be represented in this codepage. In this case it will fall back to using the short filename (8.3) using GetShortPathNameW, which is not supported by all filesystems. My patch does not suffer from these issues as it uses the wide char Unicode API behind the scenes to open the files. I would also like to remark that Microsoft is actively discouraging the use of codepages because of their known problems and limitations, which is one of the reasons why I thought my patch would be a good idea… In your case Mikael, if you need to support versions of cairo that are older than 1.15.10, you will indeed need to do a cairo runtime version check to determine whether you need to call g_win32_locale_filename_from_utf8 or not. In summary, please do not revert this patch. It introduces proper handling of filenames in a manner that is similar to many other popular packages, and is guaranteed to work on all platforms, something that couldn’t be achieved until now on Windows. Best regards, Tom Dr Tom Schoonjans Data Analysis Scientist Tel: +44 1235 56 7507 Diamond Light Source Ltd. Diamond House Harwell Science & Innovation Campus Didcot Oxfordshire OX11 0DE United Kingdom On 5 Jan 2018, at 00:56, Bryce Harrington <<a shape="rect" ymailto="mailto:bryce@osg.samsung.com" href="mailto:bryce@osg.samsung.com">bryce@osg.samsung.com</a><mailto:<a shape="rect" ymailto="mailto:bryce@osg.samsung.com" href="mailto:bryce@osg.samsung.com">bryce@osg.samsung.com</a>>> wrote: [Adding to CC] I'd like to see further discussion on Mikael's proposal. Does the change to the handling of the filename content actually qualify as an API break? The previous implementation was underspecified as to the filename's format, although it would seem reasonable to assume it's an ordinary C string, so this does appear to at least qualify as a behavior change. Having the routine specified one way on Windows, and a different way on non-Windows seems a bit odd to me, and maybe it does raise a red flag that we're really dealing with separate functions conceptually. Unless, would there be any way for this to auto-detect the string format of the filename, and if it is not UTF-8 to fall back to the original behavior? I'll hold off reverting the patch for a week or so in hopes the discussion to come up with a good plan, and maybe patches. Bryce On Wed, Dec 27, 2017 at 02:06:07PM +0000, Mikael Claesson wrote: Hi Uli, It aborts and shows an error dialog - not ideal. Internally, I keep file names in UTF-8 so really I like the idea of passing UTF-8 filenames to cairo, but I'm concerned that we now have different incompatible behaviour in different versions of cairo. Before passing a file name to cairo I run it through g_win32_locale_filename_from_utf8(). And on Linux I use g_filename_from_utf8(). I noticed that the git commit has this comment: "This patch enforces the use of UTF-8 filenames on all platforms." That doesn't seem correct to me. While UTF-8 is by far the most common filename encoding on Linux, I don't think this is always the case, and that is why we have g_filename_from_utf8(). I may be completely wrong - I only started trying to wrap my head around this stuff recently - but I think calling fopen() with a UTF-8 string without first making sure which encoding is correct for the system is a bug? If I'm correct, then I suggest reverting the change and instead adding new API functions in parallell with the existing ones, that do take UTF-8 encoded filenames on all platforms. I would definitely use them. Best regards, Mikael On Wednesday, December 27, 2017, 9:24:19 AM GMT+1, Uli Schlachter <<a shape="rect" ymailto="mailto:psychon@znc.in" href="mailto:psychon@znc.in">psychon@znc.in</a><mailto:<a shape="rect" ymailto="mailto:psychon@znc.in" href="mailto:psychon@znc.in">psychon@znc.in</a>>> wrote: Hi, I can't answer your question, but I have a question to you: What does your application do if the file name cannot be expressed in the current codepage? Cheers, Uli On 26.12.2017 22:10, Mikael Claesson wrote: Hi, What is the recommended way of dealing with the "Use UTF-8 filenames on Windows" change? Will that not break API/ABI? In my application I currently have code in place to ensure that the file names are encoded in the current codepage. I guess I will now have to first check which cairo version the user has, and then convert as needed? Are all applications expected to do this? Best regards, Mikael On Tuesday, December 12, 2017, 2:06:42 AM GMT+1, Bryce Harrington <<a shape="rect" ymailto="mailto:bryce@osg.samsung.com" href="mailto:bryce@osg.samsung.com">bryce@osg.samsung.com</a><mailto:<a shape="rect" ymailto="mailto:bryce@osg.samsung.com" href="mailto:bryce@osg.samsung.com">bryce@osg.samsung.com</a>>> wrote: A new cairo snapshot 1.15.10 is now available from: <a shape="rect" href="http://cairographics.org/snapshots/cairo-1.15.10.tar.xz" target="_blank">http://cairographics.org/snapshots/cairo-1.15.10.tar.xz</a> which can be verified with: <a shape="rect" href="http://cairographics.org/snapshots/cairo-1.15.10.tar.xz.sha1" target="_blank">http://cairographics.org/snapshots/cairo-1.15.10.tar.xz.sha1</a> de180498ac563249b93ee5e17ba9aa26f90644b3 cairo-1.15.10.tar.xz <a shape="rect" href="http://cairographics.org/snapshots/cairo-1.15.10.tar.xz.sha1.asc" target="_blank">http://cairographics.org/snapshots/cairo-1.15.10.tar.xz.sha1.asc</a> (signed by Bryce Harrington) Additionally, a git clone of the source tree: git clone git://git.cairographics.org/git/cairo will include a signed 1.15.10 tag which points to a commit named: 95c464d5feaae58b6cc0990434ce2498cc315dc6 which can be verified with: git verify-tag 1.15.10 and can be checked out with a command such as: git checkout -b build 1.15.10 Release 1.15.10 (2017-12-07 Bryce Harrington <<a shape="rect" ymailto="mailto:bryce@osg.samsung.com" href="mailto:bryce@osg.samsung.com">bryce@osg.samsung.com</a><mailto:<a shape="rect" ymailto="mailto:bryce@osg.samsung.com" href="mailto:bryce@osg.samsung.com">bryce@osg.samsung.com</a>>>) ======================================================================== This release adds GLESv3 support to the cairo_gl backend, adds tracking of SVG units in generated svg documents, and cleans up numerous test failures and related issues in the PDF and Postscript backends. For a complete log of changes, please see <a shape="rect" href="http://cairographics.org/releases/ChangeLog.1.15.10" target="_blank">http://cairographics.org/releases/ChangeLog.1.15.10</a> Features and Enhancements ------------------------- * Add support for OpenGL ES 3.0 to the gl backend. * Use Reusable streams for forms in Level 3 Postscript. * Add CAIRO_MIME_TYPE_EPS mime type for embedding EPS files. * Add CCITT_FAX mime type for PDF and PS surfaces * svg: add a new function to specify the SVG document unit (Bug #90166) * Use UTF-8 filenames on Windows API Changes ----------- * cairo_svg_surface_set_document_unit() and cairo_svg_surface_get_document_unit() Dependency Changes ------------------ None Performance Optimizations ------------------------- None Bug Fixes --------- * Fix regression in gles version detection * Fix undefined-behavior with integer math. * Handle SOURCE and CLEAR operators when painting color glyphs. (Bug #102661) * Convert images to rgba or a8 formats when uploading with GLESv2 * Use _WIN32 instead of windows.h to check for windows build. * Fix sigabrt printing documents with fonts lacking the mandatory .nodef glyph. (Bug #102922) * Prevent curved strokes in small ctms from being culled from vector surfaces (Bug #103071) * Fix painting an unbounded recording surface with the SVG backend. * Fix falling back to system font with PDFs using certain embedded fonts, due to truncated font names. (Bug #103249) * Fix handling of truetype fonts with excessively long font names (Bug #103249) * Fix race conditions with cairo_mask_compositor_t (Bug #103037) * Fix build error with util/font-view * Fix assertion hit with PDFs using Type 4 fonts rendered with user fonts, due to error when destroying glyph page. (Bug #103335) * Set default creation date for PDFs * Prevent invalid ptr access for > 4GB images. (Bug #98165) * Prevent self-copy infinite loop in Postscript surface. * Fix padded image crash in Postscript surface. * Fix annotation bugs in PDFs and related memory leaks * Fix test failures and other assorted issues in ps and pdf code. * Fix code generation when using GCC legacy atomic operations (Bug #103559) * Fix various compilation warnings and errors. * Fix various distcheck errors with private symbols, doxygen formatting, etc. See below for a complete log of changes since 1.15.8, or see: <a shape="rect" href="http://cairographics.org/releases/ChangeLog.cairo-1.15.10" target="_blank">http://cairographics.org/releases/ChangeLog.cairo-1.15.10</a> What is cairo ------------- Cairo is a 2D graphics library with support for multiple output devices. Currently supported output targets include the X Window System (via both Xlib and XCB), quartz, win32, and image buffers, as well as PDF, PostScript, and SVG file output. Experimental backends include OpenGL, BeOS, OS/2, and DirectFB. Cairo is free software and is available to be redistributed and/or modified under the terms of either the GNU Lesser General Public License (LGPL) version 2.1 or the Mozilla Public License (MPL) version 1.1. Where to get more information about cairo ----------------------------------------- The primary source of information about cairo is: <a shape="rect" href="http://cairographics.org/" target="_blank">http://cairographics.org/</a> The latest versions of cairo can always be found at: <a shape="rect" href="http://cairographics.org/download" target="_blank">http://cairographics.org/download</a> Documentation on using cairo and frequently-asked questions: <a shape="rect" href="http://cairographics.org/documentation" target="_blank">http://cairographics.org/documentation</a> <a shape="rect" href="http://cairographics.org/FAQ" target="_blank">http://cairographics.org/FAQ</a> Mailing lists for contacting cairo users and developers: <a shape="rect" href="http://cairographics.org/lists" target="_blank">http://cairographics.org/lists</a> Roadmap and unscheduled things to do, (please feel free to help out): <a shape="rect" href="http://cairographics.org/roadmap" target="_blank">http://cairographics.org/roadmap</a> <a shape="rect" href="http://cairographics.org/todo" target="_blank">http://cairographics.org/todo</a> Changes since 1.15.8 -------------------- Adrian Johnson (47): RELEASING: use correct branch name Remove unused variable build: use _WIN32 instead of windows.h to check for windows build replace _BSD_SOURCE with _DEFAULT_SOURCE factor out ascii to double code in cff-subset into _cairo_strtod truetype: reserve space in subset arrays for .notdef truetype: clarify glyph count variables Prevent curved strokes in small ctms from being culled from vector surfaces svg: fix painting an unbounded recording surface output-stream: allow %s strings larger than 512 chars truetype: limit font name to 127 chars svg: use hash table instead of user_data to track emitted surfaces svg: source surface hash table does not need to hold the source svg2png: remove unused headers ft: prevent unused var warning when freetype < 2.8 fix unused function warnings svg: recording_surface is needed even if not emitted fix warning: variable X might be clobbered by 'longjmp' fix warning: inlining failed in call to '_csi_stack_push' util/font-view: fix build error Add CCITT_FAX mime type for PDF and PS surfaces Allow mime image to be different size to cairo image pdf: set ca/CA instead of using an smask when the mask has constant alpha pdf: set default create date pdf: remove old comment image: prevent invalid ptr access for > 4GB images Add mime-unique-id test pdf: fix mime-unique-id bounded recording test pdf: fix mime-unique-id unbounded recording test pdf: fix mime-unique-id jpeg attached to recording test ps: emit base85 strings instead of strings of base85 ps: remove unused prolog ps: use << >> for dictionaries instead of dict begin end ps: don't acquire image or snapshot in acquire_source_image_from_pattern ps: use forms for surfaces with UNIQUE_ID mime type ps: use Reusable streams for forms in Level 3 ps: add CAIRO_MIME_TYPE_EPS mime type for embedding EPS files test: use CAIRO_MIME_TYPE_UNIQUE_ID with record-text-transform ps: prevent self-copy infinite loop ps: fix padded image crash ps: fix extend-*-similar failures test: update some stale ref images pdf: fix document structure for non tagged structures ps: fix compile with old versions of MSVC pdf: fix some annotation bugs Prevent -Wundef warnings in when cairo-ft.h is used without fontconfig ps: fix compile warning Aleksander Morgado (1): build: fix minor typo in autogen.sh Antonio Ospite (2): svg: add a new function to specify the SVG document unit svg: fix compilation with MSVC which doesn't support C99 initializers Behdad Esfahbod (2): Fix undefined-behavior with integer math Handle SOURCE and CLEAR operators when painting color glyphs Bryce Harrington (15): Bump version for new development tree, 1.15.9 glesv2: Fix regression in gles version detection gl: Convert images to rgba or a8 formats when uploading with GLESv2 gl: Make _cairo_gl_ensure_framebuffer a private shared routine gl: Add support for OpenGL ES 3.0 Factor out the ISFINITE() macro configure: Check for typeof Un-doxygen disabled cairo_set_opacity image: Fix include for use of ptrdiff win32: Fix since field version number Fix various doxygen warnings found by check-doc-syntax.sh Fix distcheck errors on use of #ifdef pattern: Mark a private routine as cairo_private. 1.15.10 release Bump version for new development tree, 1.15.9 Carlos Garcia Campos (1): scaled-font: Fix assert when destroying glyph page Mikhail Fludkov (2): Surround initialisations with atomic critical section Fix code generation when using GCC legacy atomic operations Tom Schoonjans (1): Use UTF-8 filenames on Windows -- "In the beginning the Universe was created. This has made a lot of people very angry and has been widely regarded as a bad move." -- cairo mailing list <a shape="rect" ymailto="mailto:cairo@cairographics.org" href="mailto:cairo@cairographics.org">cairo@cairographics.org</a><mailto:<a shape="rect" ymailto="mailto:cairo@cairographics.org" href="mailto:cairo@cairographics.org">cairo@cairographics.org</a>><div class="yqt8691524800" id="yqtfd30462"> <a shape="rect" href="https://lists.cairographics.org/mailman/listinfo/cairo" target="_blank">https://lists.cairographics.org/mailman/listinfo/cairo</a></div> -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom<div class="yqt8691524800" id="yqtfd86484"> </div></div></div> </div> </div></div></body></html>