I like it here. Can I stay?

(And do you have a vacancy for a back-scrubber?)

Thank you, Red Hat, for generously letting me work for so long on stuff that is near and dear to my heart. At the intersection of theory and practice, of compiler technology and the LibreOffice code base. But, to keep doing what I love, I need to move on.

So, thank you, allotropia, for having me. I’m excited.

And, above all, thank you, LibreOffice, family of friends. Lets make sure to keep this a happy family, one that Tolstoy would have nothing to write home about. With quarrels and arguments, for sure; feud even. But happy at heart. I wish to keep meeting you all for years to come, virtually, and for a feast at FOSDEM or LOCon. And booze.

To paraphrase Hamburger Tafel (donations needed, btw), “Wir haben LibreOffice noch lange nicht satt.”

Have fun, stay safe, peace,
sberg

LibreOffice on Chromebooks

Until recently, Chromebooks could browse the Web and run dedicated ChromeOS and Android applications, and that was that. But things are changing now since Google announced Crostini, a technology to run arbitrary Linux applications on ChromeOS.

What you get, in short, is a Linux distribution running in a virtual machine. It is sandboxed, but with some channels set up between the virtual machine and the surrounding ChromeOS, so that e.g., icons of applications installed in the Linux VM show up in the ChromeOS launcher, and windows opened from within the VM are integrated with the overall ChromeOS desktop.

The default Linux distribution provided by Google is a Debian 9, and one should be able to also plug other flavours of Linux, at least in theory. But we can install applications as flatpaks there, at which point the exact Linux distribution becomes rather irrelevant, anyway.

How to set up your Chromebook to install flapaks is detailed elsewhere, so just follow those instructions. You can then browse flathub for the applications of your choice. But when you click on the “install” button for an app, things start to get a bit rough. The ChromeOS browser downloads a ….flatpakref file, but doesn’t know what to do with it, so stores it in the Downloads folder. The Linux VM can’t see it there (thanks to the sandboxing around that VM), so you first need to use the Files app to copy the ….flatpakref file from “Downloads” to “Linux files”. The latter is just the home directory of the Linux VM user, so when you switch to the Terminal app (which provides you with a shell for that Linux VM user), you can type flatpak install ….flatpakref to finally install the application.

But there is also a more polished solution that doesn’t require falling back to the flatpak command line: Just go back one more time to the Terminal shell to install GNOME Software in the Linux VM, sudo apt install gnome-software gnome-software-plugin-flatpak. Then you can run the Software application from the ChromeOS launcher (it may hide in a “Linux apps” collection). Search for LibreOffice, install it, launch it, and enjoy.

LibreOffice 6 available at flathub.org

We moved the LibreOffice Flatpak builds to Flathub a while ago, and updating to new versions is normally a pain-free process. (For us as producers of those updates, at least. For you as a user of Flatpak, the process should always be pain-free, anyway.)

However, for LibreOffice 6, building it on Flathub took a bit longer than expected:

Originally, the LibreOffice Flatpak version had lacked a JRE, so some functionality just wasn’t available. There is now a Flatpak extension that allows to bundle Java 9 with an app, so I was finally able to fix this long-standing issue.

In addition, I figured it could be useful to provide debug information for the LibreOffice flatpak (which comes as an extension that you can then optionally install with something like flatpak install flathub org.libreoffice.LibreOffice.Debug).

So I prepared the necessary changes well ahead of the LibreOffice 6.0.0 announcement, tested them locally on my x86_64 machine, and was confident that thinks would just work when I would trigger the Flathub build once the LibreOffice announcement was due. Turned out I was wrong.

First, one little aarch64-specific fix had been missing on the libreoffice-6-0 branch. No problem, just apply the corresponding master fix as a patch to the Flatpak version. Then builds started to fail randomly.

Enabling debug information for an application as large as LibreOffice can increase the size of what gets build, and the time to build it, quite substantially. I do such full-debug-information builds of upstream LibreOffice rather routinely on my (somewhat powerful) local build machine without any problem, so I didn’t even give a thought to this when enabling debug information for the Flathub builds. But it turns out that took some of the Flathub build machines (especially for less powerful platforms, like 32-bit x86) already close to the edge of their capacity. And something as trivial as applying the master-branch patch in the previous step meant that disk-space demand of the LibreOffice builds increased by yet another substantial amount (as Flatpak now needed to install the full LibreOffice git source repository, instead of taking shortcuts). That was apparently enough to move some of the builders over the edge, and into random “disk full” etc. failures.

So I weakened that debug feature again.  There is still unstripped symbol information in the Debug extension that you can download (i.e., you will see meaningful function names in backtraces), just not the whole debug data (e.g., you won’t see source file names and line numbers in backtraces, or be able to print variable values in gdb).

Meanwhile, there had been work going on to finally provide 32-bit arm builds of LibreOffice on Flathub. We had initially disabled that platform because of an issue with the version of GCC 6 that is used to do the builds. That arm-specific issue appears to be fixed with GCC 7, and there is now a Flatpak extension that allows to use GCC 7 instead of 6. And the test branch to do the LibreOffice build with GCC 7 and enable arm worked just fine. When building the old LibreOffice 5.4 version, that is.

When I tried to merge those two efforts that had gone on in parallel, switching to LibreOffice 6 and switching to GCC 7, things started to fail once again.

First off, the Java extension was not available for arm. So building on arm refused to work outright. There is work in progress to make it available, but by now LibreOffice 6 had officially been announced and available for download, and people waiting for the Flatpak version started to get impatient. So I decided to leave 32-bit arm unsupported for just a bit longer and dropped it from the list of supported platforms for the LibreOffice Flathub build again.

But things still didn’t work. The aarch64 build kept failing one of the tests that are executed during the build of LibreOffice. Sometimes these tests are a bit flaky and fail for “random” reasons. So I scheduled multiple builds, just to be sure. But, for sure, aarch64 kept failing, always during the same test.

I had kept using GCC 7 for these builds, even after the original reason for it (to enable 32-bit arm builds) was gone. Looks like there is some issue with building LibreOffice 6.0 with GCC 7, on aarch64 (which I still need to track down in detail). When I switched back to GCC 6, all builds were finally green. And were allowed into the Flathub repo, from where you can install/update them.

(That happened on Friday, just in time to get the LibreOffice 6 Flatpak version battle-tested at FOSDEM. I know at least gicmo’s and my presentations were powered by that version, without a glitch.)

LibreOffice, Flatpak’ed, Flathub’ed

Flathub is the app store for all things Flatpak. And LibreOffice is now available there, too!

When we started flatpak’ing LibreOffice, we made builds of the LibreOffice “Fresh” stream available on the Document Foundation download pages. Since LibreOffice 5.4.2, we now provide such builds through Flathub.

The benefit to you as a user is that there are builds for more architectures (currently aarch64, i386, and x86_64), and that you need to keep track of one less single-app Flatpak repo. (Check out what else is available in the Flathub repo!)

The benefit to me as provider of the original Document Foundation flatpak builds is that the Flathub infrastructure now does those builds for me. We will stop providing the Document Foundation builds after the current LibreOffice 5.4.3.

Too Subtle?

Spot the difference between the two C++ programs

struct S { int a = b, b; };
int main() { return (new S())->a; }

and

struct S { int a = b, b; };
int main() { return (new S)->a; }

Right, the first one terminates cleanly with exit code zero, while the second one does whatever it deems necessary to counter undefined behavior.

Why is that? The expression new S() means direct-initialization, which for () means value-initialization. Class S has a default constructor that is not user-provided, not deleted, and non-trivial (because non-static data member a has a default member initializer). So the instance of S is first zero-initialized, then default-initialized. Zero-initialization means that a and b are initialized to zero. Default-initialization for S means that the default constructor is called, which means that a is initialized from its default member initializer, by copying zero from b (and then b is default-initialized, leaving it alone).

On the other hand, just new S (without the parentheses) means just default-initialization (without previous zero-initialization). So, again, default-initialization for S means that the default constructor is called, which means that a is initialized from its default member initializer, copying the uninitialized b (and then b is default-initialized, leaving it uninitialized)…

Too subtle? Probably.

LibreOffice.flatpak update

New revisions of LibreOffice are routinely uploaded to http://www.libreoffice.org/download/flatpak/ by now. But things have changed slightly with the recent upload of LibreOffice 5.2.3:

GNOME Software (at least the version available in Fedora 25 beta) can now handle .flatpak files, so throwing the downloaded LibreOffice.flatpak at it will allow you to install the flatpak onto your system without resorting to the command line. (Just remember, for now, to first install the GNOME 3.20 runtime as detailed at http://www.libreoffice.org/download/flatpak/, or GNOME Software might refuse to install the LibreOffice.flatpak with an odd error message like “Could not find ‘org.libreoffice.LibreOffice.destkop’”.)

But for that to work the LibreOffice.flatpak had to be stripped down to advertise just a single application (a single org.libreoffice.LibreOffice.appdata.xml and a single org.libreoffice.LibreOffice.desktop). So when you have installed the flatpak, you’ll now only see a single application icon (the “generic” document icon) instead of the various sub-application icons for Writer, Calc, Impress, etc. (The upside being that you’ll thus no longer confuse your Linux distro’s LibreOffice installation with the flatpak one.) And opening new Writer, Calc, Impress, etc. documents still works from the “generic” application icon’s context menu.

Another recent improvement is that flatpak’s xdg-desktop-portal (and GNOME-specific xdg-desktop-portal-gtk backend) is now available (again, at least in Fedora 25 beta), so that clicking on hyperlinks in LibreOffice documents (or requesting the LibreOffice online help, e.g. by pressing F1) will now forward those requests to your browser and open the links there.

Clang in Fedora 24

I’m only ever using a bleeding edge Clang trunk build myself. So when Fedora enabled use of the GCC abi_tag attribute in libstdc++ (i.e., switched the _GLIBCXX_USE_CXX11_ABI macro to 1 by default), it was only a matter of applying a set of (evolving) patches to Clang to keep things working.

However, others who wanted to use the Clang version included in Fedora were in for more of a problem. The typical symptom being that what you compiled with Clang fails to link against system libraries, most notably libstdc++.

For Fedora 23, we included a somewhat slapdash early version of the relevant Clang patches, but that didn’t survive the split-out of Clang from the overall LLVM package in Fedora 24. But now that the Clang upstream patches are deemed stable (and have been integrated into trunk towards Clang 3.9), I backported them to the Clang 3.8 in Fedora 24 (and rawhide).

And a lot of thanks to all of those who created these patches!

Plugin, Flamed

Just how much is our Clang plugin slowing down builds of LibreOffice? We are constantly adding more and more useful functionality to it, and some of it is well known to be written in a way that is likely performing not too well. But I never dared look at the performance implications too closely, for fear of being disappointed. The build always seemed to go along at more or less the same speed, so who cares for details?

Enter FlameGraph, generating beautiful and easy-to-digest graphs on top of the Linux perf tool’s (or others’) output. So I gathered data about half a minute of CPU activity during the LibreOffice build (randomly picked a time when all the cores were busy compiling .cxx files in the sw module, to keep noise contributed by other processes down to a minimum), and passed it through FlameGraph. And sobering set in:98.56% of all the time was spent in the clang-3.9 process (i.e., there was almost no noise from unrelated processes). And 19.22% of all time was spent in our plugin. That is, the plugin slows down compilation by about 25%. Quite a number.

Zooming in and breaking this down further per individual plugin, the distribution is

• 1.47% CommaOperator
• 1.04% VCLWidgets
• 0.91% BodyNotInBlock
• 0.79% SalBool
• 0.69% PtrVector
• 0.65% RefCounting
• 0.63% FailedDynCast
• 0.62% ReservedId
• 0.61% Nullptr
• 0.59% RedundantCast
• 0.56% StringConstant
• 0.56% ImplicitBoolConversion
• 0.55% ExternAndNotDefined
• 0.50% PassStuffByRef
• 0.47% DefaultParams
• 0.46% StringConcat
• 0.45% UnusedVariableCheck
• 0.45% SalLogAreas
• 0.45% LiteralToBoolConversion
• 0.44% UnrefFun
• 0.44% ReturnByRef
• 0.43% StaticCall
• 0.43% BadStatics
• 0.42% Override
• 0.41% StaticAnonymous
• 0.41% InlineVisible
• 0.38% SimplifyBool
• 0.38% FpComparison
• 0.37% SfxPoolItem
• 0.35% StaticMethods
• 0.35% PrivateBase
• 0.35% BadVectorInit
• 0.34% DerefNullPtr
• 0.34% CStyleCast
• 0.32% RangeForCopy
• 0.32% LoopVarTooSmall
• 0.29% OnceVar

The recursive nature of RecursiveASTVisitor makes it a bit hard to easily spot each individual plugin’s hotspots, but one thing sticks out: CommaOperator is a rather simple piece of code (that shouldn’t need much heavy computation), but does use lopluign::Pluign::parentStmt, which rebuilds the AST to determine a node’s parent, and is really, really expensive.

Room for improvement. Time to clean up.

(At first, the generated graph looked way less interesting. One thing that was necessary was to build both LLVM/Clang and our plugin with -fno-omit-frame-pointer. The other thing was to teach FlameGraph to support “(anonymous namespace)” in C++ function names.)

LibreOffice 5.2 Beta Flatpak

Flatpak (formerly known under its working title “xdg-app”) is a cool new way of distributing images of Linux applications that run on a wide range of different distros, and run there in a secure way. I have blogged and spoken about making LibreOffice available in this format before in various places (see, e.g., these two previous blog posts and this conference talk).

Now is the time to announce availability of a LibreOffice.flatpak bundle of LibreOffice 5.2 Beta at download.documentfoundation.org/libreoffice/flatpak/5.2.0/LibreOffice.flatpak!

To try it out, do the following:

• First, you need to install Flatpak itself. See here for details.
• Second, you need to add the Flatpak repository that contains the GNOME runtime (on which the LibreOffice.flatpak depends; see here for details):

    $ wget https://sdk.gnome.org/keys/gnome-sdk.gpg
    $ flatpak remote-add --user --gpg-import=gnome-sdk.gpg gnome https://sdk.gnome.org/repo/
    $ flatpak install --user gnome org.gnome.Platform 3.20

  Update: If you use another locale than en-US, and want LibreOffice to automatically come up with a matching UI localization, additionally

    $ flatpak install --user gnome org.gnome.Platform.Locale 3.20

• Then, with the LibreOffice.flatpak file downloaded from the link above, do

    $ flatpak install --user --bundle LibreOffice.flatpak

  to install it. (Don’t worry, it does not interfere with any other LibreOffice you might already have installed on your machine, like the one shipped with your Linux distro.)

• And now you can run it, either from the command line

    $ flatpak run org.libreoffice.LibreOffice

  or by clicking on one of its icons (LibreOffice Start Center, Writer, etc.) in your distro’s application launcher.

(And if you happen to have an older version of xdg-app installed, installing the LibreOffice.flatpak bundle into it should work, too.)

Behind the scenes, the LibreOffice.flatpak links to a repository on the download.documentfoundation.org server that will get updated with later versions of LibreOffice Fresh. So once a new version of LibreOffice Fresh is released (starting with LibreOffice 5.2 proper), you can simply update your installation with

  $ flatpak update --user org.libreoffice.LibreOffice

A few items to note:

• LibreOffice.flatpak is based on the GNOME 3.20 runtime, using LibreOffice’s much-improved GTK3 backend. This should offer the most complete set of features in LibreOffice, paving the way to Wayland etc.
• The single bundle contains all localizations available for LibreOffice (as it doesn’t really affect the bundle’s overall size that much).
• Flatpak-based apps are not yet able to reach out to other applications installed on the machine (like browsers) to e.g. open URLs through them. That means that e.g. clicking on a hyperlink in a LibreOffice Writer document might not work yet. And since the LibreOffice.flatpak does not contain all the help documentation (for all the localizations!), but instead wants to use the online help through a browser, pressing F1 does not yet work, either.
• The LibreOffice.flatpak does not include a Java Runtime Environment (JRE). That means that some LibreOffice functionality that requires a JRE will not work.
• At this time, there is no Flatpak version of the LibreOffice Software Development Kit (SDK).

LibreOffice.flatpak

Flatpak is the new name of xdg-app. So following up on my previous LibreOffice in a Box post, here is some more technical detail on how an upcoming LibreOffice 5.2 will be available as a Flatpak bundle.

There is now a shell script demonstrating how to do all the downloading of sources, building, and bundling up. I decided against using flatpak-builder (nee xdg-app-builder) for that mostly out of convenience, mainly because the way LibreOffice fetches its external dependencies is so different from the typical way of building a Linux app. So I felt more comfortable doing these things manually, calling the raw xdg-app build steps from a script. But given enough pressure (e.g., to make the LibreOfficeKit widget buried within the LibreOffice installation set available to other Flatpak apps), this might get revisited.

The script proceeds in six steps (see the above link for details):

First, building LibreOffice requires an Archive-Zip Perl module not found in the org.gnome.Sdk we build against. The easiest solution (easier than to try and get rid of that Perl dependency in the LibreOffice build machinery, that is) is to download it and make it available to the build via the PERLLIB environment variable.

Second, we need the LibreOffice sources themselves. The current master branch has all the necessary changes needed for a Flatpak’ed LibreOffice (as will the coming libreoffice-5-2 branch).

Third, we need to fetch LibreOffice’s external dependencies. LibreOffice has a long list of external projects that it can either use from the underlying system or build/bundle itself. Those external projects covered by Flatpak’s org.gnome.Platform runtime are taken from there, but whatever remains is bundled with the LibreOffice app. The sources for these external projects are normally downloaded during the build, but an xdg-app build does not have network connectivity, so they need to be downloaded upfront. So bootstrap enough of a LibreOffice build here to execute the make fetch part.

Fourth, LibreOffice is built in xdg-app. The choice of building against the org.gnome.Platform 3.20 is somewhat arbitrarty, but it’s the latest revision available. And given LibreOffice’s strong GTK3 support (which e.g. also enables running on Wayland), this should make the resulting app more versatile than basing it on the more fundamental org.freedesktop.Platform. Building LibreOffice doesn’t exactly follow the make && make install mantra, but there happened to already be a make distro-pack-install target that produces the build artifacts in (almost) the right way for our purposes here.

Fifth, the build artifacts from the previous step are assembled in the way xdg-app expects them. Some of the metadata desktop and icon files need to be renamed, to match xdg-app’s expectation that metadata files belonging to the org.libreoffice.LibreOffice app have names actually starting with “org.libreoffice.LibreOffice”.

Finally, everything is bundled up into a LibreOffice.flatpak that can be distributed to users, who can then install it with xdg-app install --bundle LibreOffice.flatpak.

The bundle contains all the available LibreOffice localizations. It doesn’t make that much of a difference for the size of the resulting LibreOffice.flatpak, and should be easier to handle for both producers and consumers than to provide dedicated single-language apps. For reasons of size, the help content is not bundled, though; the intent is to (transparently) use the online help content. However, one caveat with the current state of xdg-app is that an app cannot reach out to other applications on the machine via xdg-open (a portal will be needed for that). When asked to process “external” URLs (e.g., when clicking on a hyperlink in a text document, but also when accessing the online help), LibreOffice tries to pass those URLs to xdg-open, but that doesn’t work yet in xdg-app.