Troubleshooting

Symptom-based Troubleshooting

Service Unavailable (error 503)

When you see this error after trying to load a document, it is possible, that Collabora Online service (coolwsd) does not run properly.

First thing to try on the Collabora Online host from the administrator‘s (root) command line (commands listed for systemd):

systemctl status coolwsd

If the status is not active, and error is indicated, then check the logs:

journalctl -r -u coolwsd

The default log level is warning. If the root cause of the error is not clear, you can set the log level to be more detailed. Edit /etc/coolwsd/coolwsd.xml configuration file, and set the log level e.g. to trace. Then restart the service with:

systemctl restart coolwsd

and check the log again (reset log level afterwards, as verbose logging has performance implications).

The most common problem is forgetting the SSL setup. The path to SSL certificate, CA certificate and private key must be valid in /etc/coolwsd/coolwsd.xml. They can be set under ssl as cert_file_path, ca_file_path and key_file_path respectively.

Exit code 70 from coolwsd means internal software error, and this is most likely an issue with accessing certificates. Verify that certificates are set properly in the configuration file, and the files themselves are available with the correct access rights, i.e. readable by the cool user.

Gray document area, no document loaded

When you see gray document are, i.e. the iframe of Collabora Online is not loaded, you can check the browser‘s console to see what is going on. For the browser‘s diagnostic tools, press F12, and see network activity.

A common mistake, when http and https content is mixed on the same page, and the browser refuses to load insecure http iframe into an https page. If SSL is used, which is highly recommended, make sure that all components, including Collabora Online is set up for SSL.

The iframe can be blocked due to a content security policy setting. The problem should be reported on the browser‘s console.

Gray screen could be the result of incorrectly set up reverse proxy (see below), and related server_name setting.

Network connectivity problems

Collabora Online host, document storage host and user‘s browser have to see each other all times. To this end, it is always recommended to set up a reverse proxy on Collabora Online host, because the default port of Collabora Online, port 9980 is sometimes blocked by users‘ corporate firewall, only standard https port 443 is allowed.

If the reverse proxy is not preserving the Collabora Online host name, it has to be set in server_name setting.

Preferably do not use non-routable internal IP addresses or domain names that DNS cannot resolve on all hosts (otherwise they have to be present in the /etc/hosts file).

A command line example: coolconfig set server_name office.example.com.

Load balancing problems

In incorrectly configured load balanced multi-node Collabora Online cluster you can experience different characteristic symptoms:

Pasting into document doesn’t work. The browser’s request gets response 400 Bad Request and in the server logs you can find: ERR Cluster configuration error: mis-matching serverid AAAAAAA vs. BBBBBBB

When two users join to the same document but doesn’t see each other edits or saved content is lost or overwritten by other users working on the same file.

That means the HTTP requests related to the same file are not directed to the same Collabora Online node.

‘Unauthorized WOPI host error’ when opening a file.

The document storage host is not among the allowed WOPI hosts in /etc/coolwsd/coolwsd.xml configuration file (under element wopi). Add the domain name or the IP address, and restart coolwsd.

‘External data source not allowed’ when trying to insert content (image, clipboard paste).

We block requests from the Collabora Online server to the hosts which are not trusted. If you want to enable access to some hosts then you have to add regex into coolwsd.xml file in the``net.lok_allow`` section (host without port or scheme, IP or domain name depending on which one will be used in the request, we don’t resolve names). Hosts already defined in storage.wopi.alias_groups or net.post_allow should be enabled automatically.

Further symptoms

Tiles load slowly or missing when working with a file.

The document area is rendered in tiles on the server, and those tiles are transferred to the client. Slow tile display is likely caused by network latency issues, however if tiles do not load at all sometimes, then a ticket should be opened.

File does not display correctly or editing actions do not give expected result.

The cause is likely a bug in the document rendering/editing layer, a ticket should be opened.

Issues that were supposed to be fixed in the current version are present.

The installation might be outdated, verify the version using Help → About or in the console log accessible via journalctl -r -u coolwsd. If the version is as expected, a ticket should be opened.

Package upgrade issues

Webserver restart

The WOPI discovery.xml file may be cached at WOPI host. It contains versioned URLs that will point to a non-existing location after coolwsd upgrade. The symptom is that the content is not loaded into the iframe, and there are error message in the coolwsd log file, related to missing files. In this case restart the web server.

Diagnostic capabilities

Opening http(s)://<host>:<port>/ (as set up) in browser shows “OK” if service is running correctly
Opening http(s)://<host>:<port>/hosting/discovery (as set up) in browser brings up the WOPI discovery file
The following command brings up logs for the web socket daemon service: journalctl -r -u coolwsd

Preferably set log level to trace in /etc/coolwsd/coolwsd.xml configuration file under ‘logging’/’level’ beforehand
When opening a file, the browser’s developer console logs potential errors and information related to the web page
When a file is opened, Ctrl-Alt-Shift-D brings up a debug view that shows information related to rendered tiles, including server latency
The admin console is accessible separately under the following location: http(s)://<host>:<port>/browser/dist/admin/admin.html

It provides details on consumed memory, users online and documents open.

Note that a user/password combination has to be set up for admin console with the following command: coolconfig set-admin-password

Case studies

Case study No. 1

A partner installed Collabora Online from the provided packages, and set up the configuration. However, upon starting, coolwsd service exited with an error.

The partner provided their configuration and logs (after setting log level to ‘trace’).

From the logs it was apparent the service exited with status code 70, and the error was also in the log file:

Error loading private key from file <file name> (error:0200100D:system
library:fopen:Permission denied).

After correcting file permissions, the service started successfully.

After this there was another issue, the WOPI host was missing from the /etc/coolwsd/coolwsd.xml configuration file. After adding host name to ‘wopi’ element the service was running and accessible.

Case study No. 2

A partner installed Collabora Online from the provided packages, and set up the configuration to access files locally (enabled setting storage / filesystem), but was not able to access files. In a different, container-based setup, accessing local files worked.

Browser console logs showed that the server was accessed without the required port, and the requested file was not found. At first the reverse proxy seemed to be the culprit, but the issue persisted without proxy as well.

The /hosting/discovery file already listed the target host and path without the port. This pointed to the ‘server_name’ setting in /etc/coolwsd/coolwsd.xml, which was set, but without the necessary port. However in this case the setting was not necessary at all, and clearing it fixed the issue.