| Current Path : /usr/share/doc/nextcloud-desktop/html/ |
Linux ift1.ift-informatik.de 5.4.0-216-generic #236-Ubuntu SMP Fri Apr 11 19:53:21 UTC 2025 x86_64 |
| Current File : //usr/share/doc/nextcloud-desktop/html/architecture.html |
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Appendix B: History and Architecture — Nextcloud Client Manual 2.4.0 documentation</title>
<link rel="stylesheet" href="_static/classic.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<script type="text/javascript" src="_static/language_data.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Appendix C: Troubleshooting" href="troubleshooting.html" />
<link rel="prev" title="Appendix A: Building the Client" href="building.html" />
</head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="troubleshooting.html" title="Appendix C: Troubleshooting"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="building.html" title="Appendix A: Building the Client"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">Client Manual</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="appendix-b-history-and-architecture">
<h1>Appendix B: History and Architecture<a class="headerlink" href="#appendix-b-history-and-architecture" title="Permalink to this headline">¶</a></h1>
<p id="index-0">Nextcloud provides desktop sync clients to synchronize the contents of local
directories from computers, tablets, and handheld devices to the Nextcloud
server.</p>
<p>Synchronization is accomplished using <a class="reference external" href="http://www.csync.org">csync</a>, a bidirectional file
synchronizing tool that provides both a command line client as well as a
library. A special module for csync was written to synchronize with the
Nextcloud built-in WebDAV server.</p>
<p>The Nextcloud Client software is written in C++ using the <a class="reference external" href="http://www.qt-project.org">Qt Framework</a>. As a
result, the Nextcloud Client runs on Linux, Windows, and MacOS.</p>
<div class="section" id="the-synchronization-process">
<h2>The Synchronization Process<a class="headerlink" href="#the-synchronization-process" title="Permalink to this headline">¶</a></h2>
<p>The process of synchronization keeps files in two separate repositories the
same. When synchronized:</p>
<ul class="simple">
<li>If a file is added to one repository it is copied to the other synchronized repository.</li>
<li>When a file is changed in one repository, the change is propagated to any other
synchronized repository.</li>
<li>If a file is deleted in one repository, it is deleted in any other.</li>
</ul>
<p>It is important to note that the Nextcloud synchronization process does not use
a typical client/server system where the server is always master. This is a
major difference between the Nextcloud synchronization process and other systems
like a file backup, where only changes to files or folders and the addition of
new files are propagated, but these files and folders are never deleted unless
explicitly deleted in the backup.</p>
<p>During synchronization, the Nextcloud Client checks both repositories for
changes frequently. This process is referred to as a <em>sync run</em>. In between
sync runs, the local repository is monitored by a file system monitoring
process that starts a sync run immediately if something was edited, added, or
removed.</p>
</div>
<div class="section" id="synchronization-by-time-versus-etag">
<h2>Synchronization by Time versus ETag<a class="headerlink" href="#synchronization-by-time-versus-etag" title="Permalink to this headline">¶</a></h2>
<p id="index-1">Until the release of the client version 1.1, the Nextcloud
synchronization process employed a single file property – the file modification
time – to decide which file was newer and needed to be synchronized to the
other repository.</p>
<p>The <em>modification timestamp</em> is part of the files metadata. It is available on
every relevant filesystem and is the typical indicator for a file change.
Modification timestamps do not require special action to create, and have a
general meaning. One design goal of csync is to not require a special server
component. This design goal is why csync was chosen as the backend component.</p>
<p>To compare the modification times of two files from different systems, csync
must operate on the same base. Before client version 1.1.0, csync
required both device repositories to run on the exact same time. This
requirement was achieved through the use of enterprise standard <a class="reference external" href="http://en.wikipedia.org/wiki/Network_Time_Protocol">NTP time
synchronization</a> on all machines.</p>
<p>Because this timing strategy is rather fragile without the use of NTP, the Nextcloud
server provides a unique number that changes whenever the file
changes. Although this number is a unique value, it is not a hash of the file.
Instead, it is a randomly chosen number, that is transmitted in the <a class="reference external" href="http://en.wikipedia.org/wiki/HTTP_ETag">Etag</a>
field. Because the file number changes if the file changes, its use is
guaranteed to determine if one of the files has changed and, thereby, launching
a synchronization process.</p>
<p>Before the 1.3.0 release of the Desktop Client, the synchronization process
might create false conflict files if time deviates. Original and changed files
conflict only in their timestamp, but not in their content. This behavior was
changed to employ a binary check if files differ.</p>
<p>Like files, directories also hold a unique ID that changes whenever one of the
contained files or directories is modified. Because this is a recursive
process, it significantly reduces the effort required for a synchronization
cycle, because the client only analyzes directories with a modified ID.</p>
</div>
<div class="section" id="comparison-and-conflict-cases">
<h2>Comparison and Conflict Cases<a class="headerlink" href="#comparison-and-conflict-cases" title="Permalink to this headline">¶</a></h2>
<p>As mentioned above, during a <em>sync run</em> the client must first detect if one of
the two repositories have changed files. On the local repository, the client
traverses the file tree and compares the modification time of each file with an
expected value stored in its database. If the value is not the same, the client
determines that the file has been modified in the local repository.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">On the local side, the modification time is a good attribute to use for
detecting changes, because
the value does not depend on time shifts and such.</p>
</div>
<p>For the remote (that is, Nextcloud server) repository, the client compares the
ETag of each file with its expected value. Again, the expected ETag value is
queried from the client database. If the ETag is the same, the file has not
changed and no synchronization occurs.</p>
<p>In the event a file has changed on both the local and the remote repository
since the last sync run, it can not easily be decided which version of the file
is the one that should be used. However, changes to any side will not be lost. Instead,
a <em>conflict case</em> is created. The client resolves this conflict by renaming the
local file, appending a conflict label and timestamp, and saving the remote file
under the original file name.</p>
<p>Example: Assume there is a conflict in message.txt because its contents have
changed both locally and remotely since the last sync run. The local file with
the local changes will be renamed to message_conflict-20160101-153110.txt and
the remote file will be downloaded and saved as message.txt.</p>
<p>Conflict files are always created on the client and never on the server.</p>
</div>
<div class="section" id="ignored-files">
<span id="ignored-files-label"></span><h2>Ignored Files<a class="headerlink" href="#ignored-files" title="Permalink to this headline">¶</a></h2>
<p>The Nextcloud Client supports the ability to exclude or ignore certain files from the synchronization process.
Some system wide file patterns that are used to exclude or ignore files are included with the client by default and the Nextcloud Client provides the ability to add custom patterns.</p>
<p>By default, the Nextcloud Client ignores the following files:</p>
<ul class="simple">
<li>Files matched by one of the patterns defined in the Ignored Files Editor.</li>
<li>Files starting with <code class="docutils literal notranslate"><span class="pre">._sync_*.db*</span></code>, <code class="docutils literal notranslate"><span class="pre">.sync_*.db*</span></code>, <code class="docutils literal notranslate"><span class="pre">.csync_journal.db*</span></code>, <code class="docutils literal notranslate"><span class="pre">.owncloudsync.log*</span></code>, as these files are reserved for journalling.</li>
<li>Files with a name longer than 254 characters.</li>
<li>The file <code class="docutils literal notranslate"><span class="pre">Desktop.ini</span></code> in the root of a synced folder.</li>
<li>Files matching the pattern <code class="docutils literal notranslate"><span class="pre">*_conflict-*</span></code> unless conflict file uploading is enabled.</li>
<li>Windows only: Files containing characters that do not work on typical Windows filesystems <code class="docutils literal notranslate"><span class="pre">(`\,</span> <span class="pre">/,</span> <span class="pre">:,</span> <span class="pre">?,</span> <span class="pre">*,</span> <span class="pre">",</span> <span class="pre">>,</span> <span class="pre"><,</span> <span class="pre">|`)</span></code>.</li>
<li>Windows only: Files with a trailing space or dot.</li>
<li>Windows only: Filenames that are reserved on Windows.</li>
</ul>
<p>If a pattern selected using a checkbox in the <cite>ignoredFilesEditor-label</cite> (or if
a line in the exclude file starts with the character <code class="docutils literal notranslate"><span class="pre">]</span></code> directly followed by
the file pattern), files matching the pattern are considered <em>fleeting meta
data</em>.</p>
<p>These files are ignored and <em>removed</em> by the client if found in the
synchronized folder.
This is suitable for meta files created by some applications that have no sustainable meaning.</p>
<p>If a pattern ends with the forward slash (<code class="docutils literal notranslate"><span class="pre">/</span></code>) character, only directories are matched.
The pattern is only applied for directory components of filenames selected using the checkbox.</p>
<p>To match filenames against the exclude patterns, the UNIX standard C library
function <code class="docutils literal notranslate"><span class="pre">fnmatch</span></code> is used.
This process checks the filename against the specified pattern using standard shell wildcard pattern matching.
For more information, please refer to <a class="reference external" href="http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_13_01">The opengroup website</a>.</p>
<p>The path that is checked is the relative path under the sync root directory.</p>
<p><strong>Pattern and File Match Examples:</strong></p>
<table border="1" class="docutils">
<colgroup>
<col width="27%" />
<col width="73%" />
</colgroup>
<thead valign="bottom">
<tr class="row-odd"><th class="head">Pattern</th>
<th class="head">File Matches</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td><code class="docutils literal notranslate"><span class="pre">~$*</span></code></td>
<td><code class="docutils literal notranslate"><span class="pre">~$foo</span></code>, <code class="docutils literal notranslate"><span class="pre">~$example.doc</span></code></td>
</tr>
<tr class="row-odd"><td><code class="docutils literal notranslate"><span class="pre">fl?p</span></code></td>
<td><code class="docutils literal notranslate"><span class="pre">flip</span></code>, <code class="docutils literal notranslate"><span class="pre">flap</span></code></td>
</tr>
<tr class="row-even"><td><code class="docutils literal notranslate"><span class="pre">moo/</span></code></td>
<td><code class="docutils literal notranslate"><span class="pre">map/moo/</span></code>, <code class="docutils literal notranslate"><span class="pre">moo/</span></code></td>
</tr>
</tbody>
</table>
</div>
<div class="section" id="the-sync-journal">
<h2>The Sync Journal<a class="headerlink" href="#the-sync-journal" title="Permalink to this headline">¶</a></h2>
<p>The client stores the ETag number in a per-directory database, called the
<em>journal</em>. This database is a hidden file contained in the directory to be
synchronized.</p>
<p>If the journal database is removed, the Nextcloud Client CSync backend rebuilds
the database by comparing the files and their modification times. This process
ensures that both server and client are synchronized using the appropriate NTP
time before restarting the client following a database removal.</p>
</div>
<div class="section" id="custom-webdav-properties">
<h2>Custom WebDAV Properties<a class="headerlink" href="#custom-webdav-properties" title="Permalink to this headline">¶</a></h2>
<p>In the communication between client and server a couple of custom WebDAV properties
were introduced. They are either needed for sync functionality or help have a positive
effect on synchronization performance.</p>
<p>This chapter describes additional XML elements which the server returns in response
to a successful PROPFIND request on a file or directory. The elements are returned in
the namespace <code class="docutils literal notranslate"><span class="pre">oc</span></code>.</p>
</div>
<div class="section" id="server-side-permissions">
<h2>Server Side Permissions<a class="headerlink" href="#server-side-permissions" title="Permalink to this headline">¶</a></h2>
<p>The XML element <code class="docutils literal notranslate"><span class="pre"><oc:permissions></span></code> represents the permission- and sharing state of the
item. It is a list of characters, and each of the chars has a meaning as outlined
in the table below:</p>
<table border="1" class="docutils">
<colgroup>
<col width="9%" />
<col width="25%" />
<col width="66%" />
</colgroup>
<tbody valign="top">
<tr class="row-odd"><td>Code</td>
<td>Resource</td>
<td>Description</td>
</tr>
<tr class="row-even"><td>S</td>
<td>File or Folder</td>
<td>is shared</td>
</tr>
<tr class="row-odd"><td>R</td>
<td>File or Folder</td>
<td>can share (includes re-share)</td>
</tr>
<tr class="row-even"><td>M</td>
<td>File or Folder</td>
<td>is mounted (like on Dropbox, Samba, etc.)</td>
</tr>
<tr class="row-odd"><td>W</td>
<td>File</td>
<td>can write file</td>
</tr>
<tr class="row-even"><td>C</td>
<td>Folder</td>
<td>can create file in folder</td>
</tr>
<tr class="row-odd"><td>K</td>
<td>Folder</td>
<td>can create folder (mkdir)</td>
</tr>
<tr class="row-even"><td>D</td>
<td>File or Folder</td>
<td>can delete file or folder</td>
</tr>
<tr class="row-odd"><td>N</td>
<td>File or Folder</td>
<td>can rename file or folder</td>
</tr>
<tr class="row-even"><td>V</td>
<td>File or Folder</td>
<td>can move file or folder</td>
</tr>
</tbody>
</table>
<p>Example:</p>
<blockquote>
<div><oc:permissions>RDNVCK</oc:permissions></div></blockquote>
</div>
<div class="section" id="file-or-directory-size">
<h2>File- or Directory Size<a class="headerlink" href="#file-or-directory-size" title="Permalink to this headline">¶</a></h2>
<p>The XML element <code class="docutils literal notranslate"><span class="pre"><oc:size></span></code> represents the file- or directory size in bytes. For
directories, the size of the whole file tree underneath the directory is accumulated.</p>
<p>Example:</p>
<blockquote>
<div><oc:size>2429176697</oc:size></div></blockquote>
</div>
<div class="section" id="fileid">
<h2>FileID<a class="headerlink" href="#fileid" title="Permalink to this headline">¶</a></h2>
<p>The XML element <code class="docutils literal notranslate"><span class="pre"><oc:id></span></code> represents the so called file ID. It is a non volatile string id
that stays constant as long as the file exists. It is not changed if the file changes or
is renamed or moved.</p>
<p>Example:</p>
<blockquote>
<div><oc:id>00000020oc5cfy6qqizm</oc:id></div></blockquote>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h3><a href="index.html">Table of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Appendix B: History and Architecture</a><ul>
<li><a class="reference internal" href="#the-synchronization-process">The Synchronization Process</a></li>
<li><a class="reference internal" href="#synchronization-by-time-versus-etag">Synchronization by Time versus ETag</a></li>
<li><a class="reference internal" href="#comparison-and-conflict-cases">Comparison and Conflict Cases</a></li>
<li><a class="reference internal" href="#ignored-files">Ignored Files</a></li>
<li><a class="reference internal" href="#the-sync-journal">The Sync Journal</a></li>
<li><a class="reference internal" href="#custom-webdav-properties">Custom WebDAV Properties</a></li>
<li><a class="reference internal" href="#server-side-permissions">Server Side Permissions</a></li>
<li><a class="reference internal" href="#file-or-directory-size">File- or Directory Size</a></li>
<li><a class="reference internal" href="#fileid">FileID</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="building.html"
title="previous chapter">Appendix A: Building the Client</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="troubleshooting.html"
title="next chapter">Appendix C: Troubleshooting</a></p>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="_sources/architecture.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="troubleshooting.html" title="Appendix C: Troubleshooting"
>next</a> |</li>
<li class="right" >
<a href="building.html" title="Appendix A: Building the Client"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">Client Manual</a> »</li>
</ul>
</div>
<div class="footer" role="contentinfo">
© Copyright 2013-2020, The Nextcloud developers.
Last updated on Mar 24, 2020.
</div>
</body>
</html>