Updated documentation for v1.00 release
This commit is contained in:
parent
903379ec84
commit
1d7c433158
|
@ -91,7 +91,7 @@
|
||||||
\begin_body
|
\begin_body
|
||||||
|
|
||||||
\begin_layout Title
|
\begin_layout Title
|
||||||
Osync v0.99 RC4 Documentation
|
Osync v1.00 Documentation
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Author
|
\begin_layout Author
|
||||||
|
@ -107,7 +107,7 @@ Ozy
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Date
|
\begin_layout Date
|
||||||
26 November 2014
|
02 July 2015
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
|
@ -160,13 +160,13 @@ A quickstart guide might be found in the README.md file.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Subsection
|
\begin_layout Subsection
|
||||||
Basic synchronization concepts
|
Basic synchronization problems
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
Synchronization is usually found in two flavors, filesystem sync and file
|
Synchronization is usually found in two flavors, bloc level sync and file
|
||||||
level sync.
|
level sync.
|
||||||
While whole filesystem synchronization is generally a good way, it's also
|
While whole bloc level synchronization is generally a good way, it's also
|
||||||
very greedy in network ressources and is not easy to setup.
|
very greedy in network ressources and is not easy to setup.
|
||||||
That's where file level sync comes in handy, where only some directories
|
That's where file level sync comes in handy, where only some directories
|
||||||
need to be synced.
|
need to be synced.
|
||||||
|
@ -176,22 +176,22 @@ Synchronization is usually found in two flavors, filesystem sync and file
|
||||||
Now imagine you're syncing two remote offices of a same company.
|
Now imagine you're syncing two remote offices of a same company.
|
||||||
If you're syncing a user's home directory or it's roaming profile as a
|
If you're syncing a user's home directory or it's roaming profile as a
|
||||||
night task, the next day, the user will find it's roaming profile up to
|
night task, the next day, the user will find it's roaming profile up to
|
||||||
date at the other office.
|
date at the remote office.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
But what would happen if two users work on the same file in a public folder,
|
But what would happen if two users work on the same file in a public folder,
|
||||||
at the same time, on both offices ? Most sync software would stop sync
|
at the same time, on both offices ? Some sync software would stop sync
|
||||||
and ask what to do.
|
and ask what to do.
|
||||||
Others might simply deleted the oldest version of the file, even if it
|
Others might simply deleted the oldest version of the file, even if it
|
||||||
was modified on both sides.
|
was modified on both sides.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
Also, what would happen if a user uploads too much new data ? If the link
|
Also, what would happen if a user uploads a lot of data ? If the link between
|
||||||
between both offices cannot handle enough data transfer in a given time,
|
both offices cannot handle enough data transfer in a given time, any other
|
||||||
any other sync task won't be run, and the sync would continue during the
|
sync task won't be run, and the sync would continue during the day, when
|
||||||
day, when bandwidth is necessary elsewhere.
|
bandwidth is necessary elsewhere.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
|
@ -200,7 +200,11 @@ What would happen if a power fault occurs while synchronization is going
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Subsection
|
\begin_layout Subsection
|
||||||
What exactly can do Osync
|
What exactly osync can do
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Subsubsection
|
||||||
|
Making synchronisation reliable
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
|
@ -210,50 +214,58 @@ Osync is designed to synchronize two folders on both local and / or remote
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
It is time controlled, which means you can decide how much time it should
|
It is time controlled, which means you can decide how much time it should
|
||||||
spend on a sync task before stopping it.
|
spend on a sync task before stopping it and launching the next one.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
It can resume failed or stopped sync tasks, or totally restart the sync
|
It's designed to resume failed or stopped sync tasks, and totally restart
|
||||||
task if more earlier runs failed.
|
the sync task if resume fails.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
It can keep an earlier version of a file in case of a conflict.
|
It can keep an multiple versions of a file in case of a conflict.
|
||||||
It can also keep multiple earlier versions.
|
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
It can soft delete files, meaning if a user deletes a file on a replica,
|
It handles soft deletion.
|
||||||
it will keep a copy of that file on the other replica in a
|
If a user deletes a file on replica A, it will move that file on replica
|
||||||
|
B to the
|
||||||
\begin_inset Quotes sld
|
\begin_inset Quotes sld
|
||||||
\end_inset
|
\end_inset
|
||||||
|
|
||||||
recycle bin
|
deleted
|
||||||
\begin_inset Quotes srd
|
\begin_inset Quotes srd
|
||||||
\end_inset
|
\end_inset
|
||||||
|
|
||||||
.
|
folder.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
It will automatically clean old files (soft deleted and conflict backed
|
It will automatically clean old files (soft deleted and conflict backups)
|
||||||
up ones) after a defined amount of days.
|
after a defined amount of days.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
It will check local disk space before trying to sync.
|
It will check local disk space before trying to sync.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Subsubsection
|
||||||
|
Making a sysadmin's life easier
|
||||||
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
It will send a warning email including the whole sync process execution
|
osync is also desgined to ease synchronisation setups.
|
||||||
log if a warning or error is triggered.
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
It will trigger a warning email including the whole sync process execution
|
||||||
|
log if an error is triggered.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
Pre-processing and post-processing commands can be launched on local and
|
Pre-processing and post-processing commands can be launched on local and
|
||||||
/ or remote systems (usefull to launch snapshot, flush or virtual machines
|
/ or remote systems, which may be usefull to launch snapshot software,
|
||||||
standby scripts).
|
flush or standby virtual machines, etc).
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
|
@ -261,18 +273,24 @@ Multiple concurrent instances of osync can be run as long as they don't
|
||||||
sync the same replicas at the same time.
|
sync the same replicas at the same time.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
A batch processing script is included to launch sequential sync tasks.
|
||||||
|
Failed sync tasks are rerun when every other task has completed, and there's
|
||||||
|
still some time left in a given timespan.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
Osync can use rsync or ssh tunnel compression to gain bandwidth.
|
Osync can use rsync or ssh tunnel compression to gain bandwidth.
|
||||||
Bandwidth can also be limited.
|
Bandwidth can also be limited for slow link sharing.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
It can run in quicksync mode for the impatient (nothing to configure except
|
It can be run in quicksync mode for the impatient (nothing to configure
|
||||||
the replica paths), or with a full blown config file.
|
except the replica paths), or with a full blown config file.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
You may run Osync manually, schedule its runs with cron, or have it monitor
|
You may run osync manually, schedule its runs with cron, or have it monitor
|
||||||
a directory as a daemon.
|
a directory as a daemon.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
|
@ -280,27 +298,63 @@ You may run Osync manually, schedule its runs with cron, or have it monitor
|
||||||
Osync has been succesfully tested on RHEL / CentOS 5, CentOS 6, Centos 7,
|
Osync has been succesfully tested on RHEL / CentOS 5, CentOS 6, Centos 7,
|
||||||
Debian Linux 6.0.7, Linux Mint 14, FreeBSD 8.3, Mac OS X, and Windows using
|
Debian Linux 6.0.7, Linux Mint 14, FreeBSD 8.3, Mac OS X, and Windows using
|
||||||
MSYS environment.
|
MSYS environment.
|
||||||
|
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Subsection
|
\begin_layout Subsubsection
|
||||||
How Osync tries to resolve sync issues
|
What osync cannot do
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
Let's get back to the example above, where Osync is used to sync two remote
|
Osync is a simple bash script that relies on other tools like rsync.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
Hence, it has some advantages and disavantages:
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
Advantages:
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
- It's easily customisable
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
- It's fast
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
Disavantages:
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
- There's no way to detect file moves.
|
||||||
|
If you move a directory on replica A, osync will soft delete the directory
|
||||||
|
on replica B and copy the new directory from replica A.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
- There's no mulimaster replication in osync V1.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Subsection
|
||||||
|
How osync tries to resolve sync issues
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
Let's get back to the example above, where osync is used to sync two remote
|
||||||
offices with users' home directories.
|
offices with users' home directories.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
Now imagine a user uploaded 100GB of data, whereas the WAN link can only
|
Now imagine a user uploaded 100GB of data, and the WAN link between local
|
||||||
handle 60GB of data transfer every night (let's say a regular maintenance
|
and remote systems can only handle 6GB/hour of data transfer.
|
||||||
night goes from 10:00 pm to 6:00 am).
|
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
Now if cron launches Osync every night at 10:00 pm, and Osync is configured
|
Now if osync is scheduled every night at 10:00 pm, and it's configured to
|
||||||
to run for maximum 10 hours, it would stop at 6am, after having transferred
|
run for maximum 10 hours, it would stop at 6am, after having transferred
|
||||||
60GB.
|
60GB.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
|
@ -310,12 +364,14 @@ Then, on the next day, it would transfer the remaining 40GB from 10:00 pm
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
Also, if you run chained instances of Osync, one per user, you can decide
|
Also, if you run sequential instances of osync (with osync-batch), one per
|
||||||
how much time Osync would spend per user.
|
user directory, you can decide how much time osync should spend per user.
|
||||||
So if a user uploads too much data, Osync will keep running for that user
|
So if a user uploads too much data, and Osync cannot finish the synchronisation
|
||||||
for a given amount of time, and then runs for the next user so everyone
|
task for that user directory in a given timespan, it will stop that sync
|
||||||
will get it's data synced, regarless if the first user has uploaded too
|
task and run next user synchronisation task so every user sync task gets
|
||||||
much data.
|
run, regardless of the amount of data.
|
||||||
|
If there's time left, osync-batch reprograms the user sync task that has
|
||||||
|
been stopped.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Subsection
|
\begin_layout Subsection
|
||||||
|
@ -328,7 +384,7 @@ Osync's goal is to synchronize two directories, that can be hosted on the
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
The computer that runs Osync must hold at least one of the two directories,
|
The computer that runs osync must hold at least one of these two directories,
|
||||||
and will be called the
|
and will be called the
|
||||||
\emph on
|
\emph on
|
||||||
local system.
|
local system.
|
||||||
|
@ -375,6 +431,19 @@ slave replicas
|
||||||
.
|
.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
All system file configurations are meant for RedHat / CentOS style.
|
||||||
|
You may have to adapt them depending on your distribution.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
The dedicated sync user for all the examples is named
|
||||||
|
\emph on
|
||||||
|
syncuser
|
||||||
|
\emph default
|
||||||
|
.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Section
|
\begin_layout Section
|
||||||
Prerequisites
|
Prerequisites
|
||||||
\end_layout
|
\end_layout
|
||||||
|
@ -384,7 +453,7 @@ General packages
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
The following packages are needed on both local and remote systems.
|
The following packages are needed on both local and remote systems:
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
|
@ -400,6 +469,32 @@ rsync coreutils
|
||||||
\end_inset
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
Also, the local system will send emails on errors.
|
||||||
|
Make sure you have a mail package like mailx, mutt, postfix or sendmail
|
||||||
|
installed.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
Additionnaly, if you intend to run osync in daemon mode, you'll need the
|
||||||
|
following package.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
\begin_inset listings
|
||||||
|
inline false
|
||||||
|
status open
|
||||||
|
|
||||||
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
|
inotify-tools
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Subsection
|
\begin_layout Subsection
|
||||||
|
@ -442,7 +537,10 @@ status open
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
Be aware that ACLs are tricky and default group permissions serve as mask
|
Be aware that ACLs are tricky and default group permissions serve as mask
|
||||||
for ACLs.
|
for ACLs.
|
||||||
Make always sure you can read your data with your sync user:
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
Make always sure you can read /write to both replicas with your sync user:
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
|
@ -460,11 +558,20 @@ status open
|
||||||
$ cat /master/replica/test.file
|
$ cat /master/replica/test.file
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
|
$ touch /master/replica/othertest.file
|
||||||
|
\end_layout
|
||||||
|
|
||||||
\end_inset
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
Repeat that step for the slave replica.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Subsection
|
\begin_layout Subsection
|
||||||
\begin_inset CommandInset label
|
\begin_inset CommandInset label
|
||||||
LatexCommand label
|
LatexCommand label
|
||||||
|
@ -477,10 +584,17 @@ Performing superuser sync
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
Osync may be run as superuser, which should always be avoided by granting
|
Osync may be run as superuser, which should always be avoided by granting
|
||||||
the right permissions to a dedicated sync user.
|
the read / write permissions to a dedicated sync user to both replicas.
|
||||||
Nevertheless, there are some cases where running as superuser is necessary
|
\end_layout
|
||||||
to ensure backups of certain system files.
|
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
There are still some cases where osync needs to be run as superuser, especially
|
||||||
|
when syncing system files.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
In those cases, osync can be run as dedicated sync user and ask for sudo
|
||||||
|
permissions for specific commands.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
|
@ -555,7 +669,7 @@ status open
|
||||||
|
|
||||||
\begin_layout Plain Layout
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
# which rsync
|
# type rsync
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\end_inset
|
\end_inset
|
||||||
|
@ -575,7 +689,7 @@ status open
|
||||||
|
|
||||||
\begin_layout Plain Layout
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
Defaults:backupuser !requiretty
|
Defaults:syncuser !requiretty
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\end_inset
|
\end_inset
|
||||||
|
@ -584,8 +698,28 @@ Defaults:backupuser !requiretty
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
You should be aware that there is a minor risk with having rsync command
|
Once your standard sync user is granted to run what osync needs, you can
|
||||||
run as superuser.
|
enable sudo in osync's config file:
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
\begin_inset listings
|
||||||
|
inline false
|
||||||
|
status open
|
||||||
|
|
||||||
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
|
SUDO_EXEC=yes
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
You should be aware that there is a risk with having rsync command run as
|
||||||
|
superuser.
|
||||||
A user who can run rsync command as superuser can upload any file he wants
|
A user who can run rsync command as superuser can upload any file he wants
|
||||||
to the system, including a tweaked /etc/sudoers or /etc/passwd file.
|
to the system, including a tweaked /etc/sudoers or /etc/passwd file.
|
||||||
Please read chapter
|
Please read chapter
|
||||||
|
@ -602,6 +736,32 @@ reference "sub:Enhancing-remote-backup"
|
||||||
Remote sync
|
Remote sync
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
Osync can perform local or remote synchronization tasks.
|
||||||
|
For local sync, pelease refer to chapters
|
||||||
|
\begin_inset CommandInset ref
|
||||||
|
LatexCommand ref
|
||||||
|
reference "sec:Running-Osync-in"
|
||||||
|
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
,
|
||||||
|
\begin_inset CommandInset ref
|
||||||
|
LatexCommand ref
|
||||||
|
reference "sec:Running-Osync-with-config-file"
|
||||||
|
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
and
|
||||||
|
\begin_inset CommandInset ref
|
||||||
|
LatexCommand ref
|
||||||
|
reference "sec:Running-Osync-as-daemon"
|
||||||
|
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
Remote synchronization is done through an SSH tunnel.
|
Remote synchronization is done through an SSH tunnel.
|
||||||
To be able to establish such a tunnel without having to enter a password,
|
To be able to establish such a tunnel without having to enter a password,
|
||||||
|
@ -615,9 +775,12 @@ The private part is kept by the computer that initiates the connection,
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
The following steps will be required to generate a ssh key.
|
The following steps will be required to generate a ssh key:
|
||||||
Please create a dedicated sync user and log in as that user on the local
|
\end_layout
|
||||||
system to perform the following actions.
|
|
||||||
|
\begin_layout Standard
|
||||||
|
Create a dedicated sync user and log in as that user on the local system
|
||||||
|
to perform the following actions.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
|
@ -640,16 +803,7 @@ This should create two files named ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
The remote system should also have a dedicated sync user (
|
You should also create a dedicated sync user on the remote system.
|
||||||
\begin_inset Quotes eld
|
|
||||||
\end_inset
|
|
||||||
|
|
||||||
syncuser
|
|
||||||
\begin_inset Quotes erd
|
|
||||||
\end_inset
|
|
||||||
|
|
||||||
in this example).
|
|
||||||
Both local and remote users don't need to have the same name.
|
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
|
@ -684,12 +838,12 @@ status open
|
||||||
|
|
||||||
\begin_layout Plain Layout
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
# chmod 600 /home/syncuser/.ssh/authorized_keys
|
$ chmod 600 /home/syncuser/.ssh/authorized_keys
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Plain Layout
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
# chown syncuser:root /home/syncuser/.ssh/authorized_keys
|
$ chown syncuser:root /home/syncuser/.ssh/authorized_keys
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\end_inset
|
\end_inset
|
||||||
|
@ -739,7 +893,7 @@ Mail transport agent
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
You should make sure your system can send emails so Osync can warn you if
|
You should make sure your system can send emails so osync can warn you if
|
||||||
something bad happens.
|
something bad happens.
|
||||||
Osync will use mutt or mail command.
|
Osync will use mutt or mail command.
|
||||||
Please make sure you can send a test mail with at least one of the following
|
Please make sure you can send a test mail with at least one of the following
|
||||||
|
@ -774,7 +928,7 @@ Check your antispam if you don't get your message.
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
If you run on windows environment, please make sure you can launch sendemail.exe
|
If you run on windows environment, please make sure you can launch sendemail.exe
|
||||||
(found
|
by adding it to the %PATH% variable (found
|
||||||
\begin_inset CommandInset href
|
\begin_inset CommandInset href
|
||||||
LatexCommand href
|
LatexCommand href
|
||||||
name "here"
|
name "here"
|
||||||
|
@ -796,10 +950,10 @@ Enhancing remote system security
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
We may want to secure a password-less ssh access by removing non necessary
|
You may want to secure a password-less ssh access by removing non necessary
|
||||||
services offered by SSH.
|
services offered by SSH.
|
||||||
Edit the file ~/.ssh/authorized_keys created earlier and add the following
|
Edit the file ~/.ssh/authorized_keys created earlier on the remote system
|
||||||
line in the beginning of the file:
|
and add the following line in the beginning of the file:
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
|
@ -818,8 +972,8 @@ no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
Also, we may want to prevent any host except of our remote system to passwordles
|
Also, we may want to prevent any host except of our master replica system
|
||||||
s connect.
|
to passwordless connect.
|
||||||
Add the following line:
|
Add the following line:
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
|
@ -830,7 +984,7 @@ status open
|
||||||
|
|
||||||
\begin_layout Plain Layout
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
from=*.my.remote.servers.domain.tld
|
from=*.my.master.replica.server.domain.tld
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\end_inset
|
\end_inset
|
||||||
|
@ -870,7 +1024,7 @@ We may also restrict the ssh session to only a couple of commands we'll
|
||||||
\emph on
|
\emph on
|
||||||
ssh_filter.sh
|
ssh_filter.sh
|
||||||
\emph default
|
\emph default
|
||||||
that will only execute commands Osync might send.
|
that will only allow execution of commands osync needs.
|
||||||
Once again edit your authorized_keys file and add the following.
|
Once again edit your authorized_keys file and add the following.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
|
@ -941,7 +1095,7 @@ Now, only the commands
|
||||||
\begin_inset Quotes eld
|
\begin_inset Quotes eld
|
||||||
\end_inset
|
\end_inset
|
||||||
|
|
||||||
find, du, rsync, echo and sudo
|
find, du, rsync, echo, mv, mkdir and sudo
|
||||||
\begin_inset Quotes erd
|
\begin_inset Quotes erd
|
||||||
\end_inset
|
\end_inset
|
||||||
|
|
||||||
|
@ -968,7 +1122,7 @@ SUDO_EXEC=yes
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
Also, adding remote pre- and postexecution commands in your configuration
|
Also, adding remote pre- and postexecution commands in your configuration
|
||||||
files will not work if you use the ssh filter.
|
files will not work if you use the ssh filter.
|
||||||
You'll have to add your main command in ssh_filter.sh.
|
You'll have to add your optional commands in ssh_filter.sh.
|
||||||
Example if you want to perform remote snapshots you'll have to allow one
|
Example if you want to perform remote snapshots you'll have to allow one
|
||||||
of the following:
|
of the following:
|
||||||
\end_layout
|
\end_layout
|
||||||
|
@ -1022,7 +1176,8 @@ security by obscurity
|
||||||
|
|
||||||
and should generally not be the only security process, but makes any attack
|
and should generally not be the only security process, but makes any attack
|
||||||
harder.
|
harder.
|
||||||
First, let's create a symlink to rsync called let's say o_rsync
|
First, let's create a symlink to rsync called let's say o_rsync, on both
|
||||||
|
local and remote systems.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
|
@ -1032,7 +1187,7 @@ status open
|
||||||
|
|
||||||
\begin_layout Plain Layout
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
# ln -s $(which rsync) $(dirname $(which rsync))/o_rsync
|
# ln -s $(type rsync) $(dirname $(type rsync))/o_rsync
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\end_inset
|
\end_inset
|
||||||
|
@ -1066,13 +1221,24 @@ Also, edit RSYNC_EXECUTABLE value on any of your sync configuration files
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Section
|
\begin_layout Section
|
||||||
Running Osync in quicksync mode
|
\begin_inset CommandInset label
|
||||||
|
LatexCommand label
|
||||||
|
name "sec:Running-Osync-in"
|
||||||
|
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
Running osync in quicksync mode
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
You must first download a copy of osync to your computer, and make it executable.
|
You must first download a copy of osync to your computer, and make it executable.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
You may also use the included install.sh script which will put osync in /usr/loca
|
||||||
|
l/bin.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
\begin_inset listings
|
\begin_inset listings
|
||||||
inline false
|
inline false
|
||||||
|
@ -1080,7 +1246,7 @@ status open
|
||||||
|
|
||||||
\begin_layout Plain Layout
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
# chmod +x osync.sh
|
$ chmod +x osync.sh
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\end_inset
|
\end_inset
|
||||||
|
@ -1088,12 +1254,6 @@ status open
|
||||||
|
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
|
||||||
You might consider copying osync to
|
|
||||||
\emph on
|
|
||||||
/usr/local/bin
|
|
||||||
\end_layout
|
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
Then you might just launch it to sync two local dirs like this:
|
Then you might just launch it to sync two local dirs like this:
|
||||||
\end_layout
|
\end_layout
|
||||||
|
@ -1105,7 +1265,7 @@ status open
|
||||||
|
|
||||||
\begin_layout Plain Layout
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
# ./osync.sh --master=/path/to/dir1 --slave=/path/to/dir2
|
$ ./osync.sh --master=/path/to/dir1 --slave=/path/to/dir2
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\end_inset
|
\end_inset
|
||||||
|
@ -1134,13 +1294,13 @@ status open
|
||||||
|
|
||||||
\begin_layout Plain Layout
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
# ./osync.sh --master=/path/to/dir1 --slave=ssh://remoteuser@remotehost.com//path/t
|
$ ./osync.sh --master=/path/to/dir1 --slave=ssh://remoteuser@remotehost.com//path/t
|
||||||
o/dir2
|
o/dir2
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Plain Layout
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
# ./osync.sh --master=/path/to/dir2 --slave=ssh://remoteuser@remotehost.com:22//pat
|
$ ./osync.sh --master=/path/to/dir2 --slave=ssh://remoteuser@remotehost.com:22//pat
|
||||||
h/to/dir2 --rsakey=/home/user/.ssh/other_key
|
h/to/dir2 --rsakey=/home/user/.ssh/other_key
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
|
@ -1150,17 +1310,24 @@ h/to/dir2 --rsakey=/home/user/.ssh/other_key
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Section
|
\begin_layout Section
|
||||||
Running Osync with a full blown configuration file
|
\begin_inset CommandInset label
|
||||||
|
LatexCommand label
|
||||||
|
name "sec:Running-Osync-with-config-file"
|
||||||
|
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
Running osync with a full blown configuration file
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
Running Osync with a configuration will do just the same as in quicksync
|
Running osync with a configuration will do just the same as in quicksync
|
||||||
mode, except the fact that you have much more control of what's going on.
|
mode, except that you have much more control of what's going on.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
A sample configuration file is called sync.conf and is included with osync.
|
A sample configuration file is called sync.conf and is included with osync.
|
||||||
You may edit this file to fit your needs.
|
You may edit this file to fit your needs.
|
||||||
|
Basically configuration files should go to /etc/osync.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
|
@ -1179,7 +1346,7 @@ status open
|
||||||
|
|
||||||
\begin_layout Plain Layout
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
# ./osync.sh /path/to/my_sync.conf --dry --verbose
|
$ ./osync.sh /etc/osync/my_sync.conf --dry --verbose
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\end_inset
|
\end_inset
|
||||||
|
@ -1188,16 +1355,15 @@ status open
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
Osync should now enumerate which changes will be done on both sides.
|
Osync should enumerate which changes will be done on both sides.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
If everything worked out right, you might process the actual sync process.
|
If everything worked out right, you might process the actual sync process.
|
||||||
|
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
Remember that a full configuration file specifies a maximum execution delay.
|
A full configuration file specifies a maximum execution delay.
|
||||||
Initial sync tasks can take a huge amount of time depending on bandwidth
|
Initial sync tasks can take a huge amount of time depending on bandwidth
|
||||||
between replicas, in that case you might add parameter --no-maxtime to
|
between replicas, in that case you might add parameter --no-maxtime to
|
||||||
your first sync run so execution time won't be enforced.
|
your first sync run so execution time won't be enforced.
|
||||||
|
@ -1210,7 +1376,7 @@ status open
|
||||||
|
|
||||||
\begin_layout Plain Layout
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
# ./osync.sh /path/to/my_sync.conf --no-maxtime
|
$ ./osync.sh /etc/osync/my_sync.conf --no-maxtime
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\end_inset
|
\end_inset
|
||||||
|
@ -1234,29 +1400,34 @@ status open
|
||||||
|
|
||||||
\begin_layout Plain Layout
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
00 * * * * localsyncuser /usr/local/bin/osync.sh /home/localsyncuser/your_sync.con
|
00 * * * * syncuser /usr/local/bin/osync.sh /etc/osync/your_sync.conf --silent
|
||||||
f --silent
|
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\end_inset
|
\end_inset
|
||||||
|
|
||||||
You may find the sync log under /var/log/osync-version-your_sync.log or under
|
You may find the sync log under /var/log/osync-your_sync.log or under the
|
||||||
the current directory if /var/log is not writable.
|
current directory if /var/log is not writable.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Section
|
\begin_layout Section
|
||||||
Running Osync on file changes
|
\begin_inset CommandInset label
|
||||||
|
LatexCommand label
|
||||||
|
name "sec:Running-Osync-as-daemon"
|
||||||
|
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
Running Osync as deamon
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Subsection
|
\begin_layout Subsection
|
||||||
By hand
|
Manually
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
Osync may also run in file monitor mode.
|
Osync may also run in file monitor mode.
|
||||||
In this mode, osync monitors the master replica, and runs a synchronization
|
In this mode, osync monitors the master replica, and runs a synchronization
|
||||||
everytime the is file activity on master replica.
|
everytime the is file activity on master replica.
|
||||||
With this mode, you do not need cron anymore.
|
With this mode, you do not need a schedule anymore.
|
||||||
Be aware that only master replica is monitored, and slave replica sync
|
Be aware that only master replica is monitored, and slave replica sync
|
||||||
updates only occur when master replica modifications happen.
|
updates only occur when master replica modifications happen.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
@ -1268,7 +1439,7 @@ status open
|
||||||
|
|
||||||
\begin_layout Plain Layout
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
# ./osync.sh /path/to/my_sync.conf --on-changes
|
$ ./osync.sh /etc/osync/my_sync.conf --on-changes
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\end_inset
|
\end_inset
|
||||||
|
@ -1286,8 +1457,8 @@ If you plan to run Osync on a regular basis in file monitor mode, you might
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
From the directory you downloaded Osync, run the following to make Osync
|
From the directory you downloaded Osync, run the install.sh script and enable
|
||||||
a service:
|
the service.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
|
@ -1297,26 +1468,6 @@ status open
|
||||||
|
|
||||||
\begin_layout Plain Layout
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
# cp osync.sh /usr/local/bin
|
|
||||||
\end_layout
|
|
||||||
|
|
||||||
\begin_layout Plain Layout
|
|
||||||
|
|
||||||
# cp osync-srv /etc/init.d
|
|
||||||
\end_layout
|
|
||||||
|
|
||||||
\begin_layout Plain Layout
|
|
||||||
|
|
||||||
# mkdir /etc/osync
|
|
||||||
\end_layout
|
|
||||||
|
|
||||||
\begin_layout Plain Layout
|
|
||||||
|
|
||||||
# cp sync.conf /etc/osync
|
|
||||||
\end_layout
|
|
||||||
|
|
||||||
\begin_layout Plain Layout
|
|
||||||
|
|
||||||
# service osync-srv start
|
# service osync-srv start
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
|
@ -1363,7 +1514,15 @@ Quicksync only command line parameters
|
||||||
|
|
||||||
\begin_layout Labeling
|
\begin_layout Labeling
|
||||||
\labelwidthstring 00.00.0000
|
\labelwidthstring 00.00.0000
|
||||||
--slave Local or remote slave replica path.
|
--slave=
|
||||||
|
\begin_inset Quotes srd
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
|
\begin_inset Quotes srd
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
Local or remote slave replica path.
|
||||||
Can be a ssh uri like ssh://user@host.com:22//path/to/slave/replica (is
|
Can be a ssh uri like ssh://user@host.com:22//path/to/slave/replica (is
|
||||||
mandatory)
|
mandatory)
|
||||||
\end_layout
|
\end_layout
|
||||||
|
@ -1371,7 +1530,7 @@ Quicksync only command line parameters
|
||||||
\begin_layout Labeling
|
\begin_layout Labeling
|
||||||
\labelwidthstring 00.00.0000
|
\labelwidthstring 00.00.0000
|
||||||
--rsakey Alternative path to rsa private key for ssh connection to slave
|
--rsakey Alternative path to rsa private key for ssh connection to slave
|
||||||
replica
|
replica (if not ~/.ssh/id_rsa)
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Labeling
|
\begin_layout Labeling
|
||||||
|
@ -1738,6 +1897,27 @@ RSYNC_EXECUTABLE=rsync
|
||||||
\end_inset
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
Remote Rsync Executable path.
|
||||||
|
Don't change this unless your remote rsync binary isn't in the execution
|
||||||
|
path variable.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
\begin_inset listings
|
||||||
|
inline false
|
||||||
|
status open
|
||||||
|
|
||||||
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
|
REMOTE_RSYNC_PATH=""
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
|
@ -1906,6 +2086,27 @@ PRESERVE_HARDLINKS=yes|no
|
||||||
\end_inset
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
Do a full checksum on files instead of comparing file sizes and modification
|
||||||
|
times.
|
||||||
|
Enabling this will make sync tasks longer.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
\begin_inset listings
|
||||||
|
inline false
|
||||||
|
status open
|
||||||
|
|
||||||
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
|
CHECKSUM=yes|no
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
|
@ -2398,6 +2599,26 @@ You might try running osync as root to check if your problem is filesystem
|
||||||
You might add --verbose option to see what actually happens.
|
You might add --verbose option to see what actually happens.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
Also, running osync with the following command will give the exact commands
|
||||||
|
that actually happen:
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
\begin_inset listings
|
||||||
|
inline false
|
||||||
|
status open
|
||||||
|
|
||||||
|
\begin_layout Plain Layout
|
||||||
|
|
||||||
|
DEBUG=yes /usr/local/bin/osync.sh /etc/osync/my_sync.conf --verbose
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\end_inset
|
||||||
|
|
||||||
|
|
||||||
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Subsection
|
\begin_layout Subsection
|
||||||
Local-remote sync
|
Local-remote sync
|
||||||
\end_layout
|
\end_layout
|
||||||
|
@ -2452,6 +2673,25 @@ reference "sub:Enhancing-remote-backup"
|
||||||
user permission related.
|
user permission related.
|
||||||
\end_layout
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Section
|
||||||
|
Final words
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
The idea came in a discussion around a nice beer one evening.
|
||||||
|
It began as a project for a friend, who's company I was working for as
|
||||||
|
consultant.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
Today, osync is still used by this company, and some others around the globe.
|
||||||
|
\end_layout
|
||||||
|
|
||||||
|
\begin_layout Standard
|
||||||
|
I will try to provide technical help in my spare time for this project,
|
||||||
|
and appreciate every contribution i get on Github :)
|
||||||
|
\end_layout
|
||||||
|
|
||||||
\begin_layout Standard
|
\begin_layout Standard
|
||||||
\begin_inset CommandInset line
|
\begin_inset CommandInset line
|
||||||
LatexCommand rule
|
LatexCommand rule
|
Loading…
Reference in New Issue