Transfering logs with rsync and automating parsing

[Panther]

Here are some details on using rsync to transfer log files from a game server to the web server.

First, you need to create a script, such as /etc/utstatslogs:

Code: Select all

#!/bin/sh rsync --rsh=/bin/ssh --remove-source-files user@server.com:/usr/local/ut/UserLogs/Stats_*.log /var/http/ut/Logs/ cd /var/http/ut /usr/bin/php ./logs.php pass=<updatepass> nohtml

Then give it executable permission:

Code: Select all

chmod +x /etc/utstatslogs

Create a cron job in /etc/cron.d (this sets a 5 minute delay between runs, but can be changed):

Code: Select all

*/5 * * * * apache /etc/utstatslogs >/dev/null 2>&1

The above can also be used for automating stat parsing for logs already on the local server, just leave out rsync.
When using rsync to transfer the files, this setup expects key-based authentication through SSH between the servers. You create a private/public key pair on your webserver, such as:

Code: Select all

ssh-keygen -t ed25519 -a 100 -o -C "user@domain.com"

Then on the game server, you can do the following:

Code: Select all

mkdir .ssh chmod 700 .ssh nano -w .ssh/authorized_keys <paste publickey.pub> chmod 600 .ssh/authorized_keys

You'll need to manually connect once from the webserver to save the known_host ID, and also make sure the key pair exists in the user directory that will be handling the log sync, but that's basically it. You can also add restrictions in the authorized_keys to limit what the user can do, such as using this in place of just your key:

Code: Select all

command="/usr/bin/rrsync -OPTION /usr/ut/UserLogs",no-pty,no-agent-forwarding,no-port-forwarding,no-X11-forwarding <publickey>

When adding your public key to the .ssh/authorized_keys file, you include the entire line starting with "ssh-ed25519 " through the user@domain portion.