ShareDigest blog

• Execute a script at a specified time once a week, which peforms the following:
• Create a zipped-tar backup of the folder /backup/cpbackup/ at /backup/cpbackup_ddmmyy.tar.gz
• Clear the entire /backup/cpbackup/ folder of all contents, freeing up this space
• Email the administrator that the new backup_x.tar.gz file has been created
• (Optionally): Automatically copy this file over to another (larger capacity) server

Step 1: Install A Cron Task

A great little tutorial showing exactly how easy it is to install a cron task on any linux system, can be found here.

Step 2: Create the Shell Script

The script which performs the magic, looks like this:

#!/bin/sh
# Create a zipped-tar backup of the folder /backup/cpbackup/ at /backup/cpbackup_ddmmyy.tar.gz
# Clear the entire /backup/cpbackup/ folder of all contents, freeing up this space
# Email the administrator that the new backup_x.tar.gz file has been created
# (Optionally): Automatically copy this file over to another (larger capacity) server
 
EMAIL="youremail@gmail.com"
NOW=$(date +"%d-%b-%y_%H%M")
BFILE="/backup/wz_cbackup_$NOW.tar.gz"
MSG=""
SUBJECT=""
 
#create tar zip file:
tar -czf $BFILE /backup/cpbackup/
 
if [ "$?" -ne "0" ]; then
	MSG="tar -czf $BFILE /backup/cpbackup/ failed."
	SUBJECT="weeklyZip: $BFILE creation failed!"
else
  	MSG="$BFILE successfully created. Please either download or copy to new server."
  	SUBJECT="weeklyZip: $BFILE  successfully created."
 
	#Success, so we clear out the /backup/cpbackup/ folder:
	rm -R /backup/cpbackup/* -f
fi
 
/bin/mail -s "$SUBJECT" "$EMAIL" <<< "$MSG"

Create And Import Into a New Subversion (SVN) Repository:
Replace ‘$repos_name’ with the name of your SVN repository – typically the name of the project folder.
(Note: For further repository admin functions and general SVN admin, see here).

$ svnadmin create /var/svn/$repos_name
 
#Note: To remove a repository, simply delete the directory directly as root user: (don't do this now unless you want to remove it :-):
sudo rm -R /var/svn/$repos_name

The recommended organization for a repository is to use the folders trunk, branches and tags, which we create using:

$ svn mkdir file:///var/svn/$repos_name/trunk
$ svn mkdir file:///var/svn/$repos_name/branches
$ svn mkdir file:///var/svn/$repos_name/tags

Now let’s import our project folder to this repository’s ‘trunk’ (main development line) folder): We do this the FIRST TIME we want to start tracking a project in the repository.

$ svn import localProjFolder file:///var/svn/$repos_name/trunk

Checking Out a Folder From a Repository:

Before having a local ‘working copy’ we need to check out the project folder for the first time:

$ svn checkout file:///var/svn/$repos_name/trunk {optional: localProjFolder}

Now we have a local working copy our project. We can make whatever changes we like, as in a usual file system structure. Note that any changes, APART FROM A STRAIGHT MODIFICATION TO A FILE (using a text editor for example), needs to be preceded by an ’svn’ to the command. For example, ’svn mkdir’ needs to be run to add a directory to the project. This is because SVN needs to be made aware of these changes.

Editing A File

Let’s say I have edited the file SecNew. java. We can now check the status of scheduled changes to our folder by running:

$ svn status
M      SecNew.java

In the above, we can see that the file SecNew.java was modified (in the working copy), and is due for modification in the repository.

Add A Folder To Our SVN Project

Let’s now add a folder to our project. I am going to add ‘custom’ to the folder SDCommon/src/org/:

$ svn mkdir ~/workspace/SDCommon/src/org/custom
A      custom

Note how I used ’svn’ before the usual ‘mkdir’ command. Remember, only pure editing of files does NOT require this preceding ’svn’ to be used.
The last line shows us that the folder was successfully created and is scheduled to be (A)dded to the repositoy.

Seeing What Changes We’ve Made To Our Folder: svn status & svn diff

Simply run the ’svn status’ command:

$ svn status
M      SecNew.java
A      custom

We can see the changes that are scheduled to be made to the repository on a commit.

Let’s now check out what specific changes have occurred to files that have been modified (SecNew.java in this case):

pacific@mainbox:~/workspace/SDCommon/src/org$ svn diff
 
Index: SecNew.java
===================================================================
--- SecNew.java (revision 6)
+++ SecNew.java (working copy)
@@ -5,6 +5,8 @@
 import org.apache.log4j.Logger;
 
 public class Security { //can be overridden if need be
+       protected DateTime StartDate = null;
+
        protected String ticker = null;
        protected String exchcode = null; //yahoo ec
        protected String name = null;

Lines that have a ‘+’ next to them, have been added. (a ‘-’ indicates lines that have been removed).

So you’d like to host a SVN repository which allows fine-grained access control and different users and projects? Look no further, as we’ll be covering how to achieve just that in this post. What we’ll be using is the Apache Web server (with SSL encryption), SubVersion and Trac which is a web-based software management package. I’ll be doing this on my Kubuntu Hardy Heron (Linux debian) machine, so if you’re installing it under a different version / OS instructions may vary slightly.

Note: You will need sudo access rights and a basic text editor (such as Kate) to do these.

Step 1: Install Apache httpd

This is as simple as installing the ‘apache2′ package using apt-get or aptitude:

$ sudo aptitude install apache2

For further detail on the installation of the Apache web server, see this.

Step 2: Install Subversion & Apache SVN Libraries
Similar to step 1, to install the subversion and the svn libraries for apache, simply run:

$ sudo aptitude install subversion libapache2-svn

Step 3: Enable SSL
First we enable the Apache ssl module, followed by adding ‘Listen 443′ to the Apache2 port configuration file (see below):

$ sudo a2enmod ssl
$ sudo kate /etc/apache2/ports.conf

After opening the ports.conf file, make sure that the following is in the file:

<IfModule mod_ssl.c>
        Listen 443
</IfModule>

Step 4: Generate SSL Certificate
Once we have enabled the SSL module, we need to generate a certificate to use:

#create dir where ssl certificates will go:
$ sudo mkdir /etc/apache2/ssl  
 
#create certificate (should launch application asking some basic questions)
$ sudo /usr/sbin/make-ssl-cert /usr/share/ssl-cert/ssleay.cnf /etc/apache2/ssl/apache.pem

Step 5: Create a Virtual Host

#copy default folder to svnserver folder
$ sudo cp /etc/apache2/sites-available/default /etc/apache2/sites-available/svnserver
 
#Open svnserver:
$ sudo kate /etc/apache2/sites-available/svnserver

Now, within the svnserver file, change the following:

"NameVirtualHost *" to "NameVirtualHost *:443"
"<VirtualHost *>"  to  <VirtualHost *:443>"

Directly under ‘ServerAdmin’, add the following:

SSLEngine on
SSLCertificateFile /etc/apache2/ssl/apache.pem
SSLProtocol all
SSLCipherSuite HIGH:MEDIUM

Step 6: Enable The Site

$ sudo a2ensite svnserver
$ sudo /etc/init.d/apache2 restart
#To avoid warnings when restarting apache2, add "ServerName $your_server_name" to /etc/apache2/apache2.conf

Step 7: Add Repositories
To host more than one repository, use the following configurations, replacing $REPOS with the name of your repository-folder, for example I’ve used ’svn-repos’

$ sudo mkdir /var/svn
$ sudo svnadmin create /var/svn/$REPOS
$ sudo chown -R www-data:www-data /var/svn/$REPOS
$ sudo chmod -R g+ws /var/svn/$REPOS

Step 8: Add Authenticated User
Add a user using the following, substituting $username with your selected username. You’ll be prompted for a password.

$ sudo htpasswd -c -m /etc/apache2/dav_svn.passwd $username

Step 9: WebDAV and SVN Configuration
Open the file: /etc/apache2/mods-available/dav_svn.conf using Kate, and either enable (uncomment) or add the following:

<Location /svn>
DAV svn
SVNParentPath /var/svn/
AuthType Basic
AuthName "Subversion Repository"
AuthUserFile /etc/apache2/dav_svn.passwd
Require valid-user
SSLRequireSSL
</Location>

Now restart the Apache2 webserver one final time:

$ sudo /etc/init.d/apache2 restart

Step 10: Test It !
Point your browser to: https://localhost/svn/$REPOS (where $REPOS is your specific svn repository)
You should now see something like “Revision 0: /” appear – this is our first SVN repository contents. Obviously it is empty, so let’s get right stuck into
testing it out with some SVN!

Recursively Remove All Files With a Specific Extension:

Suppose you want to delete, recursively, all files with a specific file extension (e.g. a Java folder containing .class files). Run the following command:

$ find folderToStart/ -name \*.extension -exec rm {} \;

Bulk Renaming All Files Within a Folder

To replace a string within all files within a folder, just run the following:

$ for i in *; do newname=`echo $i|sed 's/search_txt/replace_txt/g'`; echo $newname; mv $i $newname; done

For more handy applications to bulk renaming using regular expressions, see this.

Check Folder Sizes on System

Very handy tutorial on how to do all aspects of disk space usage, can be found here.

If we want to check if a folder exists we simply create the following .sh file:

#!/bin/bash
 
if [ ! -d "myFolder" ]; then
	echo "Creating myFolder"
	sudo mkdir myFolder
fi
 
if [ ! -d "myFolder/subFolder" ]; then
	echo "Creating myFolder/subFolder"
	sudo mkdir myFolder/subFolder
fi

Now you can run that using sudo and for example create a new folder where one didn’t exist before, if its required by some other script, for example one mounting a windows XP drive over a network to the current Linux filesystem using Samba. See the post covering that for more details.