KIT501 UNIX Scripting Assignment

KIT501 UNIX Scripting Assignment

Due Date: 3:00pm Thursday the 12^th of May – Week 11

Weight: 15% of your final KIT501 result

Intended Learning Outcomes: ILO 2 – Apply industry standard techniques and skills to configure network infrastructure and operating system processes.

Introduction

This assignment requires you to apply what you have learned from the unit’s practical classes to write a script to implement task specified below. You must complete this assignment individually and pay careful attention to the section on plagiarism to ensure the work you submit is your own.

Working environment

On the teaching server, ictteach.its.utas.edu.au, you must make a directory named kit501script in your home directory and use this directory as your working directory for all assignment development.

1. The first time you start working on the assignment, after logging on, type the following commands ($ is the prompt, do not type that):

$ mkdir kit501script

$ cd kit501script

• Every other time you log on, simply do the following and then continue working:

$ cd kit501script

Allowed Syntax/Commands

All scripts must be completed using syntax and commands discussed in past (and some future) practical classes – no additional referential syntax (that is not discussed in practicals) is allowed. This means your script solutions must not use alternative constructs and syntax e.g. solutions found through websites via google searches that use commands and shell syntax that we have not covered in the practical content, and you must not use external parties or other individuals to write the scripts for you (this is considered plagiarism and academic misconduct).

1

Assessment Process

All student script submissions will be marked by KIT501 tutors on ictteach according to the rubric on the last page of this document. If your scripts were developed elsewhere (i.e., not on ictteach) it’s your responsibility to ensure that your scripts run correctly in our teaching environment. You will lose considerable marks if your scripts do not run correctly on our teaching server, so it is imperative you develop and test your scripts on ictteach before you submit.

Scripting Task

Overview

A researcher has used a logging program to log observations for an experiment. The output of the logging program is a series of log files in a directory that have the following naming format:

fileABCDEFGH

where file is literally the word ‘file’, and A..H are single binary digits (0 or 1).

The researcher did not realise for several weeks that the logging program’s output filenames had binary digits, and, unfortunately, her further research work requires the filenames to be fixed to replace the binary part with the decimal number. The researcher also mistakenly added other files to the directory containing the log files, and these other files do not have the correct naming format. She wants you to write a script to rename the validly-named files, and then sort all the files in her directory into one of four subdirectories depending on their filename – the script must also output what files (and their associated ‘category’) are found.

Valid filenames

Any of the following would be considered potentially valid filenames produced by the researcher’s logging program (all categorised as CATEGORY1) – a filename that starts with file followed by 8 binary digits:

• file00000000
• file00000001
• file00000010
• …
• file11111111

Invalid filenames

Anything not fitting the pattern described above is invalid.

• CATEGORY2 – filenames that start with file that are followed by zero or more binary digits only (but less than 8 binary digits) e.g. file, file0101, file0000000
• CATEGORY3 – filenames that start with file that are followed by any characters that include non-binary digits e.g. filesort0101, file00112110, fileWRONG
• CATEGORY4 – any filename that does not start with file e.g. badfile, fire01001010

The researcher’s request means your script must require an arbitrary directory name to be specified when it is run, after which it will then rename any valid filenames and move the files it finds in the specified directory according to the requirements that follow:

Task Requirements

1. Your script for this task must be named binarylog.sh

If your script has a different name, it will not be assessed.

• Make sure your script is not unnecessarily complex – your script should use consistent indentation and include whitespace/blank lines (where relevant) to make the script more logical and easier for a person to read. You must also include general inline comments in the script code to indicate the purpose of more complex command combinations etc.

• Your script must start with the following first line:

#!/bin/sh

(this specifies the shell to be used to interpret the rest of the commands in the script)

• Your script must include comments at the beginning (near the top of the script) to specify:
  • the script’s purpose,
  • the author (including student number),
  • and the date (when last modified).

• One directory name must be supplied as an argument to be used by your script when it is executed by the user (the directory referred to would presumably be one that contains a mixture of validly named and invalidly named log files that are going to be processed by the script). The directory name must not be ‘hardcoded’ (written) in your script, i.e., its value must come from the command line argument. If no directory name is provided by the user, the script should provide a usage warning message and then exit. See Example Output for an example.

• Near the beginning of your script, you need to check that the directory associated with the directory name provided as an argument to the script:

1. Exists. If the directory does not exist, your script must display an error message and then it should exit. The directory name provided can use an absolute path, or a relative path – see Example Output for examples.

• Is readable, writeable and
  • If the directory is not readable, your script must display an error message and then it should exit.
  • If the directory is not writeable, your script must display an error message and then it should exit.
  • If the directory is not executable, your script must display an error message and then it should exit.
  executable.

• For every file in the specified directory, the script must:
  • For a validly named file:

echo the following output to the terminal screen:

1. Rename the file with its specific filedecimal name and then move it to a

subdirectory called CATEGORY1 (the script should only create the directory here if it does not already exist and also only when a file in this category is found). See the section Binary to Decimal Renaming below.

• For an invalidly named file that starts with file and is then followed by less than 8 binary digits (and no other
  characters):

echo the following output to the terminal screen:

1. Move the invalid file to a subdirectory called CATEGORY2 (the script should

only create the directory here if it does not already exist and also only when a file in this category is found).

• For an invalidly named file that starts with file and is followed by any characters that include some non-binary
  digits:

echo the following output to the terminal screen:

1. Move the invalid file to a subdirectory called CATEGORY3 (the script should

only create the directory here if it does not already exist and also only when a file in this category is found).

• For an invalidly named file that does not start with file:

echo the following output to the terminal screen:

1. Move the invalid file to a subdirectory called CATEGORY4 (the script should only create the directory here if it does not already exist and also only when a file in this category is found).

• If the command-line argument specified directory contains no files:
  • echo the following output to the terminal screen:

• exit the script

This also means if you run the script twice, the first time should process any files, displaying the outputs specified in requirement 7 above, renaming where appropriate, and moving files to specified category subdirectories. Running the script for a second time (with the same directory argument) will produce the output specified in this requirement (requirement 8), as all files should have already moved to subdirectories. Your output should never “process” or mention the subdirectory names.

Binary to Decimal Renaming

As previously mentioned, a validly named file will have the form fileABCDEFGH, eg file01010111

Your script must implement an algorithm only using commands and syntax as discussed in practical classes. Your solution is restricted (i.e not allowed) to use the shell “builtin” conversion

e.g. echo $((2#101010101))

This restriction also extends to any other command that has not been discussed in classes – the following list shows some of the common methods (but not all) found through a brief google search – none are permitted:

awk sed printf xxd od perl ibase, obase and bc

Instead, your solution must take an iterative (looping) approach to demonstrate your understanding of iteration and selecting individual characters from a string (a string is just a sequence of characters).

Algorithm hint

The suggested approach is you take the binary part from the filename (what command can extract part of a string?), and then examine each binary digit one by one in turn (the syntax to do this is given in a practical) – the digit’s position in the binary string will determine the value (a multiple of 2) to add to the decimal total. Note the syntax for the character position starts numbering from zero, and from the left hand side.

For example, consider the binary value in the string “01001001”

Column Value	128	64	32	16	8	4	2	1
String Position	0	1	2	3	4	5	6	7
String Value	0	1	0	0	1	0	0	1

Assume you start in position 0 of the string, and total is 0.

In position 0, the bit value is 0, so add 0*128 to the total (total still equals 0)

Next, in position 1, the bit value is 1, so add 1*64 to the total (total now equals 64) Next, in position 2, the bit value is 0, add 0*32 to the total (total still equals 64) Next, in position 3, the bit value is 0, so add 0*16 to the total (total still equals 64) Next, in position 4, the bit value is 1, so add 1*8 to the total (total now equals 72) Next, in position 5, the bit value is 0, so add 0*4 to the total (total still equals 72) Next, in position 6, the bit value is 0, so add 0*2 to the total (total still equals 72) Finally, in position7, the bit value is 1, so add 1*1 to the total (total now equals 73)

When implementing your solution, you may consider converting the 8-bit binary portion of a filename to either:

1. its positive decimal equivalent (as demonstrated above), or
2. for a higher grade, assume the 8-bit value is using twos complement representation (see criterion 4 in the marking rubric)

8-bit positive binary values

Assuming you implement conversion of 8-bit positive numbers only, the following would be valid conversions:

8-  bit twos complement binary values

Assuming you implement conversion of 8-bit twos complement number representation, the following would be valid conversions:

Example output

Example files

You of course are free to create any directory you like in your kit501script directory and populate it with files for testing purposes (this is highly recommended – you need to test your script works correctly!). The marking process will not use the filenames or directory name that are listed below, marking will use a different directory and different testing files.

In this example, assume a subdirectory called research has been created inside kit501script. The contents of research are the following files:

The following are sample outputs of the script you must develop. The text up to and including the $ is the shell prompt:

Example 1. No directory name specified as an argument:

[username@ictteach kit501script]$ ./binarylog.sh Usage: ./binarylog.sh dirname

where dirname is a directory containing binary-named files

Example 2. Invalid directory name specified as an argument:

[username@ictteach kit501script]$ ./binarylog.sh reserch Directory reserch does not exist!

Example 3. Valid directory name specified as an argument, but (e.g.) is not readable:

[username@ictteach kit501script]$ ./binarylog.sh research Directory research is not readable!

Example 4. If the specified directory contained no files:

[username@ictteach kit501script]$ ./binarylog.sh research The directory research contains no files…

Example 5. Normal output with mixture of valid/invalid files:

[username@ictteach kit501script]$./binarylog.sh research

invalid  – badfile                           does not start with ‘file’ valid               – file00000001                 [renamed to file1] invalid  – file0101   does not have 8 bits

valid       – file01111111               [renamed to file127]

invalid – file01whoops                   non-binary characters after ‘file’ valid               – file10000001                 [renamed to file-127]

invalid  – file1111111                    does not have 8 bits valid               – file11111111                 [renamed to file-1]

invalid – filefilefile                 non-binary characters after ‘file’

Example 6. Example contents of specified directory (research in this example) after the script has run:

[username@ictteach kit501script]$ ls research/* research/CATEGORY1:

file1     file-1     file127    file-127

research/CATEGORY2: file0101       file1111111

research/CATEGORY3: file01whoops     filefilefile

research/CATEGORY4: badfile

Submitting Your Assignment

All assignment submissions are through MyLO. Submitting your assignment to MyLO requires you to first make a compressed copy of your script file on ictteach, copy that compressed file from ictteach to your local computer, and then upload the compressed file from your local computer to the MyLO submission area. Please read and follow the instructions below carefully

– ask your tutor if you have any difficulty, and we suggest you try the instructions well-ahead of the due date so you do not encounter any last minute problems.

1.     Start on ictteach

You must first create a compressed¹ version (a copy) of your assignment script on ictteach, and then copy this version to your local computer:

On ictteach, run the following commands ($ is the prompt):

$ cd ~/kit501script

$ gzip -c binarylog.sh > binarylog.sh.gz

You then need to copy the binarylog.sh.gz file to your local computer – instructions on how to do this differ depending on what local operating system your local computer is using

– see below. The instructions for each operating system use a command called scp (secure copy), which uses the SSH protocol behind the scenes to copy files.

1 MyLO will not permit files to be uploaded with certain file extensions – .sh is one of them. Instead, we compress the script file (to preserve all formatting) and the result will have a .gz extension which MyLO will accept.

2.     Apple macOS (using terminal)

If you are using an Apple macOS-based computer to submit, copy your compressed script file to the local computer via:

1. From a new terminal window (i.e. not a terminal session already connected to

ictteach), type the following (% is the prompt, do not type that)

% scp username@ictteach.its.utas.edu.au:kit501script/binarylog.sh.gz

~/Desktop

(replace username with your own UTAS username (this is NOT your email address!))

• You should then be prompted for your ictteach password:

username@ictteach.its.utas.edu.au’s password:

• After typing your password (it will not be shown onscreen), if you have correctly authenticated, you should get a notification line like the following to indicate the file has been downloaded (to your Desktop in this instance – the file size and data transfer rate shown will differ to your file size and data transfer rate )

binarylog.sh.gz                      100% 3727          2.6MB/s      00:00

3.     Microsoft Windows (using putty)

If you are using your own Microsoft Windows-based computer to submit, we assume you have previously installed the putty program to access ictteach. Putty also includes some other programs when you install it – pscp.exe is one of them and it is the program needed. If you are not using putty but have been using some other access method, you will need to investigate how to use the scp command for that method.

Copy your compressed script file to the local computer via:

1. Start a new command prompt window (select the Windows start menu icon and then type cmd.exe).

• In the command prompt window, use the following command to navigate to the putty local installation directory (in the example, putty is installed in a putty subdirectory of the standard location, c:\Program  Files  –  your local putty directory may differ). (C:\Users\username>  is the prompt, do not type that):

Microsoft Windows [Version 10.0.19042.1110]

(c) Microsoft Corporation. All rights reserved.

C:\Users\username>  cd  c:\program  files\putty

• If successful, the current directory should now be the putty install directory. pscp.exe is putty’s version of the scp command, and it should be installed here. Run the following command (C:\Program  files\PuTTY> is the prompt, do not type that):

C:\Program  files\PuTTY>  .\pscp.exe

username@ictteach.its.utas.edu.au:kit501script/binarylog.sh.gz

%userprofile%\Desktop

(Replace username with your UTAS username (this is NOT your email address))

• You will be prompted for your ictteach password (the password will not appear on screen):

username@ictteach.its.utas.edu.au’s password:

If you have typed the commands correctly, your compressed script file, binarylog.sh.gz

should now be on your Windows Desktop.

4.     MyLO submission (all students):

If you have successfully copied your compressed binarylog.sh.gz script file from ictteach to your local computer, it should now be on your desktop. You can now submit this file to MyLO:

1. In a web browser, go to the unit’s MyLO site – in semester 1 2022 this is https://mylo.utas.edu.au/d2l/home/506348).
2. Navigate through the MyLO site’s top menu to the submission link:

Assessments → Assignments → Assignment 2 (scripting)

• Scroll down and select the Add a File button.
• Chose the My Computer option, and then choose the Upload button. Navigate to your local desktop and select the binarylog.sh.gz file that you copied earlier and finally choose Open.
• Back in the Add a File dialog window, choose the Add button at the bottom.
• Then, if everything appears to be ok (verify you can see your file listed) choose the Submit

button.

Revised submission?

If it is before the due date and time, follow all the submission steps again to make a compressed copy of your script, download the copy to your computer and then upload your copy to MyLO. We will only assess the latest submission.

Late submission?

If your assignment is late then you should submit your compressed script file to MyLO as above – the University’s new assessment policy means you will lose 5% of your score for every day the submission is late – assignments will not be accepted after 10 days past the scheduled submission date.

Need Help?

You are encouraged to seek assistance from your lecturer or tutors in practicals or during consultation after you have seriously thought about the assignment. Please note that we can provide general advice – but you are expected to write and debug (correct) your own code.

When writing your script, think about what steps you need to do, don’t try to write the code all at once. Implement a small part first, test that it is working correctly, then implement a bit more, repeating the process of thinking, writing (coding), testing and refining/fixing. Do not leave the development of your script too late – you will run out of time!

Extra Hints:

Worksheets for Practicals 7 – 10 will be released early for those students who want to start work on this assignment early. For example, some referential syntax needed (e.g., looping) is not covered in the practical material until Practical 9.

• Start by determining the provided directory’s name from the script’s argument parameters (see Practical 7)
• Verify the directory exists – exit if it doesn’t (see Practical 8)
• Verify the directory is readable/writeable/executable – exit if it isn’t (see Practical 8)
• Work out how to list the contents of the directory, one file at a time – use a loop (see Practical 9)

• Work out how to ‘categorise’ a filename into one of the four categories – use a case statement (see Practical 8)
  • You’ll need to determine if a filename starts with file, and then also if it only has binary digits following, or if it also has other characters. (see Practical 5)
  • The binary part (if it exists) will always be in the same positions (the 5^th to 12^th characters) of the filename! (see Practical 6)
  • To work out how many characters a string has, there was some syntax mentioned in Practical 8.
  • You will have to use syntax for decision-making – use the if statement (see Practical 8)
  • Simple arithmetic (e.g. addition and multiplication) is covered in Practical 8.
• When testing your script it’s recommended you “clean up” any temporary files or directories already created so you can verify your script correctly creates directories when required. You might like to write another simple setup script that you can keep quickly reuse to recreate your testing log directory and populate it with test files prior to each time you try to run your assignment script. A reminder – the touch command can be used to quickly create an empty file if it doesn’t already exist e.g. touch file00000001

Plagiarism

Plagiarism is a very serious matter, and ignorance of this is not an excuse. If you submit work claiming it to be your own, and it is not original work of your own, this is cheating. This means for example that you cannot submit sections of code that have been written by other students or sourced from the Internet and claim you wrote the code yourself. Plagiarism can result in academic penalties both in relation to your assignment, and also on your permanent university record.

As an example, you cannot take code written by someone else and change variable names, comments and spacing to make it appear you wrote the code – this would still be considered plagiarism.

KIT501 Scripting Assignment Marking Rubric

Criteria	HD	DN	CR	PP	NN
1 Script Execution	The script runs without any	The script runs without any	The script runs without	The script runs with	The script does
(5%)	syntax errors or unusual output	syntax errors, but some	any syntax errors, but	very minor syntax	not run, or it has
		minor unusual output occurs	some significant unusual	errors that do not	significant
			output occurs	prevent it from	syntactic errors,
				continuing	or the script
					contents are
					trivial
2 Log source	The source directory is provided	The source directory is	The source directory is	The source	The source
directory (10%)	as a command line argument, it	provided as a command line	provided as a command	directory is	directory is not
	is verified to exist, and all of the	argument, it is verified to	line argument, it is verified	provided as a	provided as a
	directory permissions are	exist, and some of the	to exist, but none of the	command line	command line
	correctly verified	directory permissions are	directory permissions are	argument, but no	argument
		correctly verified	correctly verified	verification is	(and/or it is
				performed for	hardcoded in the
				existence or	code)
				permissions
3 Log source	Script correctly outputs when	n/a	n/a	Script correctly	Script does not
directory	the directory is completely			outputs when the	correctly output
contents (5%)	empty, and correctly outputs			directory is	when a directory
	when the directory contains no			completely empty	contains no files
	files (but has subdirectories).
	Only syntax and commands
	discussed in classes have been
	used
4 Binary	The script correctly converts all	The script correctly converts	n/a	The script correctly	No binary to
Conversion (40%)	valid filenames to their decimal	all valid filenames to their		converts all valid	decimal
	equivalent, correctly converts	decimal equivalent, correctly		filenames to their	conversion is
	values using twos complement	converts values using		decimal equivalent,	correctly
	representation and displays the	positive binary		correctly converts	displayed
	result in the output. Only syntax	representation and displays		values using
	and commands discussed in	the result in the output. Only		positive binary
	classes have been used	syntax and commands		representation and
		discussed in classes have		displays the result
		been used		in the output.
5 invalid files	All Invalid files are correctly	All Invalid files are correctly	Some Invalid files are	Some Invalid files	No invalid files in
categorisation	identified in the output, and they	identified in the output, and	correctly identified in the	are correctly	this category are
(20%)	are moved to the corresponding	they are moved to the	output, and they are	identified in the	correctly
	category subdirectory. If not	corresponding category	moved to the	output, but they	identified, nor
	initially present, the	subdirectory. The	corresponding category	are not moved to	are they moved
	corresponding category	corresponding category	subdirectory, but some	the corresponding
	subdirectory is only created	subdirectory is created even	files are misclassified	category
	when and if files in this category	if no files in the category are		subdirectory, and
	are discovered. Only syntax and	found. Only syntax and		some files are
	commands discussed in classes	commands discussed in		misclassified
	have been used	classes have been used
6 Script	Informative script comments are	Informative script comments	Informative script	Informative script	No comments or
Comments (10%)	included at the start, as well as	are included at the start, and	comments are included at	comments are	only extremely
	throughout the script (where	many other comments are	the start, but there are	included at the	sparse or
	appropriate)	included, but some	few other comments –	start, but very	inappropriate
		comments are obviously	there is an obvious need	limited comments	comments are
		missing	for other comments	elsewhere	included
7 Script Layout	Excellent, consistent use of	Excellent use of indentation	Good use of indentation	Occasional, correct	No attention
(10%)	indentation and whitespace	and whitespace however	and whitespace but a few	use of indentation	provided to
	throughout	there are some minor	sections are inconsistent	and whitespace but	layout, major
		examples of inconsistency		many sections are	lack of
				inconsistent	consistent use of
					indentation and
					whitespace

Note for criteria 1, 6 and 7 grades above NN are only applicable if the script contents are non-trivial

14