KIT501 UNIX Scripting Assignment
KIT501 UNIX Scripting Assignment
Due Date: 3:00pm Thursday the 12th of May – Week 11
Weight: 15% of your final KIT501 result
Intended Learning Outcomes: ILO 2 – Apply industry standard techniques and skills to configure network infrastructure and operating system processes.
Introduction
This assignment requires you to apply what you have learned from the unit’s practical classes to write a script to implement task specified below. You must complete this assignment individually and pay careful attention to the section on plagiarism to ensure the work you submit is your own.
Working environment
On the teaching server, ictteach.its.utas.edu.au, you must make a directory named kit501script in your home directory and use this directory as your working directory for all assignment development.
- The first time you start working on the assignment, after logging on, type the following commands ($ is the prompt, do not type that):
$ mkdir kit501script
$ cd kit501script
- Every other time you log on, simply do the following and then continue working:
$ cd kit501script
Allowed Syntax/Commands
All scripts must be completed using syntax and commands discussed in past (and some future) practical classes – no additional referential syntax (that is not discussed in practicals) is allowed. This means your script solutions must not use alternative constructs and syntax e.g. solutions found through websites via google searches that use commands and shell syntax that we have not covered in the practical content, and you must not use external parties or other individuals to write the scripts for you (this is considered plagiarism and academic misconduct).
1
Assessment Process
All student script submissions will be marked by KIT501 tutors on ictteach according to the rubric on the last page of this document. If your scripts were developed elsewhere (i.e., not on ictteach) it’s your responsibility to ensure that your scripts run correctly in our teaching environment. You will lose considerable marks if your scripts do not run correctly on our teaching server, so it is imperative you develop and test your scripts on ictteach before you submit.
Scripting Task
Overview
A researcher has used a logging program to log observations for an experiment. The output of the logging program is a series of log files in a directory that have the following naming format:
fileABCDEFGH
where file is literally the word ‘file’, and A..H are single binary digits (0 or 1).
The researcher did not realise for several weeks that the logging program’s output filenames had binary digits, and, unfortunately, her further research work requires the filenames to be fixed to replace the binary part with the decimal number. The researcher also mistakenly added other files to the directory containing the log files, and these other files do not have the correct naming format. She wants you to write a script to rename the validly-named files, and then sort all the files in her directory into one of four subdirectories depending on their filename – the script must also output what files (and their associated ‘category’) are found.
Valid filenames
Any of the following would be considered potentially valid filenames produced by the researcher’s logging program (all categorised as CATEGORY1) – a filename that starts with file followed by 8 binary digits:
- file00000000
- file00000001
- file00000010
- …
- file11111111
Invalid filenames
Anything not fitting the pattern described above is invalid.
- CATEGORY2 – filenames that start with file that are followed by zero or more binary digits only (but less than 8 binary digits) e.g. file, file0101, file0000000
- CATEGORY3 – filenames that start with file that are followed by any characters that include non-binary digits e.g. filesort0101, file00112110, fileWRONG
- CATEGORY4 – any filename that does not start with file e.g. badfile, fire01001010
The researcher’s request means your script must require an arbitrary directory name to be specified when it is run, after which it will then rename any valid filenames and move the files it finds in the specified directory according to the requirements that follow:
Task Requirements
- Your script for this task must be named binarylog.sh
If your script has a different name, it will not be assessed.
- Make sure your script is not unnecessarily complex – your script should use consistent indentation and include whitespace/blank lines (where relevant) to make the script more logical and easier for a person to read. You must also include general inline comments in the script code to indicate the purpose of more complex command combinations etc.
- Your script must start with the following first line:
#!/bin/sh
(this specifies the shell to be used to interpret the rest of the commands in the script)
- Your script must include comments at the beginning (near the top of the script) to specify:
- the script’s purpose,
- the author (including student number),
- and the date (when last modified).
- One directory name must be supplied as an argument to be used by your script when it is executed by the user (the directory referred to would presumably be one that contains a mixture of validly named and invalidly named log files that are going to be processed by the script). The directory name must not be ‘hardcoded’ (written) in your script, i.e., its value must come from the command line argument. If no directory name is provided by the user, the script should provide a usage warning message and then exit. See Example Output for an example.
- Near the beginning of your script, you need to check that the directory associated with the directory name provided as an argument to the script:
- Exists. If the directory does not exist, your script must display an error message and then it should exit. The directory name provided can use an absolute path, or a relative path – see Example Output for examples.
- Is readable, writeable and
- If the directory is not readable, your script must display an error message and then it should exit.
- If the directory is not writeable, your script must display an error message and then it should exit.
- If the directory is not executable, your script must display an error message and then it should exit.
- For every file in the specified directory, the script must:
- For a validly named file:
- For a validly named file:
echo the following output to the terminal screen:
- Rename the file with its specific filedecimal name and then move it to a
subdirectory called CATEGORY1 (the script should only create the directory here if it does not already exist and also only when a file in this category is found). See the section Binary to Decimal Renaming below.
- For an invalidly named file that starts with file and is then followed by less than 8 binary digits (and no other
echo the following output to the terminal screen:
- Move the invalid file to a subdirectory called CATEGORY2 (the script should
only create the directory here if it does not already exist and also only when a file in this category is found).
- For an invalidly named file that starts with file and is followed by any characters that include some non-binary
echo the following output to the terminal screen:
- Move the invalid file to a subdirectory called CATEGORY3 (the script should
only create the directory here if it does not already exist and also only when a file in this category is found).
- For an invalidly named file that does not start with file:
echo the following output to the terminal screen:
- Move the invalid file to a subdirectory called CATEGORY4 (the script should only create the directory here if it does not already exist and also only when a file in this category is found).
- If the command-line argument specified directory contains no files:
- echo the following output to the terminal screen:
- exit the script
This also means if you run the script twice, the first time should process any files, displaying the outputs specified in requirement 7 above, renaming where appropriate, and moving files to specified category subdirectories. Running the script for a second time (with the same directory argument) will produce the output specified in this requirement (requirement 8), as all files should have already moved to subdirectories. Your output should never “process” or mention the subdirectory names.
Binary to Decimal Renaming
As previously mentioned, a validly named file will have the form fileABCDEFGH, eg file01010111
Your script must implement an algorithm only using commands and syntax as discussed in practical classes. Your solution is restricted (i.e not allowed) to use the shell “builtin” conversion
e.g. echo $((2#101010101))
This restriction also extends to any other command that has not been discussed in classes – the following list shows some of the common methods (but not all) found through a brief google search – none are permitted:
awk sed printf xxd od perl ibase, obase and bc
Instead, your solution must take an iterative (looping) approach to demonstrate your understanding of iteration and selecting individual characters from a string (a string is just a sequence of characters).
Algorithm hint
The suggested approach is you take the binary part from the filename (what command can extract part of a string?), and then examine each binary digit one by one in turn (the syntax to do this is given in a practical) – the digit’s position in the binary string will determine the value (a multiple of 2) to add to the decimal total. Note the syntax for the character position starts numbering from zero, and from the left hand side.
For example, consider the binary value in the string “01001001”
Column Value | 128 | 64 | 32 | 16 | 8 | 4 | 2 | 1 |
String Position | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
String Value | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 1 |
Assume you start in position 0 of the string, and total is 0.
In position 0, the bit value is 0, so add 0*128 to the total (total still equals 0)
Next, in position 1, the bit value is 1, so add 1*64 to the total (total now equals 64) Next, in position 2, the bit value is 0, add 0*32 to the total (total still equals 64) Next, in position 3, the bit value is 0, so add 0*16 to the total (total still equals 64) Next, in position 4, the bit value is 1, so add 1*8 to the total (total now equals 72) Next, in position 5, the bit value is 0, so add 0*4 to the total (total still equals 72) Next, in position 6, the bit value is 0, so add 0*2 to the total (total still equals 72) Finally, in position7, the bit value is 1, so add 1*1 to the total (total now equals 73)
When implementing your solution, you may consider converting the 8-bit binary portion of a filename to either:
- its positive decimal equivalent (as demonstrated above), or
- for a higher grade, assume the 8-bit value is using twos complement representation (see criterion 4 in the marking rubric)
8-bit positive binary values
Assuming you implement conversion of 8-bit positive numbers only, the following would be valid conversions:
8- bit twos complement binary values
Assuming you implement conversion of 8-bit twos complement number representation, the following would be valid conversions:
Example output
Example files
You of course are free to create any directory you like in your kit501script directory and populate it with files for testing purposes (this is highly recommended – you need to test your script works correctly!). The marking process will not use the filenames or directory name that are listed below, marking will use a different directory and different testing files.
In this example, assume a subdirectory called research has been created inside kit501script. The contents of research are the following files:
The following are sample outputs of the script you must develop. The text up to and including the $ is the shell prompt:
Example 1. No directory name specified as an argument:
[username@ictteach kit501script]$ ./binarylog.sh Usage: ./binarylog.sh dirname
where dirname is a directory containing binary-named files
Example 2. Invalid directory name specified as an argument:
[username@ictteach kit501script]$ ./binarylog.sh reserch Directory reserch does not exist!
Example 3. Valid directory name specified as an argument, but (e.g.) is not readable:
[username@ictteach kit501script]$ ./binarylog.sh research Directory research is not readable!
Example 4. If the specified directory contained no files:
[username@ictteach kit501script]$ ./binarylog.sh research The directory research contains no files…
Example 5. Normal output with mixture of valid/invalid files:
[username@ictteach kit501script]$./binarylog.sh research
invalid – badfile does not start with ‘file’ valid – file00000001 [renamed to file1] invalid – file0101 does not have 8 bits
valid – file01111111 [renamed to file127]
invalid – file01whoops non-binary characters after ‘file’ valid – file10000001 [renamed to file-127]
invalid – file1111111 does not have 8 bits valid – file11111111 [renamed to file-1]
invalid – filefilefile non-binary characters after ‘file’
Example 6. Example contents of specified directory (research in this example) after the script has run:
[username@ictteach kit501script]$ ls research/* research/CATEGORY1:
file1 file-1 file127 file-127
research/CATEGORY2: file0101 file1111111
research/CATEGORY3: file01whoops filefilefile
research/CATEGORY4: badfile
Submitting Your Assignment
All assignment submissions are through MyLO. Submitting your assignment to MyLO requires you to first make a compressed copy of your script file on ictteach, copy that compressed file from ictteach to your local computer, and then upload the compressed file from your local computer to the MyLO submission area. Please read and follow the instructions below carefully
– ask your tutor if you have any difficulty, and we suggest you try the instructions well-ahead of the due date so you do not encounter any last minute problems.
1. Start on ictteach
You must first create a compressed1 version (a copy) of your assignment script on ictteach, and then copy this version to your local computer:
On ictteach, run the following commands ($ is the prompt):
$ cd ~/kit501script
$ gzip -c binarylog.sh > binarylog.sh.gz
You then need to copy the binarylog.sh.gz file to your local computer – instructions on how to do this differ depending on what local operating system your local computer is using
– see below. The instructions for each operating system use a command called scp (secure copy), which uses the SSH protocol behind the scenes to copy files.
1 MyLO will not permit files to be uploaded with certain file extensions – .sh is one of them. Instead, we compress the script file (to preserve all formatting) and the result will have a .gz extension which MyLO will accept.
2. Apple macOS (using terminal)
If you are using an Apple macOS-based computer to submit, copy your compressed script file to the local computer via:
- From a new terminal window (i.e. not a terminal session already connected to
ictteach), type the following (% is the prompt, do not type that)
% scp username@ictteach.its.utas.edu.au:kit501script/binarylog.sh.gz
~/Desktop
(replace username with your own UTAS username (this is NOT your email address!))
- You should then be prompted for your ictteach password:
username@ictteach.its.utas.edu.au’s password:
- After typing your password (it will not be shown onscreen), if you have correctly authenticated, you should get a notification line like the following to indicate the file has been downloaded (to your Desktop in this instance – the file size and data transfer rate shown will differ to your file size and data transfer rate )
binarylog.sh.gz 100% 3727 2.6MB/s 00:00
3. Microsoft Windows (using putty)
If you are using your own Microsoft Windows-based computer to submit, we assume you have previously installed the putty program to access ictteach. Putty also includes some other programs when you install it – pscp.exe is one of them and it is the program needed. If you are not using putty but have been using some other access method, you will need to investigate how to use the scp command for that method.
Copy your compressed script file to the local computer via:
- Start a new command prompt window (select the Windows start menu icon and then type cmd.exe).
- In the command prompt window, use the following command to navigate to the putty local installation directory (in the example, putty is installed in a putty subdirectory of the standard location, c:\Program Files – your local putty directory may differ). (C:\Users\username> is the prompt, do not type that):
Microsoft Windows [Version 10.0.19042.1110]
(c) Microsoft Corporation. All rights reserved.
C:\Users\username> cd c:\program files\putty
- If successful, the current directory should now be the putty install directory. pscp.exe is putty’s version of the scp command, and it should be installed here. Run the following command (C:\Program files\PuTTY> is the prompt, do not type that):
C:\Program files\PuTTY> .\pscp.exe
username@ictteach.its.utas.edu.au:kit501script/binarylog.sh.gz
%userprofile%\Desktop
(Replace username with your UTAS username (this is NOT your email address))
- You will be prompted for your ictteach password (the password will not appear on screen):
username@ictteach.its.utas.edu.au’s password:
If you have typed the commands correctly, your compressed script file, binarylog.sh.gz
should now be on your Windows Desktop.
4. MyLO submission (all students):
If you have successfully copied your compressed binarylog.sh.gz script file from ictteach to your local computer, it should now be on your desktop. You can now submit this file to MyLO:
- In a web browser, go to the unit’s MyLO site – in semester 1 2022 this is https://mylo.utas.edu.au/d2l/home/506348).
- Navigate through the MyLO site’s top menu to the submission link:
Assessments → Assignments → Assignment 2 (scripting)
- Scroll down and select the Add a File button.
- Chose the My Computer option, and then choose the Upload button. Navigate to your local desktop and select the binarylog.sh.gz file that you copied earlier and finally choose Open.
- Back in the Add a File dialog window, choose the Add button at the bottom.
- Then, if everything appears to be ok (verify you can see your file listed) choose the Submit
button.
Revised submission?
If it is before the due date and time, follow all the submission steps again to make a compressed copy of your script, download the copy to your computer and then upload your copy to MyLO. We will only assess the latest submission.
Late submission?
If your assignment is late then you should submit your compressed script file to MyLO as above – the University’s new assessment policy means you will lose 5% of your score for every day the submission is late – assignments will not be accepted after 10 days past the scheduled submission date.
Need Help?
You are encouraged to seek assistance from your lecturer or tutors in practicals or during consultation after you have seriously thought about the assignment. Please note that we can provide general advice – but you are expected to write and debug (correct) your own code.
When writing your script, think about what steps you need to do, don’t try to write the code all at once. Implement a small part first, test that it is working correctly, then implement a bit more, repeating the process of thinking, writing (coding), testing and refining/fixing. Do not leave the development of your script too late – you will run out of time!
Extra Hints:
Worksheets for Practicals 7 – 10 will be released early for those students who want to start work on this assignment early. For example, some referential syntax needed (e.g., looping) is not covered in the practical material until Practical 9.
- Start by determining the provided directory’s name from the script’s argument parameters (see Practical 7)
- Verify the directory exists – exit if it doesn’t (see Practical 8)
- Verify the directory is readable/writeable/executable – exit if it isn’t (see Practical 8)
- Work out how to list the contents of the directory, one file at a time – use a loop (see Practical 9)
- Work out how to ‘categorise’ a filename into one of the four categories – use a case statement (see Practical 8)
- You’ll need to determine if a filename starts with file, and then also if it only has binary digits following, or if it also has other characters. (see Practical 5)
- The binary part (if it exists) will always be in the same positions (the 5th to 12th characters) of the filename! (see Practical 6)
- To work out how many characters a string has, there was some syntax mentioned in Practical 8.
- You will have to use syntax for decision-making – use the if statement (see Practical 8)
- Simple arithmetic (e.g. addition and multiplication) is covered in Practical 8.
- When testing your script it’s recommended you “clean up” any temporary files or directories already created so you can verify your script correctly creates directories when required. You might like to write another simple setup script that you can keep quickly reuse to recreate your testing log directory and populate it with test files prior to each time you try to run your assignment script. A reminder – the touch command can be used to quickly create an empty file if it doesn’t already exist e.g. touch file00000001
Plagiarism
Plagiarism is a very serious matter, and ignorance of this is not an excuse. If you submit work claiming it to be your own, and it is not original work of your own, this is cheating. This means for example that you cannot submit sections of code that have been written by other students or sourced from the Internet and claim you wrote the code yourself. Plagiarism can result in academic penalties both in relation to your assignment, and also on your permanent university record.
As an example, you cannot take code written by someone else and change variable names, comments and spacing to make it appear you wrote the code – this would still be considered plagiarism.
KIT501 Scripting Assignment Marking Rubric
Criteria | HD | DN | CR | PP | NN |
1 Script Execution | The script runs without any | The script runs without any | The script runs without | The script runs with | The script does |
(5%) | syntax errors or unusual output | syntax errors, but some | any syntax errors, but | very minor syntax | not run, or it has |
minor unusual output occurs | some significant unusual | errors that do not | significant | ||
output occurs | prevent it from | syntactic errors, | |||
continuing | or the script | ||||
contents are | |||||
trivial | |||||
2 Log source | The source directory is provided | The source directory is | The source directory is | The source | The source |
directory (10%) | as a command line argument, it | provided as a command line | provided as a command | directory is | directory is not |
is verified to exist, and all of the | argument, it is verified to | line argument, it is verified | provided as a | provided as a | |
directory permissions are | exist, and some of the | to exist, but none of the | command line | command line | |
correctly verified | directory permissions are | directory permissions are | argument, but no | argument | |
correctly verified | correctly verified | verification is | (and/or it is | ||
performed for | hardcoded in the | ||||
existence or | code) | ||||
permissions | |||||
3 Log source | Script correctly outputs when | n/a | n/a | Script correctly | Script does not |
directory | the directory is completely | outputs when the | correctly output | ||
contents (5%) | empty, and correctly outputs | directory is | when a directory | ||
when the directory contains no | completely empty | contains no files | |||
files (but has subdirectories). | |||||
Only syntax and commands | |||||
discussed in classes have been | |||||
used | |||||
4 Binary | The script correctly converts all | The script correctly converts | n/a | The script correctly | No binary to |
Conversion (40%) | valid filenames to their decimal | all valid filenames to their | converts all valid | decimal | |
equivalent, correctly converts | decimal equivalent, correctly | filenames to their | conversion is | ||
values using twos complement | converts values using | decimal equivalent, | correctly | ||
representation and displays the | positive binary | correctly converts | displayed | ||
result in the output. Only syntax | representation and displays | values using | |||
and commands discussed in | the result in the output. Only | positive binary | |||
classes have been used | syntax and commands | representation and | |||
discussed in classes have | displays the result | ||||
been used | in the output. | ||||
5 invalid files | All Invalid files are correctly | All Invalid files are correctly | Some Invalid files are | Some Invalid files | No invalid files in |
categorisation | identified in the output, and they | identified in the output, and | correctly identified in the | are correctly | this category are |
(20%) | are moved to the corresponding | they are moved to the | output, and they are | identified in the | correctly |
category subdirectory. If not | corresponding category | moved to the | output, but they | identified, nor | |
initially present, the | subdirectory. The | corresponding category | are not moved to | are they moved | |
corresponding category | corresponding category | subdirectory, but some | the corresponding | ||
subdirectory is only created | subdirectory is created even | files are misclassified | category | ||
when and if files in this category | if no files in the category are | subdirectory, and | |||
are discovered. Only syntax and | found. Only syntax and | some files are | |||
commands discussed in classes | commands discussed in | misclassified | |||
have been used | classes have been used | ||||
6 Script | Informative script comments are | Informative script comments | Informative script | Informative script | No comments or |
Comments (10%) | included at the start, as well as | are included at the start, and | comments are included at | comments are | only extremely |
throughout the script (where | many other comments are | the start, but there are | included at the | sparse or | |
appropriate) | included, but some | few other comments – | start, but very | inappropriate | |
comments are obviously | there is an obvious need | limited comments | comments are | ||
missing | for other comments | elsewhere | included | ||
7 Script Layout | Excellent, consistent use of | Excellent use of indentation | Good use of indentation | Occasional, correct | No attention |
(10%) | indentation and whitespace | and whitespace however | and whitespace but a few | use of indentation | provided to |
throughout | there are some minor | sections are inconsistent | and whitespace but | layout, major | |
examples of inconsistency | many sections are | lack of | |||
inconsistent | consistent use of | ||||
indentation and | |||||
whitespace |
Note for criteria 1, 6 and 7 grades above NN are only applicable if the script contents are non-trivial
14