heading.append(addParagraph("Scott Simpson - LinkedIn Learning - September 2022"));
heading.append(addParagraph("Chapter 1 - USING BASH"));
main.append(addHeader("Info on Bash"))
main.append(addParagraph("If you are using a fairly up to date Linux system and that includes the VM if you are using Codespaces for this course, you will almost certainy have an up to date Bash shell but you can check that. For this course, the examples shown will work on Bash 4 and over but may not work on earlier versions. Firstly, you can check that Bash is available to you by using the command"))
main.append(addSyntax("bash --version"))
main.append(addParagraph("Bear in mind that you will probably need to have Bash set as your default shell as well as being available and you can check that with the command"))
main.append(addSyntax("echo $SHELL"))
main.append(addParagraph("Figure 13 shows these commands along with their output when run on the Codespaces terminal."))
main.append(addImageWithCaption("./images/bash.png","Figure 13 - The Bash version and default shell in Codespaces."))
main.append(addParagraph("For comparison, figure 14 shows the same thing on the Raspberry Pi I will be using for this course."))
main.append(addImageWithCaption("./images/bash1.png","Figure 14 - The Bash version and default shell on my Raspberry Pi."))
main.append(addParagraph("If you have a different default, that's probably not a huge problem because you can change the default shell, or you can run it just by typing bash in the terminal. In addition, you may well find that if you type up your script in a different editor, it should still work fine as long as you execute it with bash. So having bash available is critical."))
main.append(addParagraph("Actually, I mentioned in the previous section that you can use the VS Code editor in Codespaces to edit your scripts and if you are writing them on a Windows machine you can use the native VS Code app or pretty much any editor that will save your file as either a bash file or just a plain text file."))
main.append(addParagraph("This leads to a question, and it is one that you will need to answer regardless of how you create your bash scripts. How do you run a script? There are a couple of ways you can do that and which you can choose will depend on whether the file is executable."))
main.append(addParagraph("Within the file, you will have a shebang on the first line, that is"))
main.append(addSyntax("#!/bin/bash"))
main.append(addParagraph("This tells the shell where the file is that will run this program and since it is a bash script, this is bash. This is always the first line of a script and it allows us to run the script without telling the shell that this is a bash script."))
main.append("To illustrate this, I have written a script that outputs the bash version and the name of the default shell. Before tunning it, I have outout a directory to show that it is not executable and then tried running it with the command")
main.append(addSyntax("bash version.sh"))
main.append(addParagraph("That works fine. We can abberviate bash to sh and it still works. Next, I will try running it with"))
main.append(addSyntax("./version.sh"))
main.append(addParagraph("This is just saying run the file with this name from the current directory, but when I execute this command, I get a response of permission denied. However, if I change the permissions so that I can execute it"))
main.append(addSyntax("chmod 722 version.sh"))
main.append(addParagraph("I can run ll again to show that it is executable and then try running the script using first bash and then sh and these work as they did before. However, if I rerun the command"))
main.append(addSyntax("bash version.sh"))
main.append(addParagraph("This now also works. Note that if you just type the script name on it's own, that won't work because the shell is looking in the places where it expects to find executable files, it doesn't look in the current directory. As a result, you will see a command not found response. If you put the file in a directory that is in the path or if you add the directory it is in to the path, you would then be able to run it with the name. That's why we had to use the ./ before ther script name because this tells bash that the script we want to run is in the current directory."))
main.append(addParagraph("As a matter of interest, if the . on its own is part of the env path (the path where the shell will look for any command that you type), you can then omit the ./ and execute the command with just the name of the script. However, this just means that you when you type in the name of your script, as well as looking in the directories the shell would normally look in, it will also look in the current directory, so this will only work, assuming that your script is in a path that is not in that environment path, if the script you are trying to run is in the current directory."))
main.append(addParagraph("You could also run it by providing the full pathname along with the script name"))
main.append(addParagraph("and that would work but also, it would work regardless of what the current directory is. The output from running all of these commands - except the version with the full path name - is shown in figure 15."))
main.append(addImageWithCaption("./images/bash1.png","Figure 15 - The ouput I get when trying to run the version.sh script on my Raspberry Pi."))
main.append(addParagraph("Just a sidebar on this, the shebang is used to tell the shell what program it needs to use to run this script. However, if we omit it, it will still run because Bash is, after all, the default shell. You will probably not run into any problems if you omit it, but it is generally considered good practive because you can't always guarantee that your script will run in an environment where bash is the default script and you might also at some point be writing a script that runs on a program that is not used by default which could cause problems. If you make a habit ofd including the shebang, you will avoid problems like that."))
main.append(addHeader("Pipes and Redirects"))
main.append(addParagraph("Pipes and redirects are used to control where the output and input for commands are sent. When you run a program or a command, it will often sedn its output to the screen and an example of that is ls. If we want to perfrom some other commands on the output of ls, such as search for a specific filename, we can pipe it to another command that will filter out the output and provide the output we actually want. The pipe is a vertical line. On a UK keyboard, the pipe command is next to the left shift key on th same key as the backslash character."))
main.append(addParagraph("Think about what would happen when we run this command."))
main.append(addSyntax("ls | grep -i l"))
main.append(addParagraph("You probably recognise the ls command. This outputs a listing of the current directory and as it happens, this is a line-based output. We then pipe that meaning that this output is not being sent to the screen, it is being sent to another command. We then execute the grep command and this acts as a kind of filter. We have a couple of arguments for that command, actually there are three and these are the -i option, the letter l and a thrird argument which I will come to in a second."))
main.append(addParagraph("The i option makes the grep command ignore case. The l is a regular expression we want grep to search for and because we are ignoring case, this will match either with L or l as we will see in a second."))
main.append(addParagraph("The grep command takes some lines of text as input. In this case, the input is the output we got from the ls command and so that is the third argument. The result of executing this command is shown in figure 16"))
main.append(addImageWithCaption("./images/pipes.png","Figure 16 - The ouput when piping the output of ls to the grep command."))
main.append(addParagraph("The result is that we have a directory listing that has been filtered, so it is only showing and entry in the directory where the file or directory name includes the letter l (regardless of case)."))
main.append(addParagraph("A redirect is very similar but it sends the output to the forst command to a file. We use a greater than symbol to redirect and sometimes we use two, which I will come to. Consider this example."))
main.append(addSyntax("ls > ls.txt"))
main.append("This command doesn't produce any output but look what happens when we rerun the previous command where the output of ls was piped to grep. This is shown in figure 17.")
main.append(addImageWithCaption("./images/pipes1.png","Figure 17 - The ouput when piping the output of ls to the grep command a second time."))
main.append(addParagraph("We can see that a new file has been added, ls.txt, and this is the result of redirecting the output of the ls command to a text file. That text file didn't exist so it was created in order to accept the input."))
main.append(addParagraph("We can comibine pipes and redirects. If you look at the ls.txt file, you will see that it is a complete directory listing. We didn't filter the contents at all. Figure 18 shows the output of ls.txt using the cat command. We have then rerun the ls command piped to grep, but this time we have redirected the output to ls.txt and output the file again."))
main.append(addImageWithCaption("./images/pipes2.png","Figure 18 - The ls file before and after piping ls to grep and redirecting that output to the ls.txt file."))
main.append(addParagraph("The result is that the contents of ls.txt were overwritten when we reditected the output of the grep command to it."))
main.append(addParagraph("Now, I have reset the contents of the ls.txt file to show the whole directory listing and output it to show that's the case. I have then redirected the contents of the grep command to ls.txt and used two greater than symbols."))
main.append(addImageWithCaption("./images/pipes3.png","Figure 18 - The ls file before and after piping ls to grep and redirecting that output to the ls.txt file with two greater than symbols."))
main.append(addParagraph("This time, we can see that our full directory listing is still there, but we also have the output of the grep command tagged on the end. As a results, any file or directory with an l is listed twice. The first entry is Bookshelf, for example, the last in the first listing was Videos, In the second listing you can see that after Videos, we have started again at Bookshelf."))
main.append(addParagraph("The reason for this is that < writes outout to a file. If the file doesn't exist, it will create it but if it does exist, it will overwrite whatever is in the file already. So the result is that when we view that file, we only see the output of the command that was redirected to it. Something siilar happens with << in that it will also create a file if it doesn't already exist but if it does, it will append its output to the end of the file rather than overwriting it."))
main.append(addParagraph("I mentioned that pipes use line-based input and output and you may think that redirects similarly work with files in terms of output but this is not quite true. They actually work with streams and there are three of these as follows:"))
main.append(addSyntax("0"))
main.append(addParagraph("This is the standard input (stdin) and is the input form the keyboard or some other input device."))
main.append(addSyntax("1"))
main.append(addParagraph("This is the standard output (stdout) and that's our regular output, usually going to the screen but it could be some other output device such as a printer."))
main.append(addSyntax("2"))
main.append(addParagraph("This is the standard error (stderr) and normally refers to any output marked as error."))
main.append(addParagraph("We already saw an example of redirection where we redirected the output of the ls command to a file. We didn't specify the standard output but that is the default. Just to be clear here, if we use a command like ls on its own, the output is sent to the standard output and in fact, that's what we mean when we call it the standard output!"))
main.append(addParagraph("When I say that the standard output is the default when using a redirect, I don't mean that by default, the redirected output is sent to stdout. I mean that by default, we are redirecting the output so a command such as"))
main.append(addSyntax("ls > ls.txt"))
main.append(addParagraph("where we haven't specifed a stream is actually telling the OS to take the output that would normally be sent to stdout and send it to a file instead. For comparison, let's consider an example where we do specify the stream. Before we do that, consider the following command."))
main.append(addSyntax("ls /mydir"))
main.append(addParagraph("This is a non-existent directory so as you might expect, we get an error when we try to list its contents. Let's try running the same command and redirect the standard output by explicitly specifying stdout. We will also redirect stderr at the same time. Again, stderr would by default be sent to the stdout stream - to the screen in other words as we saw when running ls on a non-existent directory. This time, it will redirect the stderr to a different file so we have"))
main.append(addParagraph("In figure 19, we have run ls, tried to list a non-existent directory which results in an error being displayed on the screen, run the same ls command again with the stdout and stderr both being redirected to files and then run ls again to show that these files have now been created."))
main.append(addImageWithCaption("./images/redirect.png","Figure 19 - redirecting both the standard output and standard error."))
main.append(addParagraph("You may have noticed that I also used cat to display the contents of these files to the screen. Note that output.txt will display the contents of the directory we listed but since it doesn't exist, there are no contents and this is an empty file. The error.txt file would be empty if there were no errors but in this case there is and we can see that error has been written to the file."))
main.append(addParagraph("Redirecting with the standard input is probably less useful and therefore less common, but let's look at an example."))
main.append(addSyntax("cat < error.txt"))
main.append(addParagraph("This is taking the error.txt file and using it as the input for the cat command and the cat command then outputs that to the screen. You might think that this is the same as"))
main.append(addSyntax("cat error.txt"))
main.append(addParagraph("and you would be right. That is exactly what the cat command does by default so this is just explicitly specifying the default behaviour. A slightly more interesting example is this"))
main.append(addSyntax("cat << error.txt"))
main.append(addParagraph("Based on what we have seen already, you might expect that this is appending the output to something but remember that it is the standard input we are redirecting which implies we are appending something to the standard input. Figure 20 shows this command being run."))
main.append(addImageWithCaption("./images/redirect1.png","Figure 20 - redirecting the standard input."))
main.append(addParagraph("What is actually happening is that we are using the error.txt file as input to the cat command, but we are also telling Linux that there is more input to come. In this case, I have entered some random characters and this conutinues until I press Control & C to stop execution of that command. This is referred to as a here document because (I guess) it is here in the shell. If you type in clear to clear the screen and then run that same command again, you will see all of the random text you typed before is still there but if you look at the actual error.txt file, that has't been changed."))
main.append(addParagraph("There are some cases where a here document can be useful and this might include a scenario where you want to output large blocks of text or to pass arguements or options to an interactive command."))
main.append(addParagraph("In this example, I exited from the cat command creating a here document using Control and C which will halt most running commands in Linux, but this is not the proper way to do it. You might hgave noticed something strange in the output in figure 20. THsi does not include the contents of the error.txt file and the reason is that in this case, error.txt is not specifying a file to be displayed. It certainly does look like it is and if you see a command like this in real use, the chances are that the intention was to display the text. However, the argument passed over here is something called a limit string and it is, in fact, a string you will specifiy as a way of terminate the command."))
main.append(addParagraph("Let's see a more sensible example of that."))
main.append(addSyntax("cat << exit"))
main.append(addParagraph("You can see a sample of output for this in figure 21."))
main.append(addImageWithCaption("./images/here_file.png","Figure 21 - redirecting the standard input to a here document."))
main.append(addParagraph("Notice that in the second last line, we included the word exit but this did not exit the command. In the last, we have only the word exit and this did but notice that it dumped the contents of the here document to the standard output."))
main.append(addParagraph("You might find it interesting to run the same command again and you might notice that it behaves in a slightly strange way. It outputs exactly the same document contents again but it doesn't accept any more input. It isn't showing a command prompt but if you type something like ls, it will run that command and display the directory contents and then show you a command prompt."))
main.append(addParagraph("The course only briefly mentions here doucments (or heredocs) and is a little vague on it's benefits so you might want to read up more on this and you can find a useful tutorual for that on <a href='https://www.baeldung.com/linux/heredoc-herestring'>baeldung.com</a>."))
main.append(addHeader("Bash Built-Ins and Other Commands"))
main.append(addParagraph("When you run a command in bash, whether this is in a script or at the command line, you might be running a command that is built in to Bash, but it might also be a separate program. In some cases, there will be a built in command such as echo that also exists as a separate program. As you might imagine, the bash shell will always use the built in command if it exists and if it doesn't, it will search for an external command with at name. That's the default behaviour but it is possible to override that."))
main.append(addParagraph("We can use a command that is actually called command in order to find out whether a particular command is a builtin or external command. The syntax will look something like this."))
main.append(addParagraph("where the command you want to check is shown in bold. This will be much clearer if we use it to check a different command, say echo."))
main.append(addParagraph("This will respond in one of three ways. For echo, for example, it tells us that this is a builtin command. For an external command it would return the location of the file and if the command doesn't exist, it will tell us that it can't be found. Figure 22 shows examples of each of these."))
main.append(addImageWithCaption("./images/command.png","Figure 22 - Typical output from the command named command!"))
main.append(addParagraph("Bear in mind that where there is a command that is both builtin and external, this will show that it is a builtin command because that is the command that will be executed by default. For example, if we run a command like"))
main.append(addSyntax("echo Hello World"))
main.append(addParagraph(addParagraph("this is going to use the builtin command. We can also explicitly tell the shell to use the builtin command by putting that in front of it so")))
main.append(addParagraph("Since the builtin command is used by default anyway, these two commands are exactly equivalent to each other. On the other hand, we can specify command so that the shell uses the external command."))
main.append(addParagraph("The echo command is pretty much standard so both the builtin and external versions do the same thing as you can see from figure 23."))
main.append(addImageWithCaption("./images/echo.png","Figure 23 - The different versions of the command."))
main.append(addParagraph("If we want the shell to use an external command rather than the builr in version, we can enable the external version using the enable command like this."))
main.append(addSyntax("enable -n echo"))
main.append(addParagraph("It can be hard to verify that the external command is being used but one way to do that is with"))
main.append(addSyntax("command -V echo"))
main.append(addParagraph("This now shows the path to the external file because that is now the version of the command the shell will use. We can also use the enable command with just the -n flag and this will list all of the external commands that have been enabled."))
main.append(addSyntax("enable -n"))
main.append(addParagraph("We can re-enbale the built in command by using the enable command without the -n flag so"))
main.append(addSyntax("enable echo"))
main.append(addParagraph("and we can run command again to show that the shell is not set to use the builtin version again. The output from this is shown in figure 24."))
main.append(addImageWithCaption("./images/enable.png","Figure 24 - Enabling the external version of echo and then re-enabling the builtin version."))
main.append(addParagraph("The documentation for builtins is a little different to that for external commands which have man pages. To give an example of that, since echo is both a builtin and external command, we can use either method. To get the documentation for the external command, we can se the familiar man command"))
main.append(addSyntax("man echo"))
main.append(addParagraph("A builtin that doesn't have an equivalent external version is alias so if we try"))
main.append(addSyntax("man alias"))
main.append(addParagraph("we will see a message telling us that there is no manual entry for alias."))
main.append(addParagraph("To see the documentation for a builtin, we use help rather than man so"))
main.append(addSyntax("help alias"))
main.append(addParagraph("There are not many options for alias, just one in fact and that is -p which lists all currently defined aliases. I didn't know that but that's a potentially useful option!"))
main.append(addParagraph("We can also use help with the -d option to list all builtins."))
main.append(addSyntax("help -d"))
main.append(addHeader("Brackets and Braces in Bash"))
main.append(addParagraph("There are various names used to refer to these characters but they can be very important when you are doing any type of coding, including writing a script in Bash. To avoid any ambiguity, the following characters will be described in this course with the names shown."))
main.append(addParagraph("In most coding languages, the meaning of each of these is reasonably straightgforward and its usually clear where to use them. For example, parentheses are usually used to enclose a condition or a list of funtion parameters, braces will contain a body of code for a conditional statement or a function and brackets are commonly used to hold a data set such as an array."))
main.append(addParagraph("Things can be a little bit different in Bash, in particular because these characters van often act as commands themselves. The important thing to remember is that if you have some experience with another programming language, Bash is going to look a little bit strange at times."))
main.append(addHeader("Bash Expansions and Substitutions"))
main.append(addParagraph("Expansions and substitutions are used to represent unknown values and they are very important in Bash scripting. A good example of this is something that you may have already seen without realising it was an expansion is the ~ character. This is sometimes referred to as the tilde expansion and it essentially expands to the users home directory. For example, let's say I want to list the contents of my home directory without necessarily going in to that directory first, I can that with a command like"))
main.append(addSyntax("ls ~"))
main.append(addParagraph("In this case, I am logged in with my own username - philip - so my home directory is"))
main.append(addSyntax("/home/philip"))
main.append(addParagraph("so this has the same effect as if I had typed out that whole path"))
main.append(addSyntax("ls /home/philip/"))
main.append(addParagraph("This doesn't have to be used on its own. For example, if I want to list the contents of the Downloads directory in my home folder, I could use the full path"))
main.append(addParagraph("or I can use the tilde expansion"))
main.append(addSyntax("ls ~/Downloads"))
main.append(addParagraph("When a command like that is executed, the shell recognises the expansion and replaces the tilde with the path to the home directory."))
main.append(addParagraph("This can be quite convenient and will possibly save a lot of typing, but in terms of Bash scripting, it is useful for another reason. Let's say that you write a script where you want to create a directory in the users home directory. We can do that with a line like"))
main.append(addSyntax("mkdir ~/newdir"))
main.append(addParagraph("Whenever a user runs that script, this will create the folder, newdir, in that users home directory and it will do that for any user. So essentially, we can access the users home directory without knowing in advance that the username is or what the full path to their home directory is."))
main.append(addParagraph("As a matter of interest, the tilde expansion is a representation of the environment variable, $HOME which holds the path to the users home directory so you could also acheive the same thing with"))
main.append(addSyntax("mkdir $HOME/newdir"))
main.append(addParagraph("so ~ and $HOME are interchangeable in this context but the tilde is easier to type!"))
main.append(addHeader("Brace Expansion"))
main.append(addParagraph("A brace expansion is in some ways similar to the tilde expansion, but rather than replace something with a possibly unknown value, it expands a selection to include one instance of each. For example, consider this command."))
main.append(addSyntax("echo c{a,o,u}t"))
main.append(addParagraph("This is echoing to the standard output (the screen) a three letter word that starts with a c and ends with a t. The middle letter is taken from the brace expansion and there are three letters there so we will see three words in the output, each having one of these three letters in the middle and you can see that in figure 25."))
main.append(addImageWithCaption("./images/brace_expansion.png","Figure 25 - A simple example of the use of a brace expansion."))
main.append(addParagraph("With that type of brace expansion, we can provide specific values as we did in the example shown in figure 25 or we can provide a range such as 0-9 or a-z. For example, let's say that we want to create 26 files for each letter of the alphabet where each file is called something like"))
main.append(addSyntax("Letter-A"))
main.append(addParagraph("We could create this file with a command like this"))
main.append(addSyntax("touch Letter-A"))
main.append(addParagraph("It would be tedious to have to run this command 26 times, changing the number and the letter each time and this is where the brace expansion comes in. We can use a brace expansions to change the letter each time. So, the commands to create all 26 files is"))
main.append(addSyntax("touch Letter-{A..Z}"))
main.append(addParagraph("The command and the list of files created is shown in figure 26"))
main.append(addImageWithCaption("./images/brace_expansion1.png","Figure 26 - Using a brace expansion with a range of values."))
main.append(addParagraph("The order of the letters can be reversed. For example, let's use the echo command to display every letter in the alphabet."))
main.append(addSyntax("echo {A..Z}"))
main.append(addParagraph("We can then use the same command with the letters in the expansion switched round to display the alphabet backwards."))
main.append(addSyntax("echo {Z..A}"))
main.append(addParagraph("We can also use an interval so let's say we want to display every 2nd letter of the alphabet, we can do that by adding the interval as a third value in the expansion."))
main.append(addSyntax("echo {A..Z..2}"))
main.append(addParagraph("We can also do the same thing with numerical values so we might, for example, display every number from 1 to 10 with"))
main.append(addSyntax("echo {1..10}"))
main.append(addParagraph("We can also reverse the order."))
main.append(addSyntax("echo {10..1}"))
main.append(addParagraph("We can also use intervals. To give a more complex example, let's say we want to display all multiples of 10 from 1 to 100 (including the number 100). We can do that by starting from 10, extending the expansion up to 100 and giving it an interval of 10 like this."))
main.append(addParagraph("With numberical values, we may have a range where the number of digits varies, for example if we display the numbers 1 to 100."))
main.append(addSyntax("echo {1..100}"))
main.append(addParagraph("Here, we have numbers with 1, 2 and 3 digits. This can be untidy and in some cases, you may need to have all of the numbers using the same number of digits. With our expansion, we can display all of these numbers as 3 digit numbers by adding a couple of leading zeroes to the 1. Note that this will not add 2 leading zeroes to every number which wouldn't fix the problem, it will only add leading zeroes if the number has less than 3 digits to give a three digit number."))
main.append(addSyntax("echo {001..100}"))
main.append(addParagraph("The output for these is shown in figure 27."))
main.append(addImageWithCaption("./images/brace_expansion2.png","Figure 27 - Using a brace expansions in reverse and with an interval."))
main.append(addParagraph("We can also combine expansions so let's say we run the following command."))
main.append(addParagraph("The numbers have been expanded so that we have files number 1 to 12 and for each of these, the letters have been expanded with the result that for each number, we have 4 files, each with a letter from a to d giving us 48 files in total."))
main.append(addParagraph("We can also use an expansion with set values so for example"))
main.append(addSyntax("echo {cat,dog,fox}"))
main.append(addParagraph("simply outputs each value in the set. We can also combine this with other expansions such as"))
main.append(addParagraph("This type of technique is often used when working with directories or where you have a predefined naming scheme."))
main.append(addParagraph("We can also use a code expansion with a command if we want to run it multiple times. For example, in the exercise files, we have 3 directories, dir1, dir2, dir3 and each of these has a text file inside called lorem.txt. Let's say that we want to use the head command to output the first line of each of these files. To do that in one file, we would use a command such as"))
main.append(addParagraph("If we want to use the command on all three files at the same time so that we see the line from all three files, one after the other, we can use the brace expansion like this."))
main.append(addParagraph("The output for both of these commands is shown in "))
main.append(addImageWithCaption("./images/brace_expansion5.png","Figure 30 - Using brace expansions as a command shortcut."))
main.append(addParagraph("Note that I don't have access to the Exercise files on my Raspberry Pi so rather than copy them over, I have just used Codespaces for this example. Notice that when we used the expansion with a command, we can see that the command is exexuted once for each of the three directories with a different value for the directory in each case."))
main.append(addParagraph("So, in this example, we are using the brace expansion as a kind of shorthand for the command we are using and this illustrates the point that you can use brace expansions in a variety of situations, they are not only used in scripts."))
main.append(addHeader("Parameter Expansion"))
main.append(addParagraph("Parameter expansion is a method whereby we can recall stored values and perform some sort of operations on them. The $ symbol is used to denote a parameter expansion and the parameter itself is usually enclosed in curl braces although these can sometimes be omitted."))
main.append(addParagraph("Let's look at an example starting with setting a parameter to point to a value."))
main.append(addSyntax("greeting=hello there!"))
main.append(addParagraph("The simplest way to access this is probably with echo, so we will use that to display the parameter's value."))
main.append(addSyntax("echo $greeting"))
main.append(addParagraph("We can perform an operation that modifies the value before ouputting it"))
main.append(addSyntax("echo ${greeting:6}"))
main.append(addParagraph("This will display everything starting from the character in position 6 (counting from 0). The command"))
main.append(addSyntax("echo ${greeting:6:3}"))
main.append(addParagraph("will do something similar so we have the same starting point, but this only displays 3 characters."))
main.append(addParagraph("There are a variety of uses for parameter expansion which we won't investigate in much detail here, you can get more information from the online <a href='https://www.gnu.org/software/bash/manual/html_node/index.html#SEC_Contents'>Bash Manual</a> under <a href='https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html'>Shell Parameter Expansion</a>. One feature that I will mention here is pattern substitution. Consider the following statement."))
main.append(addParagraph("This will look for the pattern after the forst / character and if it is found, it will be replaced by the text after the second / character. This only replaces the first instance found but if we use a double forward slash, it will replace all instances. Consider this example"))
main.append(addSyntax("echo $greeting/e/_"))
main.append(addParagraph("Again, this is just replacing the first instance and I wanted to start by doing that so that the difference is a little bit clearer. We will do the same thing again, this time we will use a dobule slash after the parameter"))
main.append(addSyntax("echo $greeting//e/_"))
main.append(addParagraph("and we can see that every instance of the letter e is now being replaced by an underscore. The output for all of these commands can be seen in figure 31."))
main.append(addImageWithCaption("./images/parameter_expansion.png","Figure 31 - Using a parameter expansion."))
main.append(addParagraph("We use the curly braces partly to make the code clearer but in some cases, it is necessary to tell the shell that some characters should be associated with the parameter expansion. Consider the following example."))
main.append(addSyntax("echo ${greeting:4:3}"))
main.append(addParagraph("Again, this is outputting the three characters starting from position four. What this is not really important here, we just want to compare this to the output we would get if we didn't include the braces."))
main.append(addSyntax("echo $greeting:4:3"))
main.append(addParagraph("The output for both commands is shown in figure 32."))
main.append(addImageWithCaption("./images/parameter_expansion1.png","Figure 32 - Using a parameter expansion without the braces can give unexpected results."))
main.append(addParagraph("The shell has recognised that $greeting is a parameter expansion and has therefore replaced it with the proper value, but the characters which are intended to denote a position in the string and number of characters have been interpreted as just characters by the shell and it has therefore just output them along with the parameter value. Enclosing everything in the braces tells the shell that this is intended to part of the parameter expansion so it then produces the result we wanted. Nevertheless, you will see that parameter expansion without the braces is quite common. Where there is no ambiguity, for instance with a simple statement like"))
main.append(addSyntax("echo $greeting"))
main.append(addParagraph("whether or not you include the braces is really a question of style."))
main.append(addHeader("Command Substitution"))
main.append(addParagraph("Command subsitution allows you to run a command that produces some output and use that in another command as though it were the actual data. Before we look at an example of that, the syntax is a $ symbol followed the command that is going to be substituted in parentheses. Note that there is an alternative syntax using back ticks rather than parentheses but this can look a little confusing so the parentheses are probably better, although you may see back ticks in other people's code!"))
main.append(addParagraph("To give an example of this, we will use the following command"))
main.append(addSyntax("uname -r"))
main.append(addParagraph("Used on its own, this command will return the kernel version number. Let's say that we want to display a message that is a little bit more informative and tells the user what this number actually is, in case the user doesn't recognise it. One way to do that is with an echo command that includes a command substitution, like this."))
main.append(addSyntax("echo \"The version of the kernel is $(uname -r)"))
main.append(addParagraph("The output for both of these can be seen in figure 33."))
main.append(addImageWithCaption("./images/command_substitution.png","Figure 33 - Using a command substitution."))
main.append(addParagraph("If you are familiar with coding in a language such as Java or C, you might find that the terminology here is a little bit strange, but the concept itself should be quite familiar. If a command returns a value - in this example, the uname command returns a string, you can call that command from within another command in place of the actual string."))
main.append(addParagraph("The echo command takes a string as an argument so we can insert the command substitution so that its output will become part of the string that echo will output."))
main.append(addParagraph("Let's look at another example where we output a string telling the user what version of the Python, so we will use the command"))
main.append(addSyntax("python -V"))
main.append(addParagraph("which will output the version number. We could then take that string and copy it into a larger string and output with a command like"))
main.append(addSyntax("echo \"The Python version is Python 3.9.2\""))
main.append(addParagraph("and this is really the whole point of command substitution. If we wanted to put a command like that in to a script, we could do that but we would have to get the Python version first and edit our script to hold the correct value. Just as importantly, we can't really guarantee that it is correct because it could be copied incorrectly or Python might later be updated so the version shown in the script might no longer be correct."))
main.append(addParagraph("When we run the command this will always give us the current version so it makes sense to use that command to get the version and command substitution is just away of combining these two approaches so we can output a nicely formatted and information message that includes an up to date version number. We do that by removing the version number as a literal string, inserting the command substitution and when it runs, it returns a string which is then included un the output so that gives us"))
main.append(addSyntax("echo \"The Python version is $(python -V)\""))
main.append(addParagraph("Figure 34 shows the output we get by running the command on its own and then as a command subsitution within an echo command."))
main.append(addImageWithCaption("./images/command_substitution1.png","Figure 34 - Using a command substitution, second example."))
main.append(addParagraph("Command subsitutions are fairly straightforward but you can use them to build quite complex commands by nesting one command substiution within another or by combining them with other constructs such as a pipe. For example, let's say we want to use the python command to output a string - essentially, we are using a function from the Python language to output a string. That string is then passed, via a pipe to the tr command (usually you would say that the output of the print command - that's the Python function we are using - is piped to tr) and we will use tr to convert the string to all upper-case."))
main.append(addParagraph("The full command would be"))
main.append(addParagraph("This command along with its output is shown in figure 35."))
main.append(addImageWithCaption("./images/command_substitution2.png","Figure 34 - Using a nested command substitution."))
main.append(addParagraph("The main point to take from this is that command substitutions can be quite powerful and are often used with other tools such as grep, awk and cut and can often br used by a script to determine whether the system it is running on has everything the script needs to run. For example, you can use it to get the Python version and test whether a valid value is returned as opposed to an error so you can check whether the system has Python installed."))
main.append(addHeader("Arithmetic Expansion"))
main.append(addParagraph("Where we said that we can use a command substitution anywhere where we would otherwise whatever that command outputs, with an arithmetic expansion, we can use this wherever we would otherwise use the result of the arithmetic expression. For example, let's say we want to output the result of adding 2 and 3. We could calculate the answer and then use that in an echo command like this"))
main.append(addSyntax("echo \"2 + 3 = 5\""))
main.append(addParagraph("If we use an arithmetic expansion instead, the shell will perform the calculation for us and use the result in the output string so"))
main.append(addParagraph("The output from both of these commands is shown in figure 36."))
main.append(addImageWithCaption("./images/arithmetic_expansion.png","Figure 36 - Using an arithmetic expansion."))
main.append(addParagraph("As you can see, the syntax is pretty similar to the syntax for a command substitution expansion. Note that the expression is enclosed in double parentheses. In earlier versions of artithemtic substitutions, single parentheses were used so the syntax was the same for both but this form of the syntax has been deprecated for arithmetic substitutions. If you use single parentheses, you will see an error - in this case, the error would tell you that 2 is not a command and that is because since this is the syntax for a command substitution, the shell assumes that the first thing inside the parentheses will be a command."))
main.append(addParagraph("It might seem trivial to have your scrpit perform a calculation like this for you but this is a trivial example and a real example would probably be harder to calculate in your head. Using the arithmetic expression might be less likely to incorporate an error in to your script. Perhaps more importantly, you might not know when you are writing the script, what values will be used in the calculation so you would in any case probably write your script with variables. The point is that the shell can perform these calulations for you and in can evaluate any kind of arithmetic expression. One thing to note on that is that in most cases, you can do the calculation yourself in order to evaluate the accuracy of the result but consider this example."))
main.append(addParagraph("You might expect the result to show as something like 0.667, but we actually get zero as the result and this raises a very important point and that is that Bash can only do calculations with integers. In other programming languages, you have the concept of integer division which is what we have here, but you will also have an operator that will perform division on real numbers. That's not the case in Bash."))
main.append(addParagraph("All in all, expansions and substitutions are very important in Bash scripting so it is important when writing a script to be aware of how you can use these."))
