How it works...

This recipe is a bit iterative, but it should re-iterate (pardon the pun) to demonstrate that Bash has a fair number of functionalities built-in to manipulate strings or any structured data. There is a basic assumption though and that is based on many OSes using C programs:

• A string is an array of characters
• Characters such as ',' are the same as any other character 
• Therefore, we can evaluate or test for the existence of a character to separate fields from lines and even use this to build arrays

Now, reviewing the steps in this recipe:

1. After running the script, we have the following output in the console:

$ builtin-str.sh
first character 1
first three characters 123
third character onwards 4567890asdfghjkl
forth to sixth character 456
last character l
STR2 is less than STR3
STR4 is greater than STR3

We began with the string STR="1234567890asdfghjkl" and as the script ran in the first step:

• In the first step, we printed out a single character starting at the zero (0) position. Remember that this is an array, and position 0 is the starting element.
• Next, we retrieved the first three characters to arrive at: 123.
• However, what if we wanted all of the characters after position 3? We would used ${STR: 3} instead of ${STR: 0-3}.
• Then, given the preceding point, if we wanted the characters at position 4 (the forth element in the array, but this is addressed at position three (3) because counting begins at zero (0)), we use ${STR: 3: 3}.
• And finally, to get only the last character, we can use ${STR:-1}.

To finish the first script in the recipe, we had three more strings. If we wished to compare them to each other, we can do so using conditional logic. Remember that bcd is less than BCD.

2. In the second half of the recipe, we begin with a long-winded script to duplicate in an easily explained manner. We covered some of the tricks you can use with the Bash shell without using AWK and SED:

$ ./builtin-strng.sh 
We have 2 rows in testdata/garbage.csv
BOB,JANE,NAZ,SUE,MAX,TOM
ZERO,ALPHA,BETA,GAMA,DELTA,FOXTROT

We have 3 rows in testdata/employees.csv
#1000,Bob,Green,Dec,1,1967
#2000,Ron,Brash,Jan,20,1987
#3000,James,Fairview,Jul,15,1992

Let's make Bob, Robert!
#1000,Robert,Green,Dec,1,1967
#2000,Ron,Brash,Jan,20,1987
#3000,James,Fairview,Jul,15,1992

Lets remove the column: birthday (1-31)
#1000,Robert,Green,Dec,1967
#2000,Ron,Brash,Jan,1987
#3000,James,Fairview,Jul,1992

Here is the breakdown of the script, but a brief introduction is required for arrays, readarray, IFS, and oldIFS. The point of the exercise is to not go into a great lesson on arrays (this will happen later), but to know that you can use them automatically to create dynamic lists of things such as files or lines within a file. They are referred to using the ${ARR[@]} notation, and each element can be referred to by its index value within the square brackets [...].

The readarray command parses the input into an array using the IFS and oldIFS variables. It separates the data based on a common delimiter (IFS), and oldIFS can maintain the old values, should they be altered:

• In the first step, we use read in the garbage.csv (${GB_CSV}) and then ${#ARR[@]} to retrieve the number of elements in the array. We don't use this value, but it is interesting to note the structure of your file and whether it is being read in correctly. Then, for each member of the array, we remove the empty spaces by counting the number of spaces and then removing them with an additional while loop, performing ${i/, /,} until we're done. The corrected values are then re-inserted into the array.
• In the next step, we use ${i::-1} and a for loop to remove the last character from each line. Then, the result is re-inserted into the array.
• Using a for loop and ARR[$INC]=${i^^}, all characters in the array are made uppercase, and we print out the array using printf (more on this later in another recipe).
• On to employees.csv, we read it into the array again using readarray. Then, we add a hash sign (#) to the beginning of each line and re-insert it into the ARR[$INC]="#${i}" array.
• Then, we search for the substring Bob and replace it with ARR[$INC]=${i/Bob/Robert} . To use the built-in search and replace functionality, we use the following syntax: ${variable/valueToFind/valueToReplaceWith}. Notice that this is also the same premise behind the space removal performed in an earlier step.
• The final step is a bit more complicated and a bit long in the tooth, meaning it could be shortened and performed using another tool such as AWK, but for the purpose of an easy to read example—it was written a bit like a C program. Here, we want to remove the actual birthday value (0-31), or column 5 (the index is 4 if we consider that arrays begin at 0). To begin, we iterate through the array using a for loop, and then we use read to take the input value as an array, too! Then, for each field in the array ${ELEM_ARR[@]}, we then check to see if it is not the first value, and also not the column we wish to remove. We build the correct string via concatenation and then re-insert it into the array before printing each value using echo.