官术网_书友最值得收藏!

  • Bash Cookbook
  • Ron Brash Ganesh Naik
  • 348字
  • 2021-07-23 19:17:39

How it works...

Overall, no truly abstract concepts were introduced in this script, except for counting numbers of occurrences and the benefit of sort. Sorting can be a time-consuming process to reduce unneeded or extra data or when the order matters, but it can also be rewarding when performing bulk operations, and pre-processing yields faster returns overall.

Onward and upward to the recipe:

  1. Running these two wc commands will produce both a character and line count of the file testdata/duplicates.txt. It also begins to show another problem. The data can be padded with the filename prefixed with a space:
$ wc -l testdata/duplicates.txt
18 testdata/duplicates.txt
$ wc -c testdata/duplicates.txt
438 testdata/duplicates.txt
  1. In step 2, we use awk and cut to remove the second field. The cut command is a useful command for trimming strings, which may be delimited or merely using hard-coded values such as remove X characters. Using cut, -d stands for delimiter, the space in this example (' '), and -f1 stands for field 1:
$ wc -c testdata/duplicates.txt | cut -d ' ' -f1
438
$ wc -c testdata/duplicates.txt | awk '{ print $1 }'
438
  1. In the final step, we run the sort command three times. We run it once to merely sort the elements in testdata/duplicates.txt, but then we use the -u to sort and keep only unique elements, and the final command counts the number of unique elements. Of course, the returned value is 9 because we had 18 lines in the original duplicates file:
$sort testdata/duplicates.txt

127.0.0.1 localhost
127.0.0.1 localhost
127.0.1.1 moon
127.0.1.1 moon
::1 ip6-localhost ip6-loopback
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::2 ip6-allrouters
# The following lines are desirable for IPv6 capable hosts
# The following lines are desirable for IPv6 capable hosts

$ sort -u testdata/duplicates.txt

127.0.0.1 localhost
127.0.1.1 moon
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
# The following lines are desirable for IPv6 capable hosts
$ sort -u testdata/duplicates.txt | wc -l
9
主站蜘蛛池模板: 临城县| 新沂市| 平顺县| 潜江市| 礼泉县| 密山市| 宝丰县| 襄垣县| 泉州市| 濮阳市| 菏泽市| 河间市| 平遥县| 庄浪县| 宁海县| 和田市| 襄樊市| 赤城县| 祥云县| 鄂尔多斯市| 宁德市| 金秀| 洪雅县| 延庆县| 长岛县| 于田县| 望都县| 南京市| 蒲江县| 墨玉县| 遵化市| 婺源县| 徐闻县| 江口县| 平顺县| 阜新市| 象州县| 澄江县| 图木舒克市| 琼海市| 庆云县|