Building the command-line interface

The primary means to interact with our new library will be the command-line interface we will now develop. Unfortunately, the Java SDK has nothing built in to help make sophisticated command-line utilities. If you've been using Java for any time, you've seen the following method signature:

    public static void main(String[] args) 

Clearly, there is a mechanism to process command-line arguments. The public static void main method is passed string arrays that represent arguments provided by the user on the command line, but that's about as far as it goes. To parse the options, the developer is required to iterate over the array, analyzing each entry. It might look something like this:

    int i = 0; 
    while (i < args.length) { 
      if ("--source".equals(args[i])) { 
         System.out.println("--source = " + args[++i]); 
      } else if ("--target".equals(args[i])) { 
         System.out.println("--target = " + args[++i]); 
      } else if ("--force".equals(args[i])) { 
        System.out.println("--force set to true"); 
      } 
      i++; 
    } 

This is an effective solution, if very naive and error-prone. It assumes that whatever follows --source and --target is that argument's value. If the user types --source --target /foo, then our processor breaks. Clearly, something better is needed. Fortunately, we have options.

If you were to search for Java command-line libraries, you'll find an abundance of them (at least 10 at last count). Our space (and time) is limited here, so we obviously can't discuss all of them, so I'll mention the first three that I'm familiar with: Apache Commons CLI, Airline, and Crest. Each of these has some fairly significant differences from its competitors.

Commons CLI takes a more procedural approach; the list of available options, its name, description, whether or not it has arguments, and so forth, are all defined using Java method calls. Once the list of Options has been created, the command-line arguments are then manually parsed. The preceding example could be rewritten as follows:

    public static void main(String[] args) throws ParseException { 
      Options options = new Options(); 
      options.addOption("s", "source", true, "The source"); 
      options.addOption("t", "target", true, "The target"); 
      options.addOption("f", "force", false, "Force"); 
      CommandLineParser parser = new DefaultParser(); 
      CommandLine cmd = parser.parse(options, args); 
      if (cmd.hasOption("source")) { 
        System.out.println("--source = " +  
          cmd.getOptionValue("source")); 
      } 
      if (cmd.hasOption("target")) { 
        System.out.println("--target = " +  
          cmd.getOptionValue("target")); 
      } 
      if (cmd.hasOption("force")) { 
         System.out.println("--force set to true"); 
      } 
    } 

It's certainly more verbose, but it's also clearly, I think, more robust. We can specify long and short names for the option (--source versus -s), we can give it a description, and, best of all, we get built-in validation that an option has its required value. As much of an improvement as this is, I've learned from experience that the procedural approach here gets tedious in practice. Let's take a look at our next candidate to see how it fares.

Airline is a command-line library originally written as part of the airlift organization on GitHub. After languishing for some time, it was forked by Rob Vesse and given a new life (http://rvesse.github.io/airline). Airline's approach to command-line definition is more class-based--to define a command utility, you declare a new class, and mark it up appropriately with a number of annotations. Let's implement our preceding simple command line with Airline:

    @Command(name = "copy", description = "Copy a file") 
    public class CopyCommand { 
      @Option(name = {"-s", "--source"}, description = "The source") 
      private String source; 
      @Option(name = {"-t", "--target"}, description = "The target") 
      private String target; 
      @Option(name = {"-f", "--force"}, description = "Force") 
      private boolean force = false; 
      public static void main(String[] args) { 
        SingleCommand<CopyCommand> parser =  
          SingleCommand.singleCommand(CopyCommand.class); 
        CopyCommand cmd = parser.parse(args); 
        cmd.run(); 
      } 
 
      private void run() { 
        System.out.println("--source = " + source); 
        System.out.println("--target = " + target); 
        if (force) { 
          System.out.println("--force set to true"); 
        } 
      } 
    } 

The options handling continues to grow in terms of code size, but we're also gaining more and more clarity as to what options are supported, and what they each mean. Our command is clearly defined via @Command on the class declaration. The possible options are delineated as @Option--annotated instance variables, and the business logic in run() is completely devoid of command-line parsing code. By the time this method is called, all the data has been extracted and we're ready to do our work. That looks very nice, but let's see what our last contender has to offer.

Crest is a library from Tomitribe, the company behind TomEE, the "all-Apache Java EE Web Profile certified stack" based on the venerable Tomcat Servlet container. Crest's approach to command definition is method based, where you define a method per command. It also uses annotations, and offers Bean Validation out of the box, as well as optional command discovery. Reimplementing our simple command, then, may look like this:

    public class Commands { 
      @Command 
      public void copy(@Option("source") String source, 
        @Option("target") String target, 
        @Option("force") @Default("false") boolean force) { 
          System.out.println("--source = " + source); 
          System.out.println("--target = " + target); 
          if (force) { 
            System.out.println("--force set to true"); 
          } 
       } 
    } 

That seems to be the best of both worlds: it's nice and concise, and will still keep the actual logic of the command free from any CLI-parsing concerns, unless you're bothered by the annotations on the method. Although the actual logic-implementing code is free from such concerns. While Airline and Crest both offer things the other does not, Crest wins for me, so that's what we'll use to implement our command-line interface.

With a library chosen, then, let's take a look at what our CLI might look like. Most importantly, we need to be able to specify the path (or paths) we want to search. Likely, most files in those paths will have the same extension, but that certainly won't always be the case, so we want to allow the user to specify only the file patterns to match (for example, .jpg). Some users might also be curious about how long it takes to run the scan, so let's throw in a switch to turn on that output. And finally, let's add a switch to make the process a bit more verbose.

With our functional requirements set, let's start writing our command. Crest is method-based in its command declarations, but we'll still need a class to put our method in. If this CLI were more complicated (or, for example, if you were writing a CLI for an application server), you could easily put several CLI commands in the same class, or group similar commands in several different classes. How you structure them is completely your concern, as Crest is happy with whatever you choose.

We'll start with our CLI interface declaration as follows:

    public class DupeFinderCommands { 
      @Command 
      public void findDupes( 
        @Option("pattern") List<String> patterns, 
        @Option("path") List<String> paths, 
        @Option("verbose") @Default("false") boolean verbose, 
        @Option("show-timings")  
        @Default("false") boolean showTimings) { 

Before we can discuss the preceding code, we need to declare our Java module:

    module dupefind.cli { 
      requires tomitribe.crest; 
      requires tomitribe.crest.api; 
    } 

We've defined a new module, which is named similarly to our library's module name. We also declared that we require two Crest modules.

Back to our source code, we have the four parameters that we discussed in our functional requirements. Note that patterns and paths are defined as List<String>. When Crest is parsing the command line, if it finds multiple instances of one of these (for example, --path=/path/one--path=/path/two), it will collect all of these values and store them as a List for you. Also, note that verbose and showTimings are defined as boolean, so we see a nice example of the type coercion that Crest will do on our behalf. We also have default values for both of these, so we're sure to have sane, predictable values when our method executes.

The business logic of the method is pretty straightforward. We will handle the verbose flag upfront, printing a summary of the operation requested as follows:

    if (verbose) { 
      System.out.println("Scanning for duplicate files."); 
      System.out.println("Search paths:"); 
      paths.forEach(p -> System.out.println("\t" + p)); 
      System.out.println("Search patterns:"); 
      patterns.forEach(p -> System.out.println("\t" + p)); 
      System.out.println(); 
    } 

Then we will perform the actual work. Thanks to the work we did building the library, all of the logic for the duplicate search is hidden away behind our API:

    final Instant startTime = Instant.now(); 
    FileFinder ff = new FileFinder(); 
    patterns.forEach(p -> ff.addPattern(p)); 
    paths.forEach(p -> ff.addPath(p)); 
     
    ff.find(); 
 
    System.out.println("The following duplicates have been found:"); 
    final AtomicInteger group = new AtomicInteger(1); 
    ff.getDuplicates().forEach((name, list) -> { 
      System.out.printf("Group #%d:%n", group.getAndIncrement()); 
      list.forEach(fileInfo -> System.out.println("\t"  
        + fileInfo.getPath())); 
    }); 
    final Instant endTime = Instant.now(); 

This code won't compile at first, as we've not told the system we need it. We can do that now:

    module dupefind.cli { 
      requires dupefind.lib; 
      requires tomitribe.crest; 
      requires tomitribe.crest.api; 
    } 

We can now import the FileFinder class. First, to demonstrate that the modules are, in fact, doing what they're supposed to do, let's try to import something that wasn't exported: FindFileTask. Let's create a simple class:

    import com.steeplesoft.dupefind.lib.model.FileInfo; 
    import com.steeplesoft.dupefind.lib.util.FindFileTask; 
    public class VisibilityTest { 
      public static void main(String[] args) { 
        FileInfo fi; 
        FindFileTask fft; 
      } 
    } 

If we try to compile this, Maven/javac will complain loudly with an error message like this:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.1:compile (default-compile) on project cli: Compilation failure: Compilation failure:
[ERROR] /C:/Users/jason/src/steeplesoft/DupeFinder/cli/src/main/java/com/
steeplesoft/dupefind/cli/VisibilityTest.java:[9,54] 
com.steeplesoft.dupefind.lib.util.FindFileTask is not visible because 
package com.steeplesoft.dupefind.lib.util is not visible 
[ERROR] /C:/Users/jason/src/steeplesoft/DupeFinder/cli/src/main/java/com/
steeplesoft/dupefind/cli/VisibilityTest.java:[13,9] cannot find symbol 
[ERROR] symbol:   class FindFileTask 
[ERROR] location: class com.steeplesoft.dupefind.cli.VisibilityTest 

We have successfully hidden our utility classes while exposing our public API. It may take some time for this practice to become widespread, but it should work wonders in preventing the crystallization of private APIs as pseudo-public.

Back on task, we create an instance of our FileFinder class, use String.forEach to pass our paths and patterns to the finder, then start the work with a call to find(). The work itself is threaded, but we've exposed a synchronous API, so our call here will block until the work has been completed. Once it returns, we start printing details to the screen. Since FindFiles.getDuplicates() returns Map<String, List<FileInfo>>, we call forEach() on the Map to iterate over each key, then we call forEach() on the List to print information about each file. We also use an AtomicInteger as the index, as the variable must be final or effectively final, so we just use a final instance of AtomicInteger. BigInteger may come to mind to more experienced developers, but it's immutable, so that makes it a poor choice for our use here.

The output of running the command will look something like this:

The following duplicates have been found: 
Group #1: 
     C:\some\path\test\set1\file5.txt 
     C:\some\path\test\set2\file5.txt 
Group #2: 
     C:\some\path\test\set1\file11.txt 
     C:\some\path\test\set1\file11-1.txt 
     C:\some\path\test\set2\file11.txt 

Next, we handle showTimings. I didn't call it out in the preceding code, though I will now, but we get an Instant instance (from the Java 8 date/time library in java.time) before and after processing. Only when showTimings is true do we actually do anything with them. The code that does that looks like this:

    if (showTimings) { 
      Duration duration = Duration.between(startTime, endTime); 
      long hours = duration.toHours(); 
      long minutes = duration.minusHours(hours).toMinutes(); 
      long seconds = duration.minusHours(hours) 
         .minusMinutes(minutes).toMillis() / 1000; 
      System.out.println(String.format( 
        "%nThe scan took %d hours, %d minutes, and %d seconds.%n",  
         hours, minutes, seconds)); 
    } 

With our two Instant, we get a Duration, then start calculating hours, minutes, and seconds. Hopefully, this never runs more than an hour, but it can't hurt to be ready for it. And that's all there is to the CLI, in terms of code. Crest did the heavy lifting for our command-line parameter parsing, leaving us with a straightforward and clean implementation of our logic.

There's one last thing we need to add, and that's the CLI help. It would be very helpful for the end user to be able to find out how to use our command. Fortunately, Crest has support built in to provide that information. To add the help information, we need to create a file called OptionDescriptions.properties in the same package as our command class (remember that since we're using Maven, this file should be under src/main/resource), as follows:

    path = Adds a path to be searched. Can be specified multiple times. 
    pattern = Adds a pattern to match against the file names (e.g.,
    "*.png").
    Can be specified multiple times. 
    show-timings= Show how long the scan took 
    verbose = Show summary of duplicate scan configuration 

Doing so will produce an output like this:

    $ java -jar cli-1.0-SNAPSHOT.jar help findDupes 
    Usage: findDupes [options] 
    Options: 
      --path=<String[]>    Adds a path to be searched. Can be
                            specified multiple times. 
      --pattern=<String[]> Adds a pattern to match against
                            the file names
                           (e.g., "*.png"). Can be specified
                             multiple times. 
      --show-timings       Show how long the scan took 
      --verbose            Show summary of duplicate scan configuration 

You can be as verbose as you need to be without making your source code an unreadable mess.

With that, our CLI is feature-complete. Before we move on, we need to take a look at some build concerns for our CLI and see how Crest fits in. Obviously, we need to tell Maven where to find our Crest dependency, which is shown in the following piece of code:

    <dependency> 
      <groupId>org.tomitribe</groupId> 
      <artifactId>tomitribe-crest</artifactId> 
      <version>${crest.version}</version> 
    </dependency> 

We also need to tell it where to find our duplicate finder library as follows:

    <dependency> 
      <groupId>${project.groupId}</groupId> 
      <artifactId>lib</artifactId> 
      <version>${project.version}</version> 
    </dependency> 

Note groupId and version: since our CLI and library modules are part of the same parent multi-module build, we set groupId and version to that of the parent module, allowing us to manage that from a single location, which makes changing groups or bumping versions much simpler.

The more interesting part is the build section of our POM. First, let's start with maven-compiler-plugin. While we are targeting Java 9, crest-maven-plugin, which we'll look at in a moment, does not currently seem to like the classes generated for Java 9, so we instruct the compiler plugin to emit Java 1.8 bytecode:

    <plugin> 
      <groupId>org.apache.maven.plugins</groupId> 
      <artifactId>maven-compiler-plugin</artifactId> 
      <configuration> 
         <source>1.8</source> 
         <target>1.8</target> 
      </configuration> 
    </plugin> 

Next, we need to set up crest-maven-plugin. To expose our command classes to Crest, we have two options: we can use runtime scanning for the classes, or we can have Crest scan for commands at build time. In order to make this utility as small as possible, as well as reducing the startup time as much as possible, we will opt for the latter approach, so we will need to add another plugin to the build, as follows:

    <plugin> 
      <groupId>org.tomitribe</groupId> 
      <artifactId>crest-maven-plugin</artifactId> 
      <version>${crest.version}</version> 
      <executions> 
         <execution> 
            <goals> 
              <goal>descriptor</goal> 
            </goals> 
         </execution> 
      </executions> 
    </plugin> 

When this plugin runs, it will generate a file called crest-commands.txt that Crest will process to find classes when it starts. It may not save much time here, but it's definitely something to keep in mind for larger projects.

Finally, we don't want the user to have to worry about setting up the classpath (or module path!) each time, so we'll introduce the Maven Shade plugin, which will create a single, fat jar with all of our dependencies, transitive and otherwise:

    <plugin> 
      <artifactId>maven-shade-plugin</artifactId> 
      <version>2.1</version> 
      <executions> 
         <execution> 
             <phase>package</phase> 
             <goals> 
                <goal>shade</goal> 
              </goals> 
              <configuration> 
                 <transformers> 
                   <transformer implementation= 
                     "org.apache.maven.plugins.shade.resource
                      .ManifestResourceTransformer"> 
                     <mainClass> 
                       org.tomitribe.crest.Main 
                     </mainClass> 
                   </transformer> 
                 </transformers> 
              </configuration> 
         </execution> 
      </executions> 
    </plugin> 

After the build, we can then run a search with the following command:

    java -jar target\cli-1.0-SNAPSHOT.jar findDupes \
      --path=../test/set1 --path=../test/set2 -pattern=*.txt 

Clearly, it can still be improved, so we would want to ship that, say, with script wrappers (shell, batch, and so on), but the number of jars is cut down from 18 or so to 1, so that's a big improvement.

With our CLI done, let's make a simple GUI that consumes our library as well.