Connecting to an API client

The client will be a command-line application that is used to add new features to the database using the new REST API method.

This project is a command-line application and uses the dart:io package, which has similar functionality to dart:html to work with an HTTP request, as shown in the following code:

main() async {
  print("API client");

  HttpClient client = new HttpClient();
  HttpClientRequest request = await client.postUrl(Uri.parse(apiUrl));
  request.headers.contentType = ContentType.JSON;
  await request.write(JSON.encode(getFeature()));

  HttpClientResponse response = await request.close();
  print("${response.toString()}");

  response.transform(UTF8.decoder).listen((contents) {
    request.close();
    print(contents);
    print("API client - done");
    exit(0);
  });
}

The application will post a single-feature JSON string to the API, wait for the response, display the response, and then exit. If multiple features are required, the program can simply be rerun.

Varying the data

The client will provide a generated feature to post to the API when it is run. Rather than having set data, a feature will be randomly generated. To generate random numbers in Dart, we can make use of the Random class from dart:math as follows:

Map getFeature() {
  var rng = new Random();
  var now = new DateTime.now();
  Map feature = new Map();
  feature['time'] = now.millisecondsSinceEpoch;
  feature['magnitude'] = rng.nextInt(8) + 1;
  feature['latitude'] = rng.nextDouble() * 90;
  feature['longitude'] = rng.nextDouble() * 90;
  return feature;
}

The Random class has methods to provide integer and double values, the former allowing the setting of an upper limit. The nextDouble method returns a value between 0 and 1, so this value can simply be multiplied by the maximum desired value. The time will be set to the current time.

Returning to the map

To view the generated data, the quake map view from the previous chapter can be used. Let's have a look at the visual representation of the generated data in the following screenshot:

Once the command–line API client has been run a number of times, the map will soon be populated. Try adjusting the inputs for the generated data to cover more of the map, as the current numbers are skewed toward the top-right corner.

Reporting on the data

There are numerous existing systems that can be used to create reports, Microsoft SQL Server Reporting Services (SSRS), that has a large install base. Due to Dart's support for the most ubiquitous formats, such as XML, and its ability to work with industry standard databases, solutions built with Dart can certainly be integrated.

The reporting feature that we will consider here is a pure Dart implementation. For many applications, reporting even a lightweight reporting feature and an export can greatly increase the usefulness. The added advantage here is the ease of deployment, even in enterprise situations.

The ReportSite project

Users will access the reports via a small website that will allow them to select each report type. This website is implemented in the ReportSite project, and the main.dart file sets up the initial page with event handlers for the button, as shown in the following figure:

The reports are filled with data from the REST API methods, so ensure that this is running if you wish to view reports. They are rendered in HTML and displayed on the same web page. The following excerpt from the report_latest.dart file shows how this is implemented:

  querySelector('body').setInnerHtml(rep.output.toString(),
  treeSanitizer: new ReportSanitizer());

This is a straightforward method call with the HTML string for the page, so why is a second parameter required? This is a security measure to ensure that the user-supplied content does not claim any undesired HTML string or script when being rendered in the display. Consider the following code snippet:

class ReportSanitizer implements NodeTreeSanitizer {
  void sanitizeTree(Node node) {}
}

The ReportSanitizer class is a No Operation (NOP) implementation, as the HTML is known to be from a trusted source.

Report classes

A report is made of a series of objects in a list, and the page elements are to be based on a similar interface so that the new page elements can be added to the package without having to change the display or export functions.

The page elements are implemented in the report_elements.dart file, which is part of the reports package project.

The core class will be called Element, and as it is to be used as an interface for other classes, it will be declared as follows:

abstract class Element {
  void setContent(var content);
  String toHtml();
}

This class is declared as abstract and provides no implementation for the methods.

To demonstrate this, we will consider the Title page element class, which implements the element (the methods have the @override annotation, which provides feedback to code analysis tools and other developers that the method is an implementation of a base class) as follows:

class Title implements Element {

  String content;

  Title(this.content) {}

  @override
  void setContent(var content) {
    String input = content.toString();
    content = input;
  }

  @override
  String toHtml() {
    return "<h1>$content</h1>";
  }

}

If a method is not provided an implementation, then the Dart analyzer will warn you of 'Missing concrete implementation'.

The other core page elements follow the same pattern. To create a report, only a few lines of code are required, which are as follows:

var rep = new Report('Sample');
rep..addSection(new Title("Sample Report"))
   ..addSection(new Paragraph("This is a paragraph"))
   ..addSection(new Pagebreak())
      ..addSection(new Paragraph("This is a paragraph too"));

The Report object can be found in the reports_base.dart file in the Reports project, with its most important generate method being generated. This method iterates all the page element objects to build up the HTML report that is stored in the output field, as follows:

  bool generate() {
    output = new StringBuffer();
    allContent.forEach((content){
      output.write( content.toHtml() );
    });
    generatedTimestamp = new DateTime.now();
    return true;
  }

A StringBuffer object is used to build up the HTML report. This class is more efficient than appending a number of strings.

Strings are stored internally as an immutable format in Dart (as in most programming languages), and updating a string really entails building an entirely new string object. The StringBuffer object only creates a String object when the toString method is called.

Creating a printable report

The data is retrieved from the REST API and displayed in a tabular fashion by using the HTML Table element. In the performLatest method, JSON is retrieved from the web service using the following URL:

http://127.0.0.1:8080/api/quake/v1/recent/100

This returns a JSON list of the requested length (or less, if the database does not have sufficient records), as shown in the following code snippet:

[
 "{\"properties\":{\"mag\":1.04,\"place\":\"3km SSW of San Bernardino, California\",\"time\":1440707640360,\"updated\":1440707771997,\"tz\":-420,\"url\":\"http://earthquake.usgs.gov/earthquakes/eventpage/ci37234439\",\"detail\":\"http://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/ci37234439.geojson\",\"felt\":null,\"cdi\":null,\"mmi\":null,\"alert\":null,\"status\":\"automatic\",\"tsunami\":0,\"sig\":17,\"net\":\"ci\",\"code\":\"37234439\",\"ids\":\",ci37234439,\",\"sources\":\",ci,\",\"types\":\",general-link,geoserve,nearby-cities,origin,phase-data,scitech-link,\",\"nst\":27,\"dmin\":0.09553,\"rms\":0.11,\"gap\":62,\"magType\":\"ml\",\"type\":\"earthquake\",\"title\":\"M 1.0 - 3km SSW of San Bernardino,California\"},\"geometry\":{\"type\":\"Point\",\"coordinates\":[-  117.30233,34.1023331,10.84]}}",
...

Once the data is retrieved, the report's element objects are created and the HTML for the report details are inserted as a paragraph by using the following code:

...
  String json = await HttpRequest.getString(jsonSrc);
  List<String> items = JSON.decode(json);
  DateTime dt = new DateTime.now();

  Report rep = new Report('Latest');
  rep
    ..addSection(new Title("Latest Quake Data Report"))
    ..addSection(new Paragraph(
        '<div style="align:center"><img src="img/logo.png"/></div>'))
    ..addSection(new Paragraph("Report Generated At : ${dt}"))
    ..addSection(new Pagebreak());

  var table = buildFeatureTable(items);
  rep
    ..addSection(new Paragraph('<table id="reporttable">' + table + '</table>'))
    ..addSection(new Pagebreak())
    ..addSection(new Notes("Looks like a busy day!"))
    ..generate();
...

The buildFeatureTable loops over the incoming JSON data to create the rows for the table, as shown in the following screenshot:

The report is displayed in a continuous manner on screen. Switching to the print preview will show the page breaks between the sections that are created using Cascading Style Sheets (CSS). The div element is added to the report with the page-break-after:always style by the Pagebreak page element.

Charting the data

A good visualization can make a great deal of difference to the person analyzing the data. To achieve this, we can use the modern_charts package, which provides a range of chart options. The chart report is implemented in the report_chart.dart file in the ReportSite project.

This chart will be based on the 20 most recent entries to the earthquake database using the following URL:

http://127.0.0.1:8080/api/quake/v1/recent/20

The first step to create a chart is to build up a dataset from the incoming JSON string as follows:

List dataset = [['Place', 'Magnitude']];

items.forEach((String featureJSON) {
  Map feature = JSON.decode(featureJSON);
  String place = feature['properties']['place'];
  place = place.substring(place.lastIndexOf(",") + 1);

  if (place.length > 5) place = place.substring(0, 5) + ".";
  var mag = feature['properties']['mag'];
  dataset.add([place, mag]);
});
DataTable table = new DataTable(dataset);

The first item in the dataset contains the number and names of the series to be plotted. The place name is processed so that the long and specific place names can fit on the axis of the chart. It is then added to the list with the magnitude, as shown in the following code:

  Map options = {
    'colors': ['#3333cb'],
    'series': {'labels': {'enabled': true}}
  };

The options object contains a range of details for the plotting of the chart, as follows:

  DivElement container = new DivElement();
  container
    ..id = "chartContainer"
    ..style.width = "90%"
    ..style.height = "90%";

  var Title = new HeadingElement.h1();
  Title.text = "Quake Chart";
  document.body.nodes
    ..clear()
    ..add(Title)
    ..add(container);

  LineChart chart = new LineChart(container);
  chart.draw(table, options);

Once the dataset is ready, the HTML elements for the page can be put together, and the LineChart object that is created is given a reference to the object to add it on the page; in this case, a div element container (refer to the modern_chart package documentation for complete details).

The output of the previous code is shown in the following chart:

You may find working in Dartium to be an odd experience. It is a good browser, but you may prefer your daily web browser, such as Internet Explorer, Firefox, or Chrome, with your carefully-managed settings and collection of extensions. Using Dart-to-JavaScript (dart2js) before testing on other browsers interrupts the edit and refresh cycle. Ideally, Dart would run in any web browser.

Exporting to CSV

CSV is a format that just keeps going thanks to its easy import into spreadsheets. The implementation of this report is found in the report_csv.dart file, and it retrieves the 50 most recent entries using the following URL:

http://127.0.0.1:8080/api/quake/v1/recent/50

Let's consider the following code:

performExport(MouseEvent event) async {
  String json = await HttpRequest.getString(jsonSrc);
  List<String> items = JSON.decode(json);
  String csvexport = "Type, Time, Place, Magnitude\r\n";

  items.forEach((String featureJSON) {
    Map feature = JSON.decode(featureJSON);
    String type = feature['properties']['type'];
    String place = feature['properties']['place'];
    var mag = feature['properties']['mag'];
    var time =
        new DateTime.fromMillisecondsSinceEpoch(feature['properties']['time']);
    String row = "$type, $time, ${place.replaceAll(",","")}, $mag\r\n";
    csvexport += row;
  });

  querySelector('body').setInnerHtml('<pre>$csvexport</pre>',
      treeSanitizer: new ReportSanitizer());
}

The plain text CSV is set on the page with a <pre> HTML tag to ensure that the web browser does not change the intended formatting.

Let's have a look at the following screenshot:

Exporting this data to a spreadsheet and applying some formatting produces a very usable result for further (human!) analysis.