Fetching and recording the data

The main function in bin/main.dart will prompt the data to be downloaded each minute using an instance of the DataMonitor class. There are 1440 minutes in a day and the data files average at around 2k, so 3 megabytes of disk space a day is practical:

main() async {
  setupLogging();

  DataMonitor quakeMon = new DataMonitor();
  Duration updateInterval = new Duration(seconds: 60);

  new Timer.periodic(updateInterval, quakeMon.fetchData);
}

The main function is marked async, which gives you a clue that the implementation will make use of the await feature. The call to setupLogging will be covered in the next section of this chapter.

The implementation of DataMonitor is in lib/quakemonitorfs.dart, which contains the key method fetchData:

  fetchData(Timer timerFetch) async {
    try {
      var outDir = new Directory('data');
      if (!await outDir.exists()) {
        outDir.create();
      }

      log.info("Calling web service...");
      HttpClientRequest req = await geoClient.getUrl(Uri.parse(dataUrl));
      HttpClientResponse resp = await req.close();
      String latestFilePath = path.join('data', latestFilename);
      await resp.pipe(new File(latestFilePath).openWrite());
      String fileContents = await new File(latestFilePath).readAsString();

      var dataset = await JSON.decode(fileContents);

      int newId = int.parse(dataset['metadata']['generated'].toString());

      if (idPreviousFetch != newId) {
        idPreviousFetch = newId;
        var f = new File(path.join('data', '${newId}.txt'));
        await f.writeAsString(dataset.toString());
        log.fine("Saved $newId - ${dataset.toString()}");
      }
    } catch (exception, stacktrace) {
      log.severe("Exception fetching JSON.", exception, stacktrace);
    }
  }

Nearly every other line has an await! The program only has to process information every minute or so, therefore it does not require a responsive structure and a more linear approach is sufficient, though the code is still asynchronous.

The remote resource is processed first and the response closed off. The JSON string returned can be put straight into a file. Once this is completed, the fileContents variable is populated with the data from the file, which is then parsed by the JSON to return an object into the dataset.

The Id of the response is checked so that the same response is not saved twice. Note that the generated Id may be different, while the content is the same. The feed is more of a snapshot than a steady stream.

The data is saved off to the data folder using the id of the fetch as a text (.txt) file. To give some feedback, the id and JSON are written to the log file via the call to qlog.fine.