Flat-file validation

The validation of a flat-file message such as a TSV file generated in accordance with the Digital Sales Reporting Message Suite (DSR), the Claim Detail Message Suite (CDM) and the Recording Data and Rights Revenue Reporting (RDR-R) can be undertaken using a tool developed by Google. The tool is available to download from GitHub at https://github.com/ddexnet/dsrf .

The tool can be deployed on any computer running Mac OS X, LinuX or MS Windows with a Python v2.7 (or later) interpreter installed. The tool comes with a manual and can be used with a small Perl script dsrf-conf.pl.

 The script is provided below.
#!/usr/bin/perl
use strict;
use warnings;

print "\ndsrf-conf.pl -- tool to check conformance of a DSRF file. \n"
print "(c) 2016 Digital Data Exchange, LLC, utilising Google's dsrf library\n\n";

#
# set variables (filelist being the alphabetical list of input files)
#
my $validator = "/Users/nrump/bin/dsrf";
my $parser = $validator . "/run_dsrf.py";
my $conformance = $validator . "/conformance/conformance_processor.py";
my $log = "/tmp/example.log";
my $log2 = "/tmp/example2.log";
my $usage = "Please use the script as follows:\n\tdsrfconf.pl FILE [FILE*]\nProgram aborted";
my @filelist = sort (@ARGV);
my $filecount = $#ARGV + 1;
my $profile = "";
$/ = "\r";     # to support all kinds of CR/LF/CRLF combos

#
# see if at least one file has been given and that all files exist
#
die "No file to process. $usage" if ( $filecount < 1);
-f or die "File '$_' does not exist. $usage" foreach (@filelist);

#
# open the first file (which is ...1ofx... as the array is sorted) and obtain the profile
#
open (FILE, "<$filelist[0]");
while(<FILE>){
  s/^\s+//; # make sure there are no leading spaces
  next unless (/^HEAD/);
  my @fields = split "\t";
  $profile=$fields[2];
  last;
}
close FILE;

#
# end validation if there is no profile
# if there is: remove all spaces from the profile name (as this is what the tool expects)
#

if ( $profile eq "") {
  print "    No profile given in $filelist[0]\n    Conformance cannot be tested\n";
  open FILE, $filelist[0].log;
  print FILE "No profile given in $filelist[0]\n    Conformance cannot be tested\n";
  close FILE;
  exit;
}

print "Checking input file(s) against the $profile:\n\n";
$profile =~ s/ //g;

system "python $parser @filelist | python $conformance > $log2";
system "cat $log $log2 > $filelist[0].log";
system "rm -f $log $log2";
print "\n\nAll done. Results are in $filelist[0].log\n";

The two screen shots below show a successful and an unsuccessful test. In each case, a log file containing all findings is generated.