Flat-file validation
The validation of a flat-file message such as a TSV file generated in accordance with the Digital Sales Reporting Message Suite (DSR), the Claim Detail Message Suite (CDM) and the Recording Data and Rights Revenue Reporting (RDR-R) can be undertaken using a tool developed by Google. The tool is available to download from GitHub at https://github.com/ddexnet/dsrf .
The tool can be deployed on any computer running Mac OS X, LinuX or MS Windows with a Python v2.7 (or later) interpreter installed. The tool comes with a manual and can be used with a small Perl script dsrf-conf.pl
.
The script is provided below.
#!/usr/bin/perl
use strict;
use warnings;
print "\ndsrf-conf.pl -- tool to check conformance of a DSRF file. \n"
print "(c) 2016 Digital Data Exchange, LLC, utilising Google's dsrf library\n\n";
#
# set variables (filelist being the alphabetical list of input files)
#
my $validator = "/Users/nrump/bin/dsrf";
my $parser = $validator . "/run_dsrf.py";
my $conformance = $validator . "/conformance/conformance_processor.py";
my $log = "/tmp/example.log";
my $log2 = "/tmp/example2.log";
my $usage = "Please use the script as follows:\n\tdsrfconf.pl FILE [FILE*]\nProgram aborted";
my @filelist = sort (@ARGV);
my $filecount = $#ARGV + 1;
my $profile = "";
$/ = "\r"; # to support all kinds of CR/LF/CRLF combos
#
# see if at least one file has been given and that all files exist
#
die "No file to process. $usage" if ( $filecount < 1);
-f or die "File '$_' does not exist. $usage" foreach (@filelist);
#
# open the first file (which is ...1ofx... as the array is sorted) and obtain the profile
#
open (FILE, "<$filelist[0]");
while(<FILE>){
s/^\s+//; # make sure there are no leading spaces
next unless (/^HEAD/);
my @fields = split "\t";
$profile=$fields[2];
last;
}
close FILE;
#
# end validation if there is no profile
# if there is: remove all spaces from the profile name (as this is what the tool expects)
#
if ( $profile eq "") {
print " No profile given in $filelist[0]\n Conformance cannot be tested\n";
open FILE, $filelist[0].log;
print FILE "No profile given in $filelist[0]\n Conformance cannot be tested\n";
close FILE;
exit;
}
print "Checking input file(s) against the $profile:\n\n";
$profile =~ s/ //g;
system "python $parser @filelist | python $conformance > $log2";
system "cat $log $log2 > $filelist[0].log";
system "rm -f $log $log2";
print "\n\nAll done. Results are in $filelist[0].log\n";
The two screen shots below show a successful and an unsuccessful test. In each case, a log file containing all findings is generated.