DDEX recently created a new and powerful flat-file standard for sales/usage reporting (DSR). DDEX recommends to use this standard. For users requiring the older versions – flat-file or XML-based – should contact the DDEX Secretariat. This section of the DDEX Knowledge base contains:
What is the aim of DDEX flat-file DSR Format?
The aim of DDEX's flat-file DSR format is to enable licensees to report to rights controllers information regarding the level of usage and/or revenue generated from the distribution of music or videos.
The flat-file format released in 2016 addresses the requirements for sales/usage reports to owners and controllers of rights in musical works. DDEX has, however, started looking at requirements for sales/usage reports to owners and controllers of rights in sound recordings and expects to extend the standard in the future.
The new flat-file format replaces the previous XML format, which was in production for close to a decade, as it ultimately proved to be unable to meet the requirements for a format that has to be able to be easily adapted to address the technical challenges of the ever-changing digital music industry.
The approach taken by DDEX when developing the flat-file DSR is very pragmatic as it offers simplicity but can be used (or: extended) to all business models.
Why should a company use the flat-file DDEX DSR Format?
- Already widely adopted: Even if the standard has only been published in 2016, this new standard is already widely adopted by many actors in the digital music industry;
- Simple to use and implement: Experiences from early adopters show that the format is very easy to understand, read and implement;
- Business reporting flexibility: It is possible to combine several offers and/or territories in a single sales/usage report. This allows avoiding to repeat the same information multiple times;
- Reports volume decreasing: The format keeps reports as compact as possible. The flexibility described above combined with an optimized flat-file structure reduces volumes drastically: Up to 100s of times when compared to the old XML format;
- Enrichment of necessary information: Data fields not used in the XML format were removed from the new standard whole others were added (e.g. identifiers for parties);
- Processing efficiency: the old XML format obliged receivers to compute and store first all Release, Resource and Work data before being able to process related sales/usage data. The new standard combines Release, Resource and Work data and sales lines into self-contained “block” that can be processed independently form one another;
- Cheap to implement and to run: One of the consequences of having a simple structure is the reduced cost for development and maintenance. Combined with the reduction of data volumes to be exchanged and processed, this allows a cut in operational costs; and
- Flexibility in terms of evolution of the format: The new format is based on a logic that allows the standard to be easily adapted and exchanged to changes in reporting requirements.
How to get started?
The first thing is to check which profile you need to use. This decision will depend on the business model (or models) you use. Once you have made this decision you can download the appropriate standard from the red box on the right hand side.
In order to encompass all business models, DDEX created different profiles in order to avoid creating generic format that becomes complex in the end. In order to facilitate implementation, these profiles use the different records and value sets and the same basic architecture described in a few generic documents.
In any case, implementers need to have a look at:
- Part 1: Architecture: general technical structure of DDEX DSR Flat-file;
- Part 2: Allowed-value Sets: all value sets used for different fields; and
- Part 8: Record Type Definitions: description of the records used in the different profiles;
- The Part (or Parts) of the flat-file DSR standard that documents the profile (or profiles) relevant to the implementer's operation (see next question); and
- The samples (providing at least one example for each profile).
Which profile should a company use?
Business model addressed
Basic Audio Profile
The aim of the Basic Audio Profile is to enable licensees to report to rights controllers information regarding the level of usage and/or revenue generated from the distribution of music.
The aim of the UGC Profile is to enable licensees to report to rights controllers information regarding the level of usage and/or revenue generated from the usage of music where content may have been uploaded to a licensee's platform by any user which is not necessarily the owner or right controller of the content.
The aim of the Audio-visual Profile is to enable licensees (typically digital service providers) to report to rights controllers (typically audio-visual rights societies) information regarding the level of usage and/or revenue generated from the distribution of audiovisual recordings (excluding musical videos).
Royalty Reporting Profile
The aim of the Royalty Reporting Profile is to provide a standardised mechanism for Licensees to report to Rights Controllers Royalties based on usages of musical Releases that took place in countries where there is a direct contractual relationship between Message Sender and Message Recipient.
Radio Broadcast Profile
The aim of the Radio Broadcast is to enable Licensees to report to Rights Controllers information regarding the level of usage and/or revenue generated from the distribution of Music in the context of Global Radios.
|Profile for Financial Reporting to Record Companies|
The aim of the Profile for Financial Reporting to Record Companies is to report to record companies sales and/or usage information.
The aim of the Masterlist Profile is to allow communicating Release, Resource and Work information before any usage has been generated or before such usage is ready to be communicated.
Single-Record Block Variant or Multi-Block Variant?
DDEX has defined some of its profiles in two forms:
- Multi-Record Block Variants (MRBV) and
- Single-Record Block Variants (SRBV).
For most profiles, the MRBV is the default.
In MRBVs the main body of the sales/usage report is made out of "Blocks" which contain multiple Records. The Blocks in the Basic Audio Profile, for instance is made up of one Record the Release that has been transacted, one or more Records for each of the Resources that make up the Release. For each of the Resource Records there may then be none, one or multiple Records describing Musical Works that are used in each of the Resources. Further Records can then be used to describe Sub-Releases and, finally, provide different sales/usage figures. This is depicted on the right.
SRBVs, on the other hand, collapse all of the above information into one single, albeit potentially long, Record. This makes the ingestion process simpler. Also, smaller SRBV files can easier be imported into a spreadsheet application. These benefits do come at the cost of potentially repeating a significant amount of information. MRBVs are therefore notably more efficient than SRBV.
The choice between MRBV and SRBV depends on multiple factors that sender and recipient need to weigh up:
- How large is the report likely to be? Is it a problem if the report is, multiple times larger because of data repetition?
- Is it important to easily import the sale/usage report into a spreadsheet application?
- Will sender and recipient be able to handle richer data structures?
How to concretely implement the standard and start using it?
The steps for successfully implementing the flat-file DSR standard are
- Identification of the Profile or Profiles that need implementing;
- Taking-out an implementation licence and obtaining a DDEX Party ID on dpid.ddex.net (more information on DPIDs is available here);
- Reading the relevant documentation (i.e. Parts 1, 2 and 8 as well as the Part for the relevant Profile(s)) and looking at the appropriate samples (see the red box on the right);
- As a sender of a sales/usage report it is recommended that a DSRF file is tested against the "dsrf validator tool" that can be downloaded from GitHub. While this tool cannot determine in all cases whether a DSRF file is valid, it can help implementers to identify any issues. This can be done only as part of the implementation and/or onboarding phase or as a general test before each sales/usage report is delivered;
- Recipients may find it prudent to use the "dsrf validator tool" before ingesting a received sales/usage report into their system to reduce the likelihood of buggy reports being processed.
Steps 1 and 3 as well as 5 and 6 are recommended to be undertaken with a specific partner in mind and
Delimiters and Special Characters
Clause 6.6.4 of Part 1 states that "to communicate Delimiters in a Cell [...] the Delimiters shall be immediately preceded by a backslash character (Unicode U+005C). Therefore the string A|B would have to be communicated as A\|B.":
Assume that a creation has three contributors: Paul Simon, Art Garfunkel and 24|7. This will need to be communicated as
Note that there are no quotes used to enclose either the complete string or its three components. If one adds a fourth contributor called “Mad Dog” Dolly, one would have
Assume that a creation has three contributors: Paul Simon, Art Garfunkel and 24\7. This will need to be communicated as