DDEX Standard

Skip to end of metadata
Go to start of metadata

 

 

 

1 Introduction

This standard was developed by the member organisations of the Digital Data Exchange, LLC (DDEX) and provides a message that gives a uniform mechanism to enable Licensees (typically Digital Service Providers) to report to Rights Controllers (typically Music Rights Societies, Music Publishers, Music Licensing Companies and/or Record Companies) information regarding the level of usage and/or revenue generated from the distribution of such products, as well as sales of products based on Releases, to the relevant Rights Controllers.

This version 3.0 of the Flat File Variant has been developed in response to concerns regarding the file size and computational complexity of the XML Variant, and will lead to reduced implementation and running cost for Licensees and Licensors  

This standard  defines the principles of the Flat File Variant of the Digital Sales Reporting Message Suite Standard. It is accompanied by a series of standards that define Profile-specific messages based on this standard.

Any organisation wishing to implement this (or any other DDEX standard) is required to apply for an Implementation Licence. The terms of the licence and an application form can be found on http://ddex.net/implementing-ddex-standards.


Essential Reading

Download/Print standard (PDF)

This standard is Part 1 of a multi-part standard. The other parts (incl. samples) are available here.

 

 

2 Scope

 2.1 Introduction
The message design defined in this standard provides a mechanism for Licensees (typically Digital Service Providers) to report to Rights Controllers (typically Music Rights Societies, Music Licensing Companies, Music Publishers and/or Record Companies) Usage, Revenue or Sales from the exploitation of Products based on electronic Releases containing Sound Recordings and/or Music Audio-Visual Recordings which embody Musical Works and/or other Resources.

The Flat File Variant is intended to provide a format for reporting information about the level of usage, revenue or sales generated from the exploitation of Releases and Resources for companies that do not wish to handle XML-formatted sales/usage reports.

This Part defines the architecture of the Flat File Variant of the Digital Sales Reporting Message Suite Standard. Specific Sales Report Messages, for well-defined Profiles, are defined in separate DDEX standards.

 2.2 Organisation of the Standard
This standard comprises nine clauses. Clauses 1-4 provide the scope, abbreviations and core definitions used in this standard. Clause 5 then defines the choreography for sending sales/usage reports created in accordance with this standard before Clause 6 looks at the overall structure of a Flat File Sales Reporting Message as well as defining technical details such as delimiters.

Clause 7 then provides information on how to communicate sales/usage reports for different Profiles in accordance with this standard. Finally, Clause 8  defines how to transmit a Flat File Sales Reporting Message using  SFTP.

 2.3 Release Notes (Informative)
Version 3 of the Flat File Variant of the Digital Sales Reporting Message Suite Standard utilities a fundamentally different architecture than Version 2. This new architecture allows more Profiles to be supported than was possible in Version 2.

Version 1.1 of the Architecture for this 3rd version of the Digital Sales Reporting Message Suite Standard allows communicating the version of Parts 2 (Allowed Value Sets) and 8 (Record Type Definition) that has been used to create a sales/usage report and strengthened the choreography and acknowledgement process.

3 Normative References

 Click here to expand...

The following normative documents contain provisions, which through reference in this text constitute provisions of this standard. For dated references, subsequent amendments to, or revisions of, any of these publications do not apply. For undated references, the latest version applies.

  • DDEX Data Dictionary Standard. Latest Version
  • DDEX Party Identifier (DPID) Standard. Latest Version
  • DDEX, Digital Sales Reporting Message Suite Standard - Part 8: Record Type Definitions
  • IETF. RfC2026. SSH File Transfer Protocol. October 2001
  • IFPI. Global Release Identifier (GRid) Standard. Latest Version
  • ISO 639-1:2002. Codes for the representation of names of languages — Part 1: Alpha-2 code. 2002
  • ISO 3166-1:1997. Codes for the representation of names of countries and their subdivisions – Part 1: Country codes. 1997
  • ISO 3901:2001. Information and documentation – International Standard Recording Code (ISRC). 2001
  • ISO 4217:2001. Codes for the representation of currencies and funds. Latest Version
  • ISO 8601: Data elements and interchange formats – Information interchange – Representation of dates and times. Latest Version
  • ISO 15707:2001. Information and documentation -- International Standard Musical Work Code (ISWC). 2001
  • ISO/IEC 10646, Information technology — Universal multiple-octet coded character set (UCS). Latest Version

4 Terms and Abbreviations

 4.1 Terms and Definitions
For the purposes of this standard, the following terms, written in the remainder of this standard with capital letters, should be read as having the meanings specified here. These definitions sometimes contain other capitalised terms. Most of those terms are relevant for this standard and are defined in this Clause as well. The remainder of the capitalised terms are defined for the purpose of supporting other DDEX standards and have no special meaning for the purposes of this standard.

Block

A group of Records in a Sales Report Message created in accordance with this standard, comprising the description of a “head” Release or Resource, its constituent Resource(s), any Releases that were created by the Message Sender based on the head Release (including single-track Releases), plus any associated Usage, Revenue or Sales Records. All Records in a Block carry the same Block Identifier and all Records with the same Record Identifier belong to the same Block.

All Records of a Block are contiguously provided in the Sales Report Message created in accordance with this standard.

A Block collates all Records within a report created in accordance with this standard that describe all Usage, Revenue or Sales of the “head” Release and its constituent parts or “head” Resource (notwithstanding summary Records and information contained in the header record, HEAD).

The identifier of a Block shall be unique within a Sales Report Message created in accordance with this standard.

Cell

An individual data element in a Record. Cells are separated by Delimiters.

Contractually Mandatory

An entity in a DDEX Message that has the technical cardinality of 0-1 or 0-n but that is mandatory when a DDEX message is sent in a specific commercial context.

Contractually Mandatory fields may, however, be mandatory when a DDEX message is sent in a specific commercial context. In such circumstances, a message is deemed conformant only if and when it contains all the “contractually mandatory” fields as agreed by Message Sender and Message Recipient.

Delimiter

A character that delineates, in a flat file, either one Record from another Record or one Cell from another Cell.

Note: there are also Secondary Delimiters used in this standard.

Flat File Sales Report Messages (DSRF)

A Sales Report Message created in accordance with the Digital Sales Reporting Message Suite Standards (Architecture and Profiles). A Flat File Sales Report Message may be split into several individual files.

Head Release

A Release, originally communicated by the owner (or its agent) of a sound recording or audio-visual recording to a Licensee who then distributed that Release to consumers, whose Usage, Revenue or Sales are described in a Sales Report Message created in accordance with this standard.

Head Resource

A Resource whose Usage, Revenue or Sales are described in a Sales Report Message created in accordance with this standard. Head Resources are not communicated as part of a Head Release.

Licensee

A Party to whom permission to use a Creation (e.g.: MusicalWork, Sound Recording or Release) is granted in a License Agreement.

Licensor

An Agent granting permission to use a Creation such a MusicalWork, Sound Recording or Release in a LicenseAgreement.

Note: In many cases a Licensor is also the RightsController.

Musical Work

A Work intended to be perceivable as a combination of sounds, with or without accompanying text.

Any words that are intended to be expressed with a MusicalWork (often termed Lyrics) form part of that MusicalWork; not all MusicalWorks have Lyrics.
A MusicalWork may be expressed and fixed to become part of a SoundRecording or a Video Recording, or may be used to create notated music (sheet music, scores, instrumental parts) or sound generation codes (such as MIDI files).
In some cases, the MusicalWork comes into existence simultaneously with its expression. This is common in extemporised forms such as jazz music.

Profile

A subset of a DDEX standard. Profiles define how to use the capabilities of a DDEX standard in a specific commercial context.

Record

A line in a flat file containing data about a specific data entity. The Record type is indicated in the first Cell. Data elements are contained in Cells which are separated by a specifically defined Delimiter.

Release

A Release is an abstract entity representing a bundle of one or more Resources compiled by an Issuer for the purpose of distribution to individual consumers, directly or through intermediaries. The Resources in Releases are normally primarily sound recordings or music audio-visual recordings, but this is not invariably the case. The Release is not itself the item of trade (or “Product”). Products have more extensive attributes than Releases; one Release may be disseminated in many different Products.

Resource

A digital fixation of an expression of an abstract Work (such as a sound recording, a video, an image, software or a passage of text). Resources are individual assets that make up a Release. Typical Resources are sound recordings, video clips and cover art images.

Revenue

The amount of money generated in accordance with the commercial relationship between Licensee and Licensor for the distribution of Releases to consumers.

Rights Controller

A Party who owns and/or controls rights in a Musical Work or other Creation.

Creations include Musical Works, Sound Recordings and other Resources as well as Releases. Rights Controllers are, in the context of a licence agreement, often referred to as Licensors.

Note: In many cases a RightsController is also the Licensor.

Sale

Distribution of a product to end consumers. For the avoidance of doubt, the term “sales” includes all forms of distribution of products, whether revenue was generated, or not and, where revenue is generated, regardless of the business model used to do so.

Sales/Usage Report Message

See Flat File Sales Report Messages (DSRF).

Secondary Delimiter

Delimiter used to separate data elements within a single Cell.

Sub-Release

A Release that is being created from one or more of the primary Resources of a Head Release, typically by the Message Sender.

For the purpose of this standard these Sub-Releases are being created by the Message Sender who is making Releases, and tracks from such Releases, available to consumers. These “unbundled tracks”, that is, all the tracks from a Release each made available separately, are the simplest form of Sub-Releases.

Usage

The identification of the Releases and Resources that actually get exploited by Licensees exercising a range of business and delivery models for the distribution of Releases to consumers, including the amount of use of each Release or Resource.

 4.2 Abbreviations
AMEPAutomated Message Exchange Protocol
ACAAppointed Certification Agency
AVSAllowed Value Set
BPBusiness Profile
CISAC

Confédération internationale des sociétés d'auteurs et compositeurs, the International Confederation of Societies of Authors and Composers (see cisac.org)

CACertification Agency
CTConformance Tester
DAWDigital Audio Workstation
DDEX

Digital Data Exchange

DSIGDigital Signature
DSPDigital Service Provider (incudes Mobile Service Providers)
DSRDigital Sales Reporting
ERNElectronic Release Notification
FTPFile Transfer Protocol (FTP specifically includes SFTP)
GRidGlobal Release Identifier
HTTPHypertext Transport Protocol  (HTTP specifically includes HTTPS)
HTTPSSecure Hypertext Transport Protocol
IECInternational Electrotechnical Commission (see iec.ch)
ISOInternational Organisation for Standardisation (see iso.org)
MIMEMultipurpose Internet Mail Extensions
MLCMusic Licensing Company
MWLMusical Works Licensing
MWNMusical Works Notification
MRBV

Multi-Record-Block Variant

PCAPrivate Certification Agency
PDFPortable Document Format
RESTREpresentational State Transfer
RINRecording Information Notification
SFTPSecure FTP
SRBV

Single-Record-Block Variant

TISTerritory Information System (a CISAC Standard)
TLSTransport Layer Security
UGCUser-generated content
URLUniform Resource Locator
XMLeXtensible Markup Language
XSDXML Schema Definition
W3CWorld Wide Web Consortium (see w3c.org)
WSWeb Service

5 Choreography

 5.1 Message Choreography
Figure 1 shows the choreography of processes that the Flat File Variant of the Digital Sales Reporting Message Suite enables. Table 1 summarises the point in the relevant business processes when the message is sent. The table also indicates who sends the message to whom.

The use of the Acknowledgement Message as defined in this standard is optional and the invoice message, although shown in Figure 1 and Table 1 is not defined in this standard.


Figure 1 – Choreography of the Flat File Variant of the Digital Sales Reporting Message Suite Standard

 

Table 1 – Messages in the Flat File Variant of the Digital Sales Reporting Message Suite Standard

Message NameInitiating EventSenderRecipient

Sales Report Message

Usage or sale of Releases, and/or the generation of revenues related to the usage or sale of Releases

Licensee, typically a DSP

Works Licensor (typically a Music Rights Society or Licensor) or Release Creator, typically a Record Company or Music Licensing Company

Acknowled-gement Message
(optional)

Automated acknowledgement of receipt of a Sales Report Message file; the acknowledgement message does not acknowledge the validity or quality of the Sales Report Message file(s) received.

This message is optional.

If a Sales Report is communicated in multiple files, one Acknowledgement Message should be sent for each file.

Works Licensor (typically a Music Rights Society or Music Publisher) or Release Creator, typically a Record Company or Music Licensing Company

Licensee (typically a DSP)

Invoice Message
(out of scope)

Invoice in reply to the Sales Report Message; the invoice message is out of scope for this standard.

Works Licensor (typically a Music Rights Society or Licensor) or Release Creator, typically a Record Company or Music Licensing Company

Licensee (typically a DSP)

Note that the Record Types of the Sales Report Message are defined in two places: Common Record Types are defined in this standard whereas all other Record Types are defined in separate standards.

 5.2 Multiple Linked Sales Report Messages
In some cases, a Message Sender may be required to send multiple Sales Report Messages to a specific Message Recipient. Examples include the provision of summary reports and detailed Sales Report Messages covering the same sales transactions.

In that case these Sales Report Messages are considered independent from each other and shall each be communicated in accordance with the choreography defined in Clause 5.1.

6 Structure

 6.1 Introduction
This Clause contains the specification for the messages in the Flat File Variant of the Digital Sales Reporting Message Suite.

 6.2 Single-File Sales Report Messages
The Flat File Sales Report Message defined in this standard is a delimiter-separated values text file. Sales Report Messages created in accordance with this standard comprise the following sequence of Records:
  1. Exactly one header Record of type HEAD as defined in Clause 8.2; 
  2. One or more Records in accordance with the relevant Profile Standard that provide summary information about the transactions listed in item  3 below ;
  3. None, one or more Blocks of Records in accordance to the relevant Profile Standard that provide detailed information about transacted Releases/Resources/Works and the Usage, Revenue or Sales of such transacted Releases/Resources/Works; and
  4. Exactly one Footer Record of type FOOT as defined in Clause 8.3.

This structure is shown in the Figure 2 (note: the indentation is for illustration purposes only).

Figure 2 – Overall structure of a Sales/Usage Report Message created in accordance with this standard

 

It is permissible to send a Sales Report Message with just a header Record of type HEAD and a Footer Record of type FOOT and no other Records. This would indicate that no Usage, Revenue or Sales are being reported.

 

 6.3 Multi-file Sales Report Messages
For technical reasons it is possible to split a Sales Report Message into several files.
  • All files shall start with a header record (HEAD) as defined in Clause 8.2. Some data in some of the Cells in the header record will vary between the various files;
  • All files shall have the same set of Summary Records containing summary information for the entire multi-file Sales Report Message (for the avoidance of doubt: the sets in the various files shall all contain the same data);
  • All files shall have one or more Blocks;
  • All files shall end with a footer record (FOOT) as defined in Clause 8.3. Some data in some of the Cells in the footer record will vary between the various files.

Blocks may not be split between files and may not be repeated in different files of the same Sales Report Message.

The last file can be determined in the HEAD record by comparing the FileNumber and NumberOfFiles Cells.

This structure is shown in the diagram below (note: the indentation and spacing is for illustration purposes only).

Figure 3 – Overall structure of a Multi-file Sales/Usage Report Message in accordance with this standard

 6.4 Blocks for Reporting Usage, Revenue or Sales

6.4.1 Introduction

 The main portion of a Sales Report Messages created in accordance with this standard is organised in Blocks. Each Block contains all the information about the “item of trade” that allows the Message Recipient to identify the content used, followed by the usage, revenue or sales information.

All Records of a Block are contiguously provided in the Sales Report Message created in accordance with this standard.

6.4.2 BlockIDs

All Blocks contain a BlockID Cell. The first Block in the Sales Report Messages shall have a BlockID containing the value 1, the next Block shall  have a BlockID containing the value 2, etc. 

This mechanism allows all Records in a Block to be associated with one another. If, for instance, a Block contains a Head Release, a number or Resources and a few Sub-Releases, the BlockID can be used to link the Sub-Releases to the Head Release. Together with the ReleaseRecord Cell, the BlockID also allows to unambiguously link a Sub-Release to its containing Resources.

6.4.4 Linking Sales/Usage Records to Release and Resource Records

In cases where a Block can contain more than one Release or Resource that might have been used or transacted, the Sales/Usage Record needs to indicate the Release (or Resource) it provides Usages, Revenues or Sales for. This is effected by providing the content of the ReleaseReference Cell (or ResourceReference Cell) of the used Release (or Resource) in the TransactedRelease Cell (or TransactedResource Cell) of a Sales/Usage Record.

 6.4.5 Linking Blocks to Summary Records

All Blocks contain Records that report Usages, Revenues or Sales.[1] It is essential for a Message Recipient to be able determine to which Summary Record such Usage/Revenue/Sales Records contribute. To allow this, all Summary Records contain a SummaryRecordId Cell that Usage/Revenue/Sales Records can point to. Figure 4 shows an excerpt from a Sales Report Message in which two different types of exploitation (see Clause 6.5 for details) are being reported with respect to two different Releases (reported in Records of Type RE01).


Figure 4 – Linking Blocks to Summary Records (Example)


[1] In some profiles it may be valid to send Blocks without any such Records.

 

 6.5 Scope of Sales Report Messages
A Sales Report Message created in accordance with this standard may contain data relating to multiple territories, multiple currencies, multiple use types and multiple commercial model types as well as different service descriptions (collectively sometimes referred to as the sales context).

Where more than one sales context is reported in a single Sales Report Message, Message Sender and Message Recipient both must agree how this is managed. Specifically, if either Message Sender or Message Recipient is unable to either create or ingest such multi-featured Sales Report Messages, it is not permitted to use that approach. In that case only one sales context, which would either be evident from the relevant Summary Record or agreed between Sender and Recipient, shall be contained in a Sales Report Message.

 6.6 Technical Details

 6.6.1 Baseline Format
The Sales Report Messages defined in this standard are delimiter-separated values text files. Further details are provided below.

 6.6.2 Character Coding
The Sales Report Messages defined in this standard are coded using UTF-8 with a big endian byte ordering.

 6.6.3 Delimiters

6.6.3.1 Record Delimiter

Each Record is being placed into one line terminated by a line feed (Unicode U+000A) or a carriage return and line feed pair (Unicode U+000D 000A).

6.6.3.2 Primary (Cell) Delimiter

Cells within a Record are separated by tab characters (Unicode U+0009).

6.6.3.3 Secondary  Delimiter

Should a Cell contain two or more data elements, these data elements shall be separated by a pipe character (Unicode U+007C). This is referred to as a Secondary Delimiter.

All data elements in a multi-value Cell shall be of the same primitive data type.

6.6.3.4 Namespace Delimiter

Should a Cell contain a data element whose providence needs to be provided, the data element shall be preceded by a string that provides a "namespace" and two colon characters (Unicode U+003A). The double colon is referred to as a Namespace Delimiter.

Examples include specifically identifiers where, e.g., a party ID can be communicated as  "ISNI::0000000081266409", indicating that the identifier (0000000081266409) is an International Standard Name Identifier (ISNI).

6.6.3.5  Spaces and Delimiters (I)

Delimiters shall not be surrounded by extra space characters. The same principle applies to the Secondary Delimiter.

Example: The writer pair Lennon/McCartney should be communicated as "Lennon|McCartney" and not as "Lennon | McCartney".

6.6.3.6 Spaces and Delimiters (II)

If a Message Sender has received data with extra white spaces, they are encouraged to trim any such extra white space characters when compiling a Sales/Usage Report. They may, however, also use the data with these extra white space characters in such cases. The same principle applies to the Secondary Delimiter.

Example: When the Message Sender received Lennon as “Lennon “ and McCartney as “McCartney “, then the writer pair may be communicated as “Lennon |McCartney “).

 6.6.4 Communicating Delimiters
To communicate Delimiters in a Cell, such a Cell shall not be enclosed in double quote characters. Instead the Delimiters shall be immediately preceded by a backslash character (Unicode U+005C). Therefore the string A|B would have to be communicated as A\|B.

To communicate a backslash character, two backslash characters need to be communicated. Therefore the string A\B would have to be communicated as A\\B.

This “escaping mechanism” must be used for all special characters in all Cells, whether those Cells allow multiple values or not. A non-escaped pipe character in a single-value cell is, consequently, an error.

For the avoidance of doubt, escaping a character that should not be escaped, or not escaping a character that should have been escaped, will lead to an invalid sales/usage report.

 6.6.5 Primitive Data Types
Within this standard the following primitive data types are used:

Table 2 – Primitive Data Types

Data TypeDefinitionExample

String

A sequence of characters with a length of at least one character. This standard does not define a maximum length.

Johannes Brahms

Boolean

A string containing either “true” or “false”

true

Integer

A sequence of digits to represent positive or negative integer numbers, or zero.

135672

Float

A sequence of digits to represent positive or negative integer numbers, positive or negative decimal fractions or zero.

The character used to separate the integers from fractions is the dot (“.”, Unicode U+002E).

Thousands separators or any other digit grouping shall not be used.

Details on the precision of decimals are provided in Clause 6.6.11.

123

123.567

-5.25

0

Date

A date in the ISO 6801 format to indicate a single year (YYYY), month (YYYY-MM) or day (YYYY-MM-DD).

2015-01-09

Duration

A duration in the ISO 6801 format (P[nY][nM][nD][T[nH][nM]nS]).

Elements including their designator may be omitted if their value is zero.

Note that the expressions “PT3M2S” (three minutes and two seconds) and “PT182S” (182 seconds) are both permitted and are equivalent.

PT3M2S

PartyIDA string in the form IdType::ID where IdType identifies the identification scheme. Examples are ISNI for an ISO 27729 International Standard Name Identifietr, IPI for a CISAC Interested Party Identifier or IPN for a SCAPR International Performer Number. The ID element then contains the identifier in the format defined by the Identificaiton scheme. Only one identifier can be communicated for each party in a PartyID field.ISNI::0000000081266409

DDEX Party ID

A string of 18 characters in accordance with the DDEX Party ID standard.

Note that the DDEX Party ID standard defines that DDEX Party IDs do not contain dashes in computer-to-computer communications. Therefore, DDEX Party IDs, when included in a Sales Report Message, shall not contain dashes.

PADPIDA2014122301Q

 6.6.6 Splitting Sales Report Messages
A Message Sender may separate a Sales Report Message into several individual files. When a Message Sender does so, the following rules apply:
  1. All files shall carry the header Record(s) defined in Clause 8.2; and
  2. All files shall carry the footer Record(s) defined in Clause 8.3.

It is not permissible to split a Block. Therefore a Block will always be in a single file.

 6.6.7 Empty Records
It is not permissible to include empty Records in a Sales Report Message (see also Clause 6.6.9).

 6.6.8 Empty Cells
It is possible to communicate empty Cells. In that case, two tab characters shall follow each other with no characters in-between.

The semantics of an empty cell is determined by the commercial relationship between Message Sender and Message Recipient, unless specifically defined in this standard.

 6.6.9 Records to be Ignored
Records whose first character is the hash symbol (“#”, Unicode U+0023) shall be ignored by automated ingestors. These Records are included solely to aid human readability (see also Clause 6.6.17).

 6.6.10 Unknown Records
A company receiving a file with a Record containing information identified by an unknown Record Type Indicator may ignore such Records.

 6.6.11 Communicating Decimals
This standard does not prescribe a specific precision in which decimal numbers are to be given. Message Sender and Message Recipient must agree a mutually agreeable precision for the type of transactions at hand.

Implementers are reminded that some accounting applications may only be able to handle six, or even four, decimal places.

When a field of type “decimal” is to communicate an integer, the number may be represented with a trailing decimal point, a trailing “.0” or with no trailing characters. For example, the number five can be represented as “5”, “5.” for “5.0”.

When the number 0 (zero) is to be communicated in a Cell for decimal values, this shall be represented as 0 (and not 0.0 or 0.000000).

 6.6.12 Maximum Cell Size
While some Cells have an inherent length limitation (e.g. the Record Type Indicators are limited to four characters), other Cells have no length limitation.

Note: this has an implication for developers using databases with field length limitations. Such developers will be forced to shorten fields if the data received exceeds their databases’ capabilities.

 6.6.13 Maximum File Size
An uncompressed file containing a Sales Report Message created in accordance with this standard may not exceed 4GB (4,000,000,000 Bytes). Where the data for a Sales Report Message exceeds this file size, separate files must be created in accordance with Clause 5.2.

 6.6.14 Allowed Values
Allowed values are listed, defined and provided in the DDEX Standard "Digital Sales Reporting Message Suite Standard, Part 2 – Allowed Value Sets" in accordance with the DDEX Data Dictionary and the latest version of the DDEX Data Dictionary Standard.

To override a DDEX-defined value, the DDEX-defined value “UserDefined” shall be used, followed by single space character and a string representing the term to be communicated. This string shall only comprise lower case letters, upper case letters or digits, e.g. UserDefined StreamInOnlineGame.

Such user-defined values will need to be agreed between Message Sender and Message Recipient and users are encouraged to raise these issues with DDEX (e.g. via http://www.ddex.net/implementers-forum) with a view to having these values added to the list of DDEX-defined allowed values.

 6.6.15 Linking Records
In order to link Records within a Block, some Records contain a “reference” element. Examples include the ReleaseReference Cell in the Records describing Releases and ResourceReference in Records describing Resources. These references must be unique within the Block they appear in. As a consequence, the combination of a Record’s BlockId and reference is unique within a Sales Report Message created in accordance with this standard.

Should a Cell need to reference a Record in a different Block, the syntax shall be BlockId:Reference.

To allow linking Usage/Revenue/Sales Records to the Summary Record they contribute, all Summary Records and Usage/Revenue/Sales Records contain a SummaryRecordId Cell.

Some records, such as the SalesTransactionID in the Sales/Usage Records need to be referenced from other messages such as an invoice message. In that case the combination of MessageId (as communicated in the HEAD Record), BlockId and reference are globally unique.

 6.6.16 Version
The Version of this standard is 3.1 shall be communicated in the Header Record HEAD in accordance with Clause 5.1.1 of Part 8.

 6.6.17 Aiding Human Readability
Sales Report Messages created in accordance with this standard are produced so that they can be ingested and processed automatically. However, when establishing a new data feed as well as for error handling during normal operation it may be helpful for humans to be able read a Sales Report Message.

It is therefore recommended, for each Record Type used in a Sales Report Message created in accordance with this standard, to include a Record that provide Cell “headings” for that Record once. The first Cell of such Records is populated with the relevant Record Type and shall be preceded by a hash symbol (“#”) in accordance with Clause 6.6.9 and as shown in Figure 4 in blue.

Note, that if such headings are over-used and/or repeated too often, they could unnecessarily inflate the file size (especially for large Sales Report Messages).

Figure 5 – Human readable layout of a Sales Report Message

Figure 5 shows the Sales Report Message loaded into a spreadsheet application, with some rows coloured to aid readability. 

 6.6.18 Reporting no Sales or Usages
It may be that a company wishes to send a sales/usage report to a Licensor Such a report shall comprise of four parts:
  • One Header Record (HEAD) in accordance with Part 8 of this standard;
  • One or more summary records in accordance with the  relevant Profile. The number of summary records would depend on the contractual relationship between Licensor and licensee (see below).
  • No Record to describe Usages, Revenues or Sales; and
  • One Footer Record (FOOT) in accordance with Part 8 of this standard.

If the "empty" sales/usage report is the first report ever sent from licensee to Licensor, it is recommended that one summary record is provided for each sales/usage context that the licensee is licensed for by the Licensor. Licensee and Licensor may, however, agree a different approach.

For an "empty" sales/usage report following a non-"empty" sales/usage report, it is recommended to provide a summary record for each of the contexts provided in the preceding sales/usage report. Licensee and Licensor may, however, agree a different approach.

The image below shows such an empty sales/usage report:

Figure 6 – "Empty" Sales/Usage Report.

7 Profiles

 7 Profiles

This standard  defines the architecture for all flat file sale/usage reports, It also defines common Record Types used in all sales/usage reports formatted in accordance with this standard but does not define Record Types for specific Profiles.

These Record Types are defined in Profile-specifc standards which are, in turn, built upon this standard. At the time of writing this standard, five Profile standards have been defined by DDEX:

  • Basic Audio Profile
  • UGC Profile
  • AV Profile
  • Royalty Reporting Profile
  • Broadcast Reporting Profile

DDEX expects to add further Profile standards over time. They will be published on the DDEX Knowledge Base where a complete list of Profiles defined for Version 3 of the Flat File Variant can be found.

8 Transfer of Sales Report Messages

 8.1 File Naming Convention
All files that make up a Sales Report Message shall be named in accordance with the following syntax:

DSR_MessageRecipient_MessageSender_ServiceDescription_MessageNotificationPeriod_TerritoryOfUseOrSale_xofy_MessageCreatedDateTime.ext

With:

MessageRecipient

The Party Name or DDEX Party ID (DPID) of the Message Recipient.

The MessageRecipient may be omitted if the Sales Report Message is sent to more than one recipient. This information shall be the same as the information conveyed in the file Header (HEAD). Note: DDEX Party IDs do not contain special characters.

MessageSender

The Party Name or DDEX Party ID (DPID) of the MessageSender.

This information shall be the same as the information conveyed in the appropriate Record. Note: DDEX Party IDs do not contain special characters.

ServiceDescription

A description of the service name (e.g. a service tier) to be reported on. Multiple tiers can be communicated by separating them with dashes.

This information shall be the same as the information conveyed in the file Header (HEAD).

Message-
Notification-
Period

The reporting Period covered by the Message in accordance with ISO 8601:2004. The only allowed syntaxes are:

  • yyyy for a year
  • yyyy-mm for a month
  • yyyy-mm-dd for a day
  • yyyy-Qq for a quarter
  • yyyy-Www for a week (starting on a Monday)
  • yyyy-mm-dd--yyyy-mm-dd with the two dates being the start and end date of the period (note the two dashes between the two dates).

TerritoryOf-
UseOrSale

The territory for which the Usage, Revenue or Sale transactions in this message are reported.

The TerritoryOfUseOrSale shall be provided as a single ISO Territory Code if one territory is to be communicated. Otherwise, the MessageSender and MessageRecipient(s) shall agree a value acceptable to both/all.

xofy

The file counter.

For example, the 7th file of a 9-file Sales Report Message would be 7of9. This element may be omitted when the Sales Report Message is contained in a single file.

MessageCreated-DateTime

The date and time on which the Message was created (the only allowed format is the full basic ISO 8601:2004 format without a time zone designator: yyyyymmddThhmmss). The time zone is assumed to be the one of the MessageSender’s  location.

ext

The file extension, to be “.tsv.
When the file is compressed, for example with gzip, the extension should indicate this and may be .tsv.gz.

The following constraints apply:

  1. None of the file name elements may contain an underscore character.
  2. None of the file name elements may contain an space character.
  3. When a file name element is to be omitted, the two underscore characters to the left and right follow each other immediately.
  4. Other file naming conventions may be agreed on a bilateral basis (specifically for testing).

 8.2 Sample File Names (informative)
A single-file Sales Report Message from a Company with DPID PA-DPIDA-2014122301-Q to a company with a DPID of PA-DPIDA-2015010702-O for a download service for the first quarter of 2015 in Germany, sent on 15th April 2015 at two minutes after 10am would be contained in a file called (note the two underscore characters between the territory code and the time stamp):

DSR_PADPID2015010702O_PADPIDA2014122301Q_Download_2015-Q1_DE__20150415T100200.tsv

 

If the above report would have been sent to multiple MessageRecipients applying to multiple territories (with all parties involved having agreed to use the string “multi” to indicate a multi-territory report), and would be split into several files, one of these files might be named as follows note the two underscore characters between the DSR prefix and the sender's DPID):

DSR__PADPIDA2014122301Q_Download_2015-Q1_multi_7of9_20150415T100200.tsv

 

The third example file name is for a single-file report sent from Apple iTunes to SACEM for a download in the FR covering the period from 15th January to 15th February 2015 could be:

DSR_SACEM_Apple_Download_2015-01-15--2015-02-15_FR__20150415T100200.tsv

 8.3 Choreography for Sending and Acknowledging Sales/Usage Reports
The transfer of sales/usage reports defined in accordance with this standard should be effected using SFTP.

Message Sender and Message Recipient may use the following approach to exchanging (and acknowledging) Sales/Usage Reports created in accordance with this standard.

  1. The Message Sender shall upload the file containing the Sales/Usage Report to a mutually-agreed folder on a mutually-agreed SFTP server;
  2. Once the Sales/Usage Report has been uploaded completely, the Message Sender shall place a zero-byte semaphore into a sub-folder of the folder containing the sales/usage report file.
    The sub-folder shall be called file_available/ and the semaphore file shall have the same file name as the Sales/Usage Report file, albeit with a underscore prefix and a file extension of ctrl instead of tsv; and
  3. Should a Sales/Usage Report comprise of more than one file in accordance with Clause 6.3 , the Message Sender shall repeat Steps 1 and 2 for each constituent file.
  4. The Message Recipient shall download a Sales/Usage Report file once it has seen the relevant semaphore file;
  5. Once the Message Recipient has successfully downloaded the file it shall delete the relevant semaphore file from the SFTP server;
  6. The Message Recipient may also delete the sales/usage report file from the SFTP server at this stage; and
  7. Should a Sales/Usage Report be comprised of more than one file in accordance with Clause 6.3 , the Message Recipient shall repeat Steps 1, 2 and potentially 3 for each constituent file.
  8. Should Message Sender and Message Recipient agree to communicate acknowledgements for Sales/Usage Reports, these should be placed, by the Message Recipient into a sub-folder of the folder containing the Sales/Usage Report file(s) called acknowledgements/. The name of the acknowledgement file shall be the name of the acknowledged Sales/Usage Report, albeit with three changes:
    • The name of the acknowledgement shall be preceded by ACK_ instead of DSR_;
    • The xofy part of the filename shall be omitted (in case the acknowledged Sales/Usage Report was provided in more than one file); and
    • The time stamp shall be replaced with a time stamp of the acknowledgement, expressed in the same syntax as the time stamp used in the file name of the Sales Report Message file.

DDEX may define the syntax of the acknowledgement message in a separate Part of this standard at a later time,

 8.4 Sample SFTP Server Organisation (informative)
The  box below shows how the structure of the SFTP server after the Message Sender has uploaded a single file Sales/Usage Report:
sftp://server.com/some/folder/
            DSR_PADPID2015010702O_PADPIDA2014122301Q_Download_2015-Q1_DE__20150415T100200.tsv
            file_available/
                   _DSR_PADPID2015010702O_PADPIDA2014122301Q_Download_2015-Q1_DE__20150415T100200.ctr


The box below shows how the structure of the SFTP server after the Message Recipient has downloaded the single file Sales/Usage Report and has removed the semaphore file:


sftp://server.com/some/folder/
            DSR_PADPID2015010702O_PADPIDA2014122301Q_Download_2015-Q1_DE__20150415T100200.tsv
            file_available/

 

 

The box below shows how the structure of the SFTP server after the Message Sender has uploaded a multi-file file Sales/Usage Report. Message Sender and Recipient have agreed that the Recipient provides an acknowledgement on Sales/Usage Report level:

sftp://server.com/some/folder/
            DSR__PADPIDA2014122301Q_Download_2015-Q1_multi_1of4_20150415T100200.tsv
            DSR__PADPIDA2014122301Q_Download_2015-Q1_multi_2of4_20150415T100200.tsv
            DSR__PADPIDA2014122301Q_Download_2015-Q1_multi_3of4_20150415T100200.tsv
            DSR__PADPIDA2014122301Q_Download_2015-Q1_multi_4of4_20150415T100200.tsv
            file_available/
                 _DSR__PADPIDA2014122301Q_Download_2015-Q1_multi_2of4_20150415T100200.ctlr
                 _DSR__PADPIDA2014122301Q_Download_2015-Q1_multi_2of4_20150415T100200.ctlr
                 _DSR__PADPIDA2014122301Q_Download_2015-Q1_multi_3of4_20150415T100200.ctlr
                 _DSR__PADPIDA2014122301Q_Download_2015-Q1_multi_4of4_20150415T100200.ctlr
            ack/


The box below shows how the structure of the SFTP after the Message Recipient has downloaded all files of the multi-file file Sales/Usage Report and subsequently acknowledged the entire Sales/Usage Report:

sftp://server.com/some/folder/
            DSR__PADPIDA2014122301Q_Download_2015-Q1_multi_1of4_20150415T100200.tsv
            DSR__PADPIDA2014122301Q_Download_2015-Q1_multi_2of4_20150415T100200.tsv
            DSR__PADPIDA2014122301Q_Download_2015-Q1_multi_3of4_20150415T100200.tsv
            DSR__PADPIDA2014122301Q_Download_2015-Q1_multi_4of4_20150415T100200.tsv
            file_available/
            ack/
                 ACK__PADPIDA2014122301Q_Download_2015-Q1_multi_20150416T093010.tsv            

Evaluation Licence for DDEX Standards

 

 

Subject to your compliance with the terms and conditions of this Agreement, DDEX™ grants you a limited, nonexclusive, non-transferable, non-sublicenseable, royalty-free licence solely to reproduce, distribute within your organisation, and use the DDEX standard specifications (“DDEX Standards”) solely for the purpose of your internal evaluation. You may not make any commercial use of the DDEX Standards under this agreement. No other licences are granted under this agreement.

No representations or warranties (either express or implied) are made or offered by DDEX with regard to the DDEX Standards. In particular, but without limitation, no representations or warranties are made in relation to:

  1. The suitability or fitness of the standards for any particular purpose;
  2. The merchantability of the standards;
  3. The accuracy, completeness, relevance or validity of the standards; or
  4. The non-infringement of any third party intellectual property rights related to the DDEX Standards.

Accordingly, DDEX and/or its members shall not be liable for any direct, indirect, special, consequential or punitive loss or damages howsoever arising out of or in connection with the use of the standards. IN THE EVENT THAT ANY COURT OF COMPETENT JURISDICTION RENDERS JUDGEMENT AGAINST DDEX AND/OR ITS MEMBERS NOTWITHSTANDING THE ABOVE LIMITATION, THE AGGREGATE LIABILITY TO YOU IN CONNECTION WITH THIS AGREEMENT SHALL IN NO EVENT EXCEED THE AMOUNT OF ONE HUNDRED U.S. DOLLARS (US$ 100.00).

Users of the DDEX Standards are cautioned that it is subject to revision. Users are recommended to use the latest versions, which are available at http://www.ddex.net. The use of outdated versions of the standards is not recommended but may be required by agreement between implementers in particular cases.