DDEX Standard

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 9 Next »

 

This section of the DDEX Knowledge Base contains Part 1 of Version 1.7 of the "ERN Choreography Standard" addressing Web Service Exchanges

 

1 Introduction

DDEX has standardised a series of Message Suite Standards that define the syntax and semantics of business metadata exchanged by members of the digital media delivery chain. DDEX has also standardised an automated message exchange protocol that enables the reliable communication of DDEX messages and associated documents such as resource files, using SFTP.

This standard defines, together with the Release Notification Message Suite Standard and its Profiles, a uniform mechanism for Release Creators to provide Release Distributors with Releases (metadata and Resource files) as well as Deals for making such Releases available by means of web services.

Any organisation wishing to implement this (or any other DDEX Standard) is required to apply for an Implementation Licence. The terms of the licence and an application form can be found at https://ddex.net/implementation/implementation-licence-and-ddex-party-identifiers.

2 Scope

 2.1 Introduction
This standard provides a standardised means for Release Creators to make Releases available via Release Distributors. It defines an Atom-based web service architecture. A separate standard defines a message exchange protocol for SFTP.

 2.2 Organisation of the Document
Clauses 1 to 5 introduce this standard and provide general information regarding the scope, normative references, defined terms and abbreviations used. Clause 6 then provides an informative choreography that shows how the web service calls defined in Clause 7 can be used. Finally, Clause 8 defines the XML message used in Clause 7. Annexes A and B then provide a list of all allowed value sets used herein and some release notes.

3 Normative References

 3 Normative References
The following normative documents contain provisions, which through reference in this text constitute provisions of this Standard. For dated references, subsequent amendments to, or revisions of, any of these publications do not apply. For undated references, the latest version applies.
  • DDEX Data Dictionary Standard. Latest Version
  • DDEX Party Identifier (DPID) Standard. Latest Version
  • DDEX Electronic Release Notification Message Suet Standard. Latest Version.
  • IETF. RfC 2616. Hypertext Transfer Protocol (HTTP/1.1). 1999
  • IETF. RfC 5023. The Atom Publishing Protocol. October 2007
  • IETF. RfC 6797. HTTP Strict Transport Security (HSTS). November 2012
  • W3C: XML Schema Part 1: Structures. Second Edition. 2004
  • W3C: XML Schema Part 2: Data types. Second Edition. 2004

4 Terms and Definitions

 4 Terms and Definitions

Atom

Formally, the Atom Syndication Format, is an XML language used for web service feeds. The Atom Publishing Protocol is a simple HTTP-based protocol for creating and updating web resources.

Product

A Manifestation of a Release (or another Resource) which is made available to Consumers, by sale, loan or other means. The attributes of a Release in its digital manifestation as a Product may be technical (e.g., the codec or bit rate); a mode of distribution (e.g., downloading or streaming); or a commercial term (e.g., price).

Release

A Release is an abstract entity representing a bundle of one or more Resources compiled by an Issuer. The Resources in Releases are normally primarily sound recordings or music audio-visual recordings, but this is not invariably the case. The Release is not itself the item of trade (or “Product”). Products have more extensive attributes than Releases; one Release may be disseminated in many different Products.

Release Distributor

Release Distributor is an organisation, which is duly authorised by a Release Creator to offer Releases manifested in the form of Products to consumers. Release Distributors include Digital Service Providers (DSPs) and Mobile Service Providers (MSPs) as well as other organisations.

Web Service

A modern set of web technologies that allow small pieces of information, typically in the form of XML files, to be exchanged. Augmented with FTP (or other file exchange mechanisms) they can be used to communicate Releases along the music supply chain.

Web Service Call

The sending of an XML document to a port/address on a web server, using HTTP or HTTPS.

Web Service Response

The sending of an XML document in direct response to a Web Service Call, using HTTP or HTTPS.

For the avoidance of doubt: the appropriate response is always the message indicated in the appropriate Choreography.

5 Abbreviations

 5 Abbreviations
AMEPAutomated Message Exchange Protocol
ACAAppointed Certification Agency
AVSAllowed Value Set
BPBusiness Profile
CISAC

Confédération internationale des sociétés d'auteurs et compositeurs, the International Confederation of Societies of Authors and Composers (see cisac.org)

CACertification Agency
CTConformance Tester
DAWDigital Audio Workstation
DDEX

Digital Data Exchange

DSIGDigital Signature
DSPDigital Service Provider (incudes Mobile Service Providers)
DSRDigital Sales Reporting
ERNElectronic Release Notification
FTPFile Transfer Protocol (FTP specifically includes SFTP)
GRidGlobal Release Identifier
HTTPHypertext Transport Protocol  (HTTP specifically includes HTTPS)
HTTPSSecure Hypertext Transport Protocol
IECInternational Electrotechnical Commission (see iec.ch)
ISOInternational Organisation for Standardisation (see iso.org)
MIMEMultipurpose Internet Mail Extensions
MLCMusic Licensing Company
MWLMusical Works Licensing
MWNMusical Works Notification
MRBV

Multi-Record-Block Variant

PCAPrivate Certification Agency
PDFPortable Document Format
RESTREpresentational State Transfer
RINRecording Information Notification
SFTPSecure FTP
SRBV

Single-Record-Block Variant

TISTerritory Information System (a CISAC Standard)
TLSTransport Layer Security
UGCUser-generated content
URLUniform Resource Locator
XMLeXtensible Markup Language
XSDXML Schema Definition
W3CWorld Wide Web Consortium (see w3c.org)
WSWeb Service

6  ERN Web Service Choreography (informative)

 6.1 Introduction
The approach to Release deliveries defined in this standard is asymmetric. Only Release Creators will need to publish and maintain a web service. Release Distributors will be able to call this service and thus effect the (subsequent) deliveries of Releases.

As a consequence it is the Release Distributor that determines the speed of the communication. Whenever the Release Distributor is able or interested in receiving content, it can call the Release Creator’s web service. It is then the responsibility of the Release Creator to deliver the appropriate information (see also Clause 6.3).

A further consequence is that the capacity management lies with the Release Creator who may need to be able to satisfy concurrent (or near concurrent) web service requests from multiple Release Distributors. The number and frequency of such web service requests is likely to vary significantly over time and Release Creators are advised to plan their infrastructure accordingly.

 6.2 Choreography
The choreography enabled by this standard is depicted in the two diagrams below. The actions on both sides (represented by red boxes) are exemplary only and only represent a typical information exchange in accordance with this standard. It comprises of these steps:
  1. A Release Distributor contacts the Release Creator by accessing its web service end point using a secure HTTPS connection, asking for a list of Releases that are available for it;

  2. The Release Creator responds with an Atom feed page containing a series of Atom feed entries.
    Each of these entries provides a brief description of a Release plus a URL that can be used by the Release Distributor to obtain the Release.
    The Release Creator should only make available an Atom feed entry for a Release, once it can deliver the relevant NewReleaseMessage and its Resources.
    The Release Creator should also take care that at no time more than one NewReleaseMessage for the same Release appears in the Atom feed.
    As a consequence, a Release Creator should remove entries from its Atom feed as soon as it needs to update a NewReleaseMessage that is referenced in the Atom feed. The Release Creator should also make sure that the appropriate status code of 404 is returned upon a Release Distributor calling the now outdated GET ERN Message or GET ERN Resource call;

  3. The Release Distributor contacts the Release Creator again by accessing the web service end point using the URL provided in Step 2 using a secure HTTPS connection, to get a NewReleaseMessage using the URL provided in the Atom feed entry;

  4. The Release Creator responds with a NewReleaseMessage in accordance with the relevant standard as agreed between the Release Creator and the Release Distributor;

  5. The Release Distributor ingests the NewReleaseMessage and extracts the URIs for each of the Resources that are part of the Release;

  6. The Release Distributor downloads and ingests each of the Resources from the Release Creator in the case of network based URIs using the GET ERN Resource call. 

  7. Alternatively, if the URIs  specify either a local path managed by the Release Distributor to where the Resources have been pre-delivered or a location on an SFTP server, the Release Distributor obtains and ingests the Resources.

  8. The Release Distributor informs the Release Creator that it has ingested the entire Release (including its Resorces) by using the DELETE ERN Message method with the URL used to obtain the NewReleaseMessage in Step 3.

Figure 1 – Basic ERN Web Service Choreography


Wherever this Clause references the NewReleaseMessage this is to be understood to also include the PurgeReleaseMessage.



Figure 2 – Additional Commands in the ERN Web Service Choreography

 

Specifically not shown in the diagrams is the approach to exception handling.

It is for instance possible that a Release Distributor requests a NewReleaseMessage based on the information received on an earlier GET ERN Feed command – but that that NewReleaseMessage no longer exists at that URL. In that case the response from the Release Creator’s web service could be a “redirect” with a 301 status code which could trigger the Release Distributor to request the Release using the information received in the reply.

 6.3 URLs
The approach to web service-based Release deliveries is based on URLs that are set at the discretion of the Release Creator who publishes the web service to Release Distributors that are its business partners to share its Releases with them.

While it can be assumed that many URLs follow a common syntax (the URL to access a NewReleaseMessage may, for instance, follow the syntax http://base_url.com/ws_endpoint/releaseid=xxxx), there is no compulsion to use this approach. Similarly it is not compulsory that all URLs from one Release Creator follow a common syntax.

All a Release Creator needs to do, is to publish the URL for the GET ERN Feed call to its Release Distributor(s) and make sure that the URLs placed in the responses to web service calls are (i) valid and (ii) active.

Wherever this standard only mentions the HTTP protocol, it is to be understood that HTTPS communications are also allowed.

 6.4 Personalising the Feed
Many Release Creators have different offerings for different sets of their Release Distributor business partners. In these cases it is essential that the web server of a Release Creator knows who is calling one of its web services.

This can be done with or without formal authentication (see Clauses 6.5 and 6.6, respectively). The benefit of using formal authentication is that it also allows the Release Creator and the Release Distributor to secure their communication.

 6.5 Web Service Set-up and Authentication
This standard does not define an authentication mechanism for the web service-based communication between the Release Creator and the Release Distributor. Should the two parties require their communication to be authenticated, they are free to use any of the existing web-based authentication methods including, but not limited to, Basic Auth, HTTP Digest Authentication, HTTPS Client Authentication, OAUTH, etc.
 6.6 Identifying Release Distributors Without Authentication
Establishing the identity of the caller of the web service can also be accomplished without formal authentication (although in that case the Release Creator cannot be certain that the caller is who it claims to be) by including, for instance, the DDEX Party Identifier (DPID) of the web service caller into the URL syntax.
 6.7 Priority communications and take-downs
When using this standard, a Release Creator can only provide information to a Release Distributor upon the Release Distributor calling the Release Creator’s web service.

As a consequence, the Release Creator and the Release Distributor may agree how often the Release Distributor should call the GET ERN Feed call and how quickly it must react to the received information.

Similarly, the Release Creator and the Release Distributor may agree how quickly the Release Creator should react to the information received from the Release Distributor. (This may be implicit, e.g. because the Release Distributor has called a specific web service call, or explicit, e.g. because the Release Distributor has submitted a status code using the POST ERN Status call).

One way of implementing different types – and priorities – of Release deliveries is by using Atom categories (DDEX-defined categories are listed in Clause 8.1) and/or by providing multiple web service end points for the GET ERN Feed call.

7 Web Service Commands

 7.1 GET ERN Feed

7.1.1    Purpose

This command can be used by a Release Distributor to request information about NewReleaseMessages that are ready for that particular Release Distributor to receive from the Release Creator.

The Release Distributor will call, after appropriate authentication if needed, the web service address previously agreed between the Release Creator and the Release Distributor.

7.1.2    Syntax of Reply

The web server shall return one of the following standard HTTP status codes with their standard HTTP response code semantics:

  • 200 (OK);
  • 400 (Bad request);
  • 401 (Unauthorised);
  • 404 (Not found);
  • 503 (Service unavailable due to a temporary overloading or maintenance of the server); and
  • 500 (Internal server error)

Other standard HTTP status codes may be used on bilateral agreement between the Release Creator and the Release Distributor.

The web server shall also return to the calling web service client an Atom Feed document as defined in Clause 8.1.

 

 7.2 GET ERN Message

7.2.1   Purpose

This command can be used by a Release Distributor to request a NewReleaseMessage or PurgeReleaseMessage. The web service address for this call is the URL for the specific instance of the requested NewReleaseMessage.

The Release Distributor would typically have received this URL from an Atom Feed Entry (see also Clauses 7.1 and 7.6).

7.2.2   Syntax of Reply

The web server shall return one of the following standard HTTP status codes with their standard HTTP response code semantics:

  • 200 (OK);
  • 301 (Moved permanently);
  • 400 (Bad request);
  • 401 (Unauthorised);
  • 404 (Not found);
  • 500 (Internal server error); and
  • 503 (Service unavailable due to a temporary overloading or maintenance of the server).

Other standard HTTP status codes may be used on bilateral agreement between the Release Creator and the Release Distributor.

If the status code is 200, the web server shall also return to the calling web service client, an XML file containing a valid NewReleaseMessage or PurgeReleaseMessage as defined in the appropriate Electronic Release Notification Message Suite Standard (including applicable profiles).

If the status code is 301, the web server shall also return to the calling web service client the correct URL.

 

 7.3 DELETE ERN Message

7.3.1 Purpose

This command can be used by a Release Distributor to signal to the Release Creator, that the Release Distributor has ingested, but not necessarily processed, a NewReleaseMessage and all of its Resources.

It therefore signals to the Release Creator, that it no longer needs to expect web service calls with respect to the said Release (including its Resources) from the Release Distributor. After receiving this call a Release Creator is free to remove the associated entry from the Atom feed as well as the associated resources.

7.3.2 Syntax of Reply

The web server shall return one of the following standard HTTP status codes with their standard HTTP response code semantics (unless clarified):

  • 204 (The server successfully processed the request but is not returning any content);
  • 208 (Already reported);
  • 400 (Bad Request) – which in this case means that the URL does not lead to a NewReleaseMessage (any more);
  • 401 (Unauthorised);
  • 404 (Not found);
  • 500 (Internal server error); and
  • 503 (Service unavailable due to a temporary overloading or maintenance of the server).

Other standard HTTP status codes may be used on bilateral agreement between the Release Creator and the Release Distributor.

 

 7.4 GET ERN Resource

7.4.1  Purpose

This command can be used by a Release Distributor to request receipt of a Resource file. The web service address for this call is the URL for the specific instance of the file as indicated in the NewReleaseMessage.

The GET ERN Resource method only applies if the Release Creator has used an HTTP link when communicating the location of the Resources in the NewReleaseMessage. Therefore, if the URI in the NewReleaseMessage uses a scheme other than http:// or https:// (or any protocol agreed by the parties), the Release Distributor may not be able to the use web services to obtain this Resource. This approach allows the combination of the web service delivery approach defined herein for Release metadata with an (S)FTP or local file delivery for Resource files.

7.4.2 Syntax of Reply

The web server shall return one of the following standard HTTP status codes with their standard HTTP response code semantics:

  • 200 (OK);
  • 301 (Moved permanently);
  • 400 (Bad request);
  • 401 (Unauthorised);
  • 404 (Not Found); and
  • 500 (Internal server error); and
  • 503 (Service unavailable due to a temporary overloading or maintenance of the server).

Other standard HTTP status codes may be used on bilateral agreement between the Release Creator and the Release Distributor.

If the status code is 200, the Web server shall also return to the calling web service client the Resource file.

If the status code is 301, the Web server shall also return the updated URL of the resource. If the Resource itself (i.e. not just its URL) has changed, the Release Creator shall add a new NewReleaseMessage into its feed(s) and the Release Distributor shall regard the entire NewReleaseMessage as invalid.

 

 7.5 POST Release Status

7.5.1 Purpose

This command can be used by a Release Distributor to provide information on the processing of a Release which is done by calling the URL indicated in the relevant Release Creator's Atom feed entry relating to the relevant Release.

The document to be posted is defined as in Clause 8.2.

7.5.2 Syntax of Reply

The web server shall return one of the following standard HTTP status codes with their standard HTTP response code semantics (unless where clarified):

  • 204 (The server successfully processed the request but is not returning any content);
  • 400 (Bad request);
  • 401 (Unauthorised);
  • 404 (Not found);
  • 500 (Internal server error); and
  • 503 (Service unavailable due to a temporary overloading or maintenance of the server).

Other standard HTTP status codes may be used on bilateral agreement between the Release Creator and the Release Distributor.

 

 7.6 GET ERN URL

7.6.1 Purpose

This command can be used by a Release Distributor to request an Atom Feed Entry for a Release for which the Release Distributor only knows a Release ID. The Release Distributor will call, after appropriate authentication if needed, the web service address previously agreed between the Release Creator and the Release Distributor (see Clause 6.4).

The syntax for this call is not prescribed. Below are two possible approaches (with base.url being the URL agreed between the Release Creator and the Release Distributor and xxx being the Release ID (typically a UPC) for which a URL is sought).

Example 1
base.url/?releaseid=xxx
Example 2
base.url/releaseid/xxx

7.6.2 Syntax of Reply

The web server shall return one of the following standard HTTP response codes with their standard HTTP response code semantics (unless where clarified):

  • 200 (OK);
  • 202 (Accepted) – which in this case means that the Release Creator will add the requested Release information into the feed once the information has been created;
  • 400 (Bad request);
  • 401 (Unauthorised);
  • 404 (Not found);
  • 500 (Internal server error); and
  • 503 (Service unavailable due to a temporary overloading or maintenance of the server).

Other standard HTTP status codes may be used on bilateral agreement between the Release Creator and the Release Distributor.

If the status code is 200, the Web server shall also return an Atom Feed Entry in accordance with Clause 8.1.

If the status code is 202, the Web server shall add the relevant Atom Feed Entry into the queue of Atom Feed Entries that the relevant Release Distributor can obtain by using the GET ERNList method.

8 Syntax of XML Messages

 8.1 Introduction

The hierarchical structure of the messages is provided in tabular form in a separate document attached to this standard.

Specific business processes between a Release Creator and a Release Distributor may require even further conformance rules. These are, however, not part of the Standard and will need to be agreed between by the parties. Rules relating to the authority of business partners to unilaterally change the Message This sentence is not complete

 8.2 Atom Feed

8.2.1 Introduction

The Atom feeds shall follow the syntax of an Atom feed as defined in the relevant XML Schema Definition file.

The following Atom feed entry elements map to the following elements in the relevant NewReleaseMesssage or PurgeReleaseMessage (depending on which one is to be made available):

Atom Feed entry ElementNewReleaseMessage ElementPurgeReleaseMessage Element
//entry/title

Either //ReleaseList/Release/DisplayTitleText or //ReleaseList/Release/DisplayTitle/TitleText. Which element is chosen depends on whether the Release Creator wishes to communicate subtitle information in the Atom feed.

//PurgedRelease/Title/TitleText


//entry/ReleaseType/ReleaseList/Release/ReleaseTypeomitted
//entry/ReleaseId/*//ReleaseList/Release/ReleaseId*//PurgedRelease/ReleaseId*
//entry/DisplayArtistName//ReleaseList/Release/DisplayArtistName/Name//PurgedRelease/Contributor/PartyName for a party that the sender of the ATOM feed considers helpful for the recipient for taking down the relevant Release.

Entries should be entered into the feed sequentially, that is the oldest entry should be at the top and the latest addition should be at the bottom. A Release Creator may, however, add and remove entries at its own discretion.

The number of entry composites in the Atom feed is not prescribed by this standard. The Release Creator publishing the web service may, however, want to set a maximum number in collaboration with its Release Distributor business partner(s).

The following link types can be communicated in the //feed/link/@rel attributes:

  • self – to provide the URL of the feed itself;
  • next – to provide the URL to the next feed "page" (in the case where there is more than one feed pages available to the Release Distributor); and
  • previous – to provide the URL to the previous feed "page" (in the case where there is more than one feed pages available to the Release Distributor).

The following link types can be communicated in the //feed/entry/link/@rel attributes:

  • alternate – to provide the URL for the GET ERN Message method defined in Clause 7.2;
  • delete – to provide the URL for the DELETE ERN Message method defined in Clause 7.3; and
  • status – to provide the URL for the POST Release Status method defined in Clause 7.5.

8.2.4 Feed Entry Categories

The following categories may be communicated in the //entry/category element:

  • MetadataUpdate – to indicate that the relevant Release is being updated and that such an update relates to the information describing the Releases and/or Resources that make up the Release;
  • ResourceUpdate – to indicate that at least one Resource of the relevant Release is being updated;
  • DealUpdate – to indicate that at least one Deal for the relevant Release is being updated; and
  • Takedown – to indicate that the relevant Release is being globally taken down.

It is permitted to communicate multiple categories, each in one category element.

 8.3 Release Status Message
The ErnAcknowledgementStatusMessage shall follow the syntax as defined in the relevant XML Schema Definition file.

 

 

Annex A (normative) Allowed Value Sets

 Annex A (normative) Allowed-value Sets
All allowed value sets with their allowed values and definitions that are valid within this standard are set out below. The allowed-value sets are maintained outside of this Standard and DDEX may add to the list below.

The Table of AVSs is provided in a separate document. See the blue box here.

Annex B (informative) Release Notes

 Annex B (informative) Release Notes

B.1 Changes between Version 1.7 and 1.6

Version 1.7 harmonises the communication of acknowledgements and Release status between the SFTP and web service choreographies.

(Version 1.6 was the first version of this standard.)

 

 

Evaluation Licence for DDEX Standards

 

 

Subject to your compliance with the terms and conditions of this Agreement, DDEX™ grants you a limited, nonexclusive, non-transferable, non-sublicenseable, royalty-free licence solely to reproduce, distribute within your organisation, and use the DDEX standard specifications (“DDEX Standards”) solely for the purpose of your internal evaluation. You may not make any commercial use of the DDEX Standards under this agreement. No other licences are granted under this agreement.

No representations or warranties (either express or implied) are made or offered by DDEX with regard to the DDEX Standards. In particular, but without limitation, no representations or warranties are made in relation to:

  1. The suitability or fitness of the standards for any particular purpose;
  2. The merchantability of the standards;
  3. The accuracy, completeness, relevance or validity of the standards; or
  4. The non-infringement of any third party intellectual property rights related to the DDEX Standards.

Accordingly, DDEX and/or its members shall not be liable for any direct, indirect, special, consequential or punitive loss or damages howsoever arising out of or in connection with the use of the standards. IN THE EVENT THAT ANY COURT OF COMPETENT JURISDICTION RENDERS JUDGEMENT AGAINST DDEX AND/OR ITS MEMBERS NOTWITHSTANDING THE ABOVE LIMITATION, THE AGGREGATE LIABILITY TO YOU IN CONNECTION WITH THIS AGREEMENT SHALL IN NO EVENT EXCEED THE AMOUNT OF ONE HUNDRED U.S. DOLLARS (US$ 100.00).

Users of the DDEX Standards are cautioned that it is subject to revision. Users are recommended to use the latest versions, which are available at http://www.ddex.net. The use of outdated versions of the standards is not recommended but may be required by agreement between implementers in particular cases.

  • No labels