File Service

The File Service is a web service that can be used to store and retrieve files.

Basic concepts

Immutability

Files stored by the File Service are immutable. You may DELETE a file, but a file can never change. To update a file you will need to upload a new version and refer to that one instead.

Parameters

How files are stored is controlled by three parameters: space, owner and path.

Space

The space parameter determines the kind of storage being used.

The tmp space is intended for files with a limited life-span, typically no more than a few days.

The content space is intended for files that should be stored permanently.

Owner

The owner parameter is used to group files.

In the tmp space the owner would typically be a user ID.

In the content space the owner would be the main alias of the content it belongs to (the implicit contentid/ namespace being skipped).

May contain any character that is a allowed in a URI path segment. May not contain a / character.

Path

Uniquely identifies the file within a space and host. May contain any character that is a allowed in a URI path, including /.

Storage

The file data is stored in Amazon S3, or a compatible web service such as minio.

See Service Configuration for example configuration.

N.b. The S3 bucket should be configured to expire objects with a key that begins with tmp after a few days.

Web Service API

File operations

These are the file operations supported by the File Service.

By default the file service is deployed at http://ace-file-service:8082/ inside Docker, and this is used in the examples below.

When a file is uploaded, a URI is generated to identify the file. This URI should be used when fetching the file from the file service. Note that while scheme, host and path are normally used to generate the URI, you must use the returned URI as it may rewrite some or all of the parameters.

Upload file

To upload a file, make a POST request to a sub path of the service containing the desired Scheme, optionally specifying host and path. The relative URL format is /file/\$scheme\\[/$host\[/$path\]\. If you do not specify host and path, they will be generated for you.

$ curl -i -s -X POST \
    -H "Content-Type: text/plain" \
    -H "Accept: application/json" \
    -H "X-Auth-Token: $TOKEN" \
    --data-binary @lorem.txt \
    "http://ace-file-service:8082/file/tmp/sysadmin/lorem"

HTTP/1.1 201 Created
Location: http://ace-file-service:8082/file/tmp/sysadmin/90a45ff9-79f3-47b9-87cd-6b8399169957
X-Original-Path: lorem
Content-Type: application/json

{
   "URI" : "tmp://sysadmin/lorem-1",
   "length" : "395",
   "accessTime" : "-1",
   "creationTime" : "1394106900000",
   "mimeType" : "text/plain",
   "modifiedTime" : "1394106900000",
   "originalPath" : "lorem"
}

Note the Location-header, the path is generated causing the actual path to the file to be different than the path in the request.

The file URI can not be used directly with the Web Service API, clients should either use the Location header, or build the path using the scheme, host and path components of the URI.

Download a file

To download a file, make a GET request to the Location returned from the previous POST.

$ curl -i \
    -H "X-Auth-Token: $TOKEN" \
    "http://ace-file-service:8082/file/tmp/sysadmin/90a45ff9-79f3-47b9-87cd-6b8399169957"

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
X-Original-Path: lorem
Cache-Control: private, no-transform, max-age=31536000
Content-Type: text/plain
Content-Length: 395

Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Mauris dignissim interdum sem a sollicitudin. Mauris orci nibh,
suscipit a justo nec, facilisis blandit arcu. Suspendisse potenti.

NOTE: The value of the Content-Type HTTP header returned when retrieving a file from the File Service will be based on the actual file data as determined by Apache Tika.

Delete a file

To remove a file from the File Service, issue a DELETE request to the Location returned from the previous POST.

$ curl -i -X DELETE \
    -H "X-Auth-Token: $TOKEN" \
    "http://ace-file-service:8082/file/tmp/sysadmin/90a45ff9-79f3-47b9-87cd-6b8399169957"

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Length: 0
Date: Thu, 06 Mar 2014 12:00:50 GMT

A subsequent GET requests for this resource will no longer succeed.

File info

The file info returned when you uploaded the file can be retrieved again by making a GET request to /info/scheme/host/path

$ curl -s \
    -H "Accept: application/json" \
    -H "X-Auth-Token: $TOKEN" \
    "http://ace-file-service:8082/file/info/tmp/sysadmin/90a45ff9-79f3-47b9-87cd-6b8399169957"

{
    "URI" : "tmp://sysadmin/90a45ff9-79f3-47b9-87cd-6b8399169957",
    "length" : "395",
    "accessTime" : "-1",
    "creationTime" : "1394107854000",
    "mimeType" : "text/plain",
    "modifiedTime" : "1394107854000",
    "originalPath" : "lorem"
}

Authentication

You must be authenticated to use the File Service.

File Delivery Service

The File Delivery Service provides a readonly way to access files from the File Service, with an added content awareness.

By default the file delivery service is deployed at http://ace-file-delivery-service:8088/ inside Docker, and this is used in the examples below.

The file service is designed without an internal cache, so a HTTP cache like Varnish is recommended. See Notes on reverse proxy for image delivery for a configuration example.

Retrieving files

There are two ways to retrieve a file from a content from the file delivery service:

http://ace-file-delivery-service:8088/version/c:MTY4M2ZlNDktMjNkYy00:MTBmMTc5/path/to/file.txt

http://ace-file-delivery-service:8088/alias/namespace/myAlias/path/to/file.txt

Content types

The File Delivery Service uses Apache Tika to determine the content type / mime type of a file being delivered. Both the file data and the file extension of the file being retrieved will be used. The determined mime type will be returned as part of the Content-Type HTTP header of the file response.

Variants

The file delivery service looks at the file aspect of the given content, containing a map from paths within the content to paths in the file service. This enables projects to override that mapping, e.g. for removing the mappings of some files that should not be made available through the File Delivery Service.

The variant config has the alias of aceFileDelivery.variant, and the default mapping behaviour is that the main image file (denoted by the aceImage aspect) is not available, but all other files are. There is of course a possibility to adjust the mappings to other behaviours.