Concepts

This chapter explains the core concepts behind ACE.

Datamodel

The core object managed by ACE is the Content.

Content and Aspects

The purpose of the typing is primarily so that systems that consume the content can assign semantics to the objects, but types are also used for some features in ACE itself. Aside from assigning semantics, a type may also define the types of the fields in an object. The type system allows for essentially arbitrary JSON as well,by supporting a special type for untyped JSON.

Data in Content is organized into named Aspects. An aspect with the same name is intended to have the same meaning in every Content it is used, so for instance the aceCategorization aspect is used to represent user-supplied categorization for all types of content. Most aspects will represent some kind of cross-cutting concern like that.

Each content also has a Content Type, which is a type with some special properties. A content's type can't be changed after creation. A Content Type can mark some aspects as mandatory, so that content can't be created without including those aspects but it cannot restrict which aspects can be added to the content.

Aliases

To identify content, ACE uses aliases. An alias consist of two parts: a namespace and a name. The namespace is used to separate aliases used for different purposes, while the meaning of the name depends on the specific namespace used.

In Content, aliases are represented as a string with a type named "alias".

Main alias

A content may have any number of aliases, but always has at least one alias, namely its main alias. The main alias always belongs to the contentid namespace.

Aliases are typically user defined, but if no main alias is supplied upon content creation, a universally unique main alias is automatically generated and assigned to the content.

Versions

Content in ACE is versioned. This means that when content is created or updated, a new version is created. Once created, a version can never change aside from being deleted. Every version of a content has a unique ID that can be used to reference it. Just like content, aspects are also versioned (except "Content Metadata").

In Content, versions are represented as a string with a type named "contentVersionId".

Revisions

Every content has a revision, in order to keep track of content changes. Whenever anything in the content is changed, a new revision is created. However, unlike versions, the revisions are not persistent, meaning that only the latest revision exists at any given time.

Unversioned Aspects aka Content Metadata

There are "unversioned Aspects" that belong to the content itself rather than a specific version of a content. These Aspects are called "Content Metadata Aspects".

Content Operations

The core content operations supported by ACE are Create, Read, Update, and Delete. See also workspaces, described later, for a variation on creating and updating.

Creating

Creating simply creates a content with one version and an initial revision. It involves permissions and the write pipeline, described later.

Read

Reading is immediately more involved: since a content is versioned, you have to decide what version you want. You can fetch a version if you know the version ID, or you can resolve an alias to a version.

Reading also involves permissions, views and variants, described later.

Update

Updating a content usually involves creating a new version, but sometimes only certain metadata is updated. Whether a new version is created or not, the revision of the content changes.

When updating a content, it is possible that someone else has also updated the same content, which introduces a potential for conflicts. This is handled with optimistic locking: when you update a content, you include an ETag which is used to determine if you had the latest changes already. If you didn't, you get a conflict response and have to resolve the conflict before resubmitting with a new ETag. Internally this is based on the revision, but as a user you should only worry about the ETag.

Like creating, updating involves permissions and the write pipeline. Revisions are used to deal with eventual consistency.

Delete

This is essentially just an update, only instead of creating a new version it removes all versions if successful.

Eventual Consistency

ACE is an eventually consistent system: different parts of the system may see different versions of content at any given point in time, but will converge to the same version over time. This mostly shows up in two ways: searching and caching.

Since the index that is searched is updated asynchronously, the content in the index may be out of date. And since the Content API has a cache, the content you get from it may be out of date.

To get around some of the issues this can lead to, it is possible to ask the Content API to make sure the content you get is at least as up to date as you are. This is done by telling the system to use a revision of the content at least as recent as yours. There is no similar system for the search service.

Variants and the Read Pipeline

To make it easy to reuse content in different contexts, ACE uses Variants. A variant is a way to request content in a form suited for a specific consumer of content, rather than forcing the consumer to understand all kinds of content.

In essence a variant is a set of rules for mapping different types of content to the variant's representation of content.

Any Content read with the Content API is pushed through a data processing pipeline. The pipeline consists of a series of stages in which the stored Content is gradually transformed into the Variant requested by the client.

Mapping Aspects

The first stage is the aspect mapping stage, where each aspect that has a mapper defined in the variant gets processed by that mapper. Aspects that were not discarded by their mapper are then reassembled into a new data shuttle that is sent to the composer stage.

An aspect mapper has access only to the aspect it is mapping, and nothing else. This is to ensure the output can be cached.

Composing content

The second and final stage is the content composer stage. A content composer gets a data shuttle as its input, and can remove, modify or add any aspects it likes. It also has access to a content service, so it can even read other content and include that in the response.

Since a composer can access things other than the content, ACE doesn't try to predict how cachable its output is. Instead the default is to never cache, but there is a mechanism for the composer to set custom cache headers.

Write Pipeline and Lifecycle Hooks

The purpose of the write pipeline is to allow content to be transformed and validated before being updating the database. This can be used for things like adding custom aspects, implementing a custom permission system, or setting default values that the client didn't set.

The write pipeline has two stages: pre-store callbacks can modify the data to be stored, and if there is a conflict additional on-conflict callbacks can attempt to resolve the conflict.

In the special write case when a content is deleted, pre-delete callbacks may prevent the delete. The other kinds of callbacks are not involved in deletes.

All these hooks can abort the operation by throwing an exception, which may optionally set the status that should be reported to the client.

Modules

ACE has a module system that can be used to organize read and write pipelines. The content service can be configured with an ordered list of modules that add functionality.

There are two ways to map callbacks (mappers, composers and lifecycle hooks): globally, or in a module. When the system looks for a callback, it first looks in the global configuration. If a matching variant or lifecycle hook is found, that configuration is used and modules are ignored. If the global configuration doesn't have the variant or a lifecycle hook, we scan the modules in order until we find a callback to apply.

Indexing and Search

To support searching among content, content can be indexed and then searched using Solr.

Indexing

Indexing is done by the Solr Indexer component, which listens for changes to content and indexes anything that is changed.

Each instance of the Solr Indexer is responsible for one Solr collection (index). Each collection has a corresponding variant (which may be shared by multiple collections), and may also be restricted to only include content that is published on certain views. By default, all versions of a content are indexed in Solr regardless of view.

The variant for a collection maps the content to be indexed to a Solr JSON document, and that is then supplied to Solr for indexing. If content is not available in the collection's variant, that content is not indexed.

Searching

Searching for content can be done either directly in Solr, or using the ACE Search Service. The Search Service is a wrapper around Solr that uses Solr query syntax, but adds a few features:

• Automatic filtering by permissions
• Filtering by view (optional)
• Filter on availability in variant
• Inlining content in results

Filtering by permissions can be disabled for specific views, so that the search service can be used without authentication for content meant to be public.

Views

Views are a way of naming versions of content. A version of a content can be assigned to, or removed from, a view, so that when the content is requested on that view it resolves to that version (or no version, if no version assigned).

Views are used to make different versions of content available for different users. The simplest example is when you want one version to be live on a website, while a new version is undergoing editorial review. The live version would be on the view used by the website, while the version undergoing editorial review is on another view only visible to editorial staff.

The system keeps track of a history for each view and each content, so you can see which version was on the view at a given point.

Symbolic views

A symbolic view associates a view A with another view B, rather than a specific version, so that the version on A resolves to whatever the version on view B currently is.

Time state

The presence of a content on a view can also be controlled by time state, i.e. an on time (before which the content is not available on the view) and/or an off time (after which the content is not available on the view).

Workspaces

Normally when you save content in the Content API, a new version is created that is added to the version history, indexed and so on. Sometimes you just need to make content available on the server but not store it permanently, for example to autosave work in progress, or preview it in a different application. The way to do this is to save it to a workspace, which is an ephemeral content store with no persistence guarantees and no history.

Workspaces are automatically created when content is saved to them. When you read content from a workspace, it transparently supplies either the version stored in the workspace, or the latest version of the content if it is not stored in the workspace.

In addition to storing content in a workspace, you can also assign an alias to a content in a workspace. Such an alias can only be resolved on a workspace. However the alias is reserved so it can't be assigned to any other content until the content is removed from the workspace or the alias is released.

Files

The system includes a file service, for dealing with large binary data. The file service comprises two sub-services: the file service, and the file delivery service. The file service simply stores files, and is completely independent of content. The file delivery service lets you access files that are associated with content.

File Service

The file service lets you upload files, and returns an identifier that lets you retrieve or delete the file. Files can never be modified, instead you upload a new file. The file service is typically only used by other ACE services to manage files.

Files on content

Files can be associated with content using the aceFiles aspect, which associates content-specific paths with file identifiers.

File Delivery Service

The file delivery service serves files referenced from an aceFiles aspect. As a special case, files that are also registered as images are not accessible to avoid the original of an edited image being available.

Images

The system includes an image service that supports image operations like cropping, scaling, rotation and pixellation.

A content can represented an image by including an aceImage aspect that selects a file in the aceFiles aspect as being the image file. Edits to the image are held in the aceImageEdits aspect.

The image service looks at these aspects, selects an appropriate crop, and delivers an image scaled to the size requested by the user.

Atex Metadata

The Atex Metadata system is a way to categorize content in such a way that it can be efficiently searched and cross-referenced. The system allows content to be categorized along many different axes of interest.

Taxonomies

The Atex Metadata system centers on entities, which are what content is categorized with. Entities may have other entities as children (e.g. "sport" may have "bowling" as a child), and belong to dimensions which group related entities (e.g. IPTC subject, person and author could all be different dimensions). Dimensions are grouped into taxonomies, which exist so that e.g. two different publications may use different subsets of the available dimensions.

Taxonomy Service

The Taxonomy Service allows access to the taxonomies, dimensions and entities, by either listing them, or using autocompletion in a specific dimension. The service does not support creating new entities, that must be done using content.

Categorization

Content is categorized by using an aspect called aceCategorization. It contains a list of the taxonomies the content is categorized in, and the entities the content is categorized with.

Permissions

In ACE, permissions are granted to users based on the type of the content, the security context(s) the content belongs to, and the roles the user has in those contexts. Types have already been covered earlier, but security context and role are specific to the permission system.

Permissions

Permissions are simply things you're allowed to do to a content, like create, update, delete, assign to view, etc.

Security Context

Content belongs to one or more security contexts. Security contexts are assigned when content is created and can be changed later. They are essentially just names that are used to determine which roles a user has in relation to content belonging to them. Security contexts have no hierarchy or other structure to them.

Roles

A role grants permissions on types. It basically lists for each type what permissions it grants on those types. They are represented as content of type aceRole.

User roles

For each user there is a content with a aceUserRoles aspect, that maps security contexts to the roles that user has in those contexts.

Naming conventions

All names used by the system, or provided by the system for your use, are prefixed with 'ace'. Names starting with 'ace' are reserved in most contexts except where explicitly stated not to be.

We recommend that you adopt a similar naming convention, but with a different prefix, for your project, to reduce problems when integrating with third party projects.

Aliases

Alias namespaces starting with an underscore ("_") - such as _config, _defaults, _role and _type - are used for ACE core functionality, and should only be used for content meant for these namespaces (such as a configuration content in the _config alias namespace).

Configuration content provided by ACE will use 'ace' as prefix in the alias part. Example: _config/aceCallbacksConfig.

Types

Types included in the system are prefixed with 'ace', followed by a name in Pascal case , e.g. 'aceImageFormats'. Because types are often used as field names (when used as aspects), type names are valid Javascript identifiers.

Variants

System variants are prefixed with 'ace', e.g. the name of the variant used for indexing is 'aceIndexing'. Because variants are sometimes used as field names, they are valid Javascript identifiers.

Configuration content

Configuration content names are not usually used as field names, so in the interest of readability we use '.' for extensions.

Configuration for variants is stored in content named for the variant with an extension '.variant', e.g. the indexing variant is configured by 'aceIndexing.variant'.

Global configuration where there is usually only one instance is stored in a content with the same name as the primary aspect in the config, e.g. 'aceCallbacksConfig' for the content the system reads the 'aceCallbacksConfig' content from.

Callbacks intended for a specific variant is named for that variant, followed by a '.' (period), followed by a descriptive name starting with lowercase. E.g. the mapper for the aceCategorization aspect in the aceIndexing variant is called 'aceIndexing.categorization'. This does not mean such callbacks can't be used for a custom variant, it's just a hint that they are specialized for a certain kind of variant.

Generic callbacks are prefixed with 'ace.' followed by a descriptive name in lowercase.

Views

System views are prefixed with 'ace'. Because views are sometimes used as field names (e.g. in time state), they are valid Javascript identifiers.

Permissions

System permissions always have names starting with capital letters.