Service Configuration
The ACE Services are configured by service .yml files. Each service has got two configuration files — one with the defaults, and one with project overrides.
To override a certain property, create a .yml file and map it to /opt/ace/xxx-service.yml in
your docker container (for example by adding a configs entry to your docker-compose.yml).
services:
...
content-service:
configs:
- source: content-service
target: /opt/ace/content-service.yml
...
configs:
content-service:
file: <path-to-config>/content-service.yml
...
Connections to other ACE services
There is communication between services in ace, and it is possible to customise some settings for this, to better suit the deployment size and use case.
The following are the services used from other services in ACE:
- Content Service; Used from most other ACE serivices.
- File Service; Used from the File Delivery Service, Content Service for content Import and the Image Service.
- Search Service; Used from the Content Service in Callbacks.
Common settings for these connections:
someService:
# uri is the URI to the file service
uri: http://ace-file-service:8082
# maxConnections is how many connections are allowed to the backend service
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
Normaly the defaults should work fine, but if you i.e have many content services, you can increase the number of connections to it from other services.
Authentication secret key
Most ACE services require authentication, implemented by the use of
JSON Web Tokens.
In order to generate and validate such tokens, the authentication properties issuer and key must be configured.
These are supplied in a token.properties file:
# token.properties
ace.token.issuer=<your organization here>
ace.token.key=<secret key>
It is strongly recommended that the secret key is a full-entropy key with a minimal length of 32 bytes.
There are a number of ways to generate the token key, like using a password generator. Alternatively you can use OpenSSL:
openssl rand -hex 32
The token properties file is read from /opt/ace/token.properties. To keep it safe, we recommend supplying it as a
Docker secret.
services:
...
content-service:
secrets:
- source: token
target: /opt/ace/token.properties
...
secrets:
token:
file: <path-to-token>/token.properties
...
Content Service
File: /opt/ace/content-service.yml
# contentService contains configuration for the content service itself.
contentService:
# noAuthViews is a list of views that can be accessed without
# authenticating. Any content version on one of these views is available to
# anyone who can connect to the service.
noAuthViews:
- acePublic
# allowedViews can restrict the service to only resolve and serve versions
# that are on this list of views, even for authenticated users. The empty
# list allows all views. Other services may expect this not to be set.
allowedViews: []
# configContent names the configuration content used for the service. This
# defines all the variants and callbacks and so on.
configContent: aceCallbacksConfig
# kafka contains settings for event listener that keeps the in-memory content
# cache up to date.
kafka:
# brokers contains a list of the Kafka brokers to connect to.
brokers:
- ace-kafka:9092
# The name of the internal changelist topic. This must match what is set in
# Kafka Connect.
topic: aceEvents
# groupId is the Kafka consumer group name for this application's use of the
# changelist. Sharing with other content service instances is fine because
# we don't care about what happened while we were down.
groupId: ace-cache
# batchSize controls how many messages are processed in a batch.
batchSize: 5000
# bufferSize is the size of the network buffer used to read from Kafka.
bufferSize: 64K
# fetchSize is the maximum number of bytes to read in one Kafka request.
fetchSize: 256K
# reconnectDelay is how long to wait before reconnecting to Kafka after a
# failure.
reconnectDelay: 1s
# partition is the Kafka partition number used by this indexer. Currently
# only partition 0 is supported.
partition: 0
# soTimeout sets the SO_TIMEOUT for the connection to Kafka.
soTimeout: 100s
reindexFeeder:
# The list of Kafka brokers to use for publishing the re-indexing events.
brokers:
- ace-kafka:9092
# The name of the topic to publish the re-indexing events to.
topic: aceReindexEvents
# The number of times to retry sending each batch of Kafka records in case of failures.
maxRetries: 5
# How long to wait before sending a batch of Kafka records to the topic. A value
# larger than 0 allows for greater batching.
lingerTime: 1ms
# auth configures the authentication subsystem.
auth:
# Location of the file containing the secret key for tokens. Don't change
# this setting, use Docker secrets instead.
keyFile: file:/opt/ace/token.properties
# login configures the login system, which checks user's passwords against
# LDAP before authenticating them.
login:
# The session timeout for the authentication token issued by ACE
sessionDuration: 12 hours
# Whether LDAP is enabled or not. Regardless, password file functionality
# is still in play in parallel to LDAP.
ldapEnabled: false
# Configures the connection to the LDAP server.
ldap:
# userObjectClass is the name of the LDAP object class that represents
# users.
userObjectClass: inetOrgPerson
# loginNameAttribute is the LDAP attribute where a user's login name is
# stored.
loginNameAttribute: uid
# userSearchBaseDN is used as a base for all user searches
userSearchBaseDN:
# an LDAP query that will be applied as a filter to user authentication
# and listing operations
userFilter:
# Provider URL for the LDAP server
providerUrl: ldaps://ace-ldap:636
# Location of the file containing the LDAP credentials. Don't change
# this setting, use Docker secrets instead.
credentialsFile: file:/opt/ace/ldap-credentials.json
# couchbase configures how the content service uses Couchbase.
couchbase:
# Location of the file containing the Couchbase credentials. Don't change
# this setting, use Docker secrets instead.
credentialsFile: file:/opt/ace/couch-credentials.json
# waitToPersist controls whether the server waits for Couchbase to persist
# before responding to the client. Turning this off improves performance but
# risks losing acknowledged writes when a node crashes. This should not be
# turned off in production, but in development mode it is disabled.
waitToPersist: ${((ACE_DEVELOPER_MODE!"false")?boolean)?string("false", "true")}
# opTimeout is the timeout for content operations. A content operation often
# involves multiple operations against Couchbase, this is the timeout for
# all those operations.
opTimeout: 10 seconds
# metricsFrequency controls how often Couchbase events are recorded to the
# Prometheus endpoint.
metricsFrequency: 10 seconds
# The following settings are standard Couchbase client settings, see
# http://docs.couchbase.com/sdk-api/couchbase-java-client-2.3.6/
# connectionString lists the Couchbase nodes to connect to on startup. This
# doesn't need to be exhaustive because Couchbase tells us which nodes are
# in the cluster, but at least one of these nodes must be available on
# startup.
connectionString: couchbase://ace-couch
#kvTimeout:
#connectTimeout:
#analyticsTimeout:
#managementTimeout:
#queryTimeout:
#socketConnectTimeout:
#searchTimeout:
#viewTimeout:
#maxRequestLifetime:
#kvConnections:
#queryConnections:
#searchConnections:
#viewConnections:
fileService:
# uri is the URI to the file service
uri: http://ace-file-service:8082
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
searchService:
# uri is the URI to the search service
uri: http://ace-search-service:8086
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 10
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# Dropwizard server configuration, see
# http://www.dropwizard.io/1.2.0/docs/manual/configuration.html#servers
server:
applicationConnectors:
- type: http
port: 8081
adminConnectors:
- type: http
port: 9081
gzip:
enabled: false
# Dropwizard logging configuration, see
# http://www.dropwizard.io/1.2.0/docs/manual/configuration.html#logging
logging:
level: INFO
# Swagger nets us alot of bogus warnings
loggers:
org.reflections: ERROR
appenders:
- type: console
threshold: TRACE
# Hystrix configuration for downstream services.
hystrix:
# This property indicates whether the Hystrix Commands should have a timeout.
timeoutEnabled: true
# Determines whether a circuit breaker will be used to track health and to
# short-circuit requests if it trips.
circuitBreakerEnabled: true
# Sets the maximum number of concurrent Commands to allow before rejecting.
maxConcurrentRequests: 1337
# Commands that are not finished before this timeout will be rejected.
executionTimeout: 30 seconds
# Configuration of the in-memory content caches
cache:
# How often the cache will attempt to evict stale content
evictionInterval: 250 millisecond
# Maximum number of aliases to keep in cache
aliasCacheMaxSize: 64000
# Maximum number of content info to keep in cache
contentInfoCacheMaxSize: 32000
# Maximum number of content versions to keep in cache
versionCacheMaxSize: 64000
File Delivery Service
File: /opt/ace/file-delivery-service.yml
# timeout is how long a request is allowed to take before it aborts.
timeout: 30 seconds
# fileService configures the connection to the file service
fileService:
# uri is the URI to the file service
uri: http://ace-file-service:8082
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# contentService configures the connection to the content service
contentService:
# uri is the URI to the content service
uri: http://ace-content-service:8081
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# Dropwizard server configuration, see
# http://www.dropwizard.io/1.2.0/docs/manual/configuration.html#servers
server:
applicationConnectors:
- type: http
port: 8088
adminConnectors:
- type: http
port: 9088
# Dropwizard server configuration, see
# http://www.dropwizard.io/1.2.0/docs/manual/configuration.html#logging
logging:
level: INFO
# Swagger nets us alot of bogus warnings
loggers:
org.reflections: ERROR
# Hystrix configuration for downstream services.
hystrix:
# This property indicates whether the Hystrix Commands should have a timeout.
timeoutEnabled: true
# Determines whether a circuit breaker will be used to track health and to
# short-circuit requests if it trips.
circuitBreakerEnabled: true
# Sets the maximum number of concurrent Commands to allow before rejecting.
maxConcurrentRequests: 1337
# Commands that are not finished before this timeout will be rejected.
executionTimeout: 30 seconds
File: /opt/ace/file-service.yml
File Service
# S3 configuration for Amazon S3 compatible object storage. This is where files are stored.
s3:
bucket: ${BUCKET}
# region to store files in.
region: ${REGION}
# serviceEndpoint lets you use a non-AWS S3 implementation.
serviceEndpoint: ${S3_ENDPOINT}
# Configures the Amazon S3 client, see
# http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/ClientConfiguration.html
client:
# Timeout when establishing a connection.
connectionTimeout: 10s
# Maximum number of connections to open.
maxConnections: 50
# socketTimeout: 50s
connectionMaxIdleTime: 60s
# Connections are automatically closed after this amount of time. If not set connections are
# only closed if left idle.
connectionTTL: 60s
# Timeout for individual HTTP requests to S3. If not set there is no timeout.
# requestTimeout: 60s
# Timeout for a complete request to S3. If not set there is no timeout.
# clientExecutionTimeout: 60s
# Whether to use gzip compression for data transport.
gzip: false
# How many times to retry a request on a retryable error from S3.
maxErrorRetry: 0
# auth configures the authentication subsystem.
auth:
# Location of the file containing the secret key for tokens. Don't change
# this setting, use Docker secrets instead.
keyFile: file:/opt/ace/token.properties
# Dropwizard server configuration, see
# http://www.dropwizard.io/1.2.0/docs/manual/configuration.html#servers
server:
applicationConnectors:
- type: http
port: 8082
adminConnectors:
- type: http
port: 9082
# Dropwizard logging configuration, see
# http://www.dropwizard.io/1.2.0/docs/manual/configuration.html#logging
logging:
level: INFO
# Swagger nets us alot of bogus warnings
loggers:
org.reflections: ERROR
# Hystrix configuration for downstream services.
hystrix:
# This property indicates whether the Hystrix Commands should have a timeout.
timeoutEnabled: true
# Determines whether a circuit breaker will be used to track health and to
# short-circuit requests if it trips.
circuitBreakerEnabled: true
# Sets the maximum number of concurrent Commands to allow before rejecting.
maxConcurrentRequests: 1337
# Commands that are not finished before this timeout will be rejected.
executionTimeout: 30 seconds
The file service needs the access and secret key for s3 mounted as volumes or as docker secrets.
/run/secrets/access_key
/run/secrets/secret_key
Image Service
File: /opt/ace/image-service.yml
# fileService configures the connection to the file service
fileService:
# uri is the URI to the file service
uri: http://ace-file-service:8082
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# auth configures the authentication subsystem.
auth:
# Location of the file containing the secret key for tokens. Don't change
# this setting, use Docker secrets instead.
keyFile: file:/opt/ace/token.properties
# contentService configures the connection to the content service
contentService:
# uri is the URI to the content service
uri: http://ace-content-service:8081
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# view is the view to get the image content from
view: acePublic
# timeout in seconds for reading and processing images.
timeout: 30s
# cache time in seconds for 303 redirects (max-age and expires)
# a redirectMaxAge of 0 means no cache headers
redirectMaxAge: 0s
# exec configures the executor where scaling image scaling runs.
exec:
# queueSize is the maximum length of the image scaling request queue. If
# more requests are queued we reject the requests with errors.
queueSize: 12
# coreSize is the number of threads to run scaling on before queueing up
# requests.
coreSize : 12
# maxSize is the maximum number of threads to run scaling on, when the queue
# is full.
maxSize : 12
# Dropwizard server configuration, see
# http://www.dropwizard.io/1.2.0/docs/manual/configuration.html#servers
server:
applicationConnectors:
- type: http
port: 8084
adminConnectors:
- type: http
port: 9084
# Dropwizard logging configuration, see
# http://www.dropwizard.io/1.2.0/docs/manual/configuration.html#logging
logging:
level: INFO
# Swagger nets us alot of bogus warnings
loggers:
org.reflections: ERROR
# Hystrix configuration for downstream services.
hystrix:
# This property indicates whether the Hystrix Commands should have a timeout.
timeoutEnabled: true
# Determines whether a circuit breaker will be used to track health and to
# short-circuit requests if it trips.
circuitBreakerEnabled: true
# Sets the maximum number of concurrent Commands to allow before rejecting.
maxConcurrentRequests: 1337
# Commands that are not finished before this timeout will be rejected.
executionTimeout: 30 seconds
Indexer
File: /opt/ace/indexer.yml
# kafka contains settings for event listener that keeps the in-memory content
# cache up to date.
kafka:
# brokers contains a list of the Kafka brokers to connect to.
brokers:
- ace-kafka:9092
# The name of the internal changelist topic. This must match what is set in
# Kafka Connect.
topic: aceEvents
# groupId is the Kafka consumer group name for this application's use of the
# changelist. For an indexer, this must be unique to that indexer.
groupId: ace-indexer
# batchSize controls how many messages are processed in a batch.
batchSize: 128
# bufferSize is the size of the network buffer used to read from Kafka.
bufferSize: 64K
# fetchSize is the maximum number of bytes to read in one Kafka request.
fetchSize: 256K
# reconnectDelay is how long to wait before reconnecting to Kafka after a
# failure.
reconnectDelay: 1s
# partition is the Kafka partition number used by this indexer. Currently
# only partition 0 is supported.
partition: 0
# soTimeout sets the SO_TIMEOUT for the connection to Kafka.
soTimeout: 100s
# auth configures the authentication subsystem.
auth:
# Location of the file containing the secret key for tokens. Don't change
# this setting, use Docker secrets instead.
keyFile: file:/opt/ace/token.properties
# solr contains settings used to talk to Solr
solr:
# collection is the Solr collection the indexer will write to.
collection: internal
# The locations of the ZooKeepers used by solrCloud.
zookeeperHosts:
- ace-solr:9983
# Indicates whether old revisions should be ignored or still indexed
onlyIndexNewerRevisions: true
# When one of the services the indexer uses responds with an error, we use an
# exponential backoff for retries.
retryDelay:
# The minimum time to wait before retries
minimum: 200ms
# The maximum time to wait
maximum: 10s
# The factor the wait time is multiplied with
backoffFactor: 1.5
# contentService configures the connection to the content service
contentService:
# uri is the URI to the content service
uri: http://ace-content-service:8081
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# content controls the connection to the content service.
content:
# variant is the variant used to get the content to index.
variant: aceIndexing
# views is a list of the views to include in the index.
views: ['aceLatest']
# fetchFrequency is how long we wait between checking for new content to
# index.
fetchFrequency: 200 ms
# Dropwizard server configuration, see
# http://www.dropwizard.io/1.2.0/docs/manual/configuration.html#servers
server:
applicationConnectors:
- type: http
port: 8085
adminConnectors:
- type: http
port: 9085
gzip:
enabled: false
# Dropwizard logging configuration, see
# http://www.dropwizard.io/1.2.0/docs/manual/configuration.html#logging
logging:
level: INFO
# Swagger nets us alot of bogus warnings
loggers:
org.reflections: ERROR
appenders:
- type: console
threshold: TRACE
# Hystrix configuration for downstream services.
hystrix:
# This property indicates whether the Hystrix Commands should have a timeout.
timeoutEnabled: true
# Determines whether a circuit breaker will be used to track health and to
# short-circuit requests if it trips.
circuitBreakerEnabled: true
# Sets the maximum number of concurrent Commands to allow before rejecting.
maxConcurrentRequests: 1337
# Commands that are not finished before this timeout will be rejected.
executionTimeout: 30 seconds
Taxonomy Service
File: /opt/ace/taxonomy-service.yml
# collection is the Solr core to use to find categorization in the search service
collection: internal
# auth configures the authentication subsystem.
auth:
# Location of the file containing the secret key for tokens. Don't change
# this setting, use Docker secrets instead.
keyFile: file:/opt/ace/token.properties
# searchService configures the connection to the search service
searchService:
# uri is the URI to the file service
uri: http://ace-search-service:8086
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# contentService configures the connection to the content service
contentService:
# uri is the URI to the content service
uri: http://ace-content-service:8081
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# Dropwizard server configuration, see
# http://www.dropwizard.io/1.2.0/docs/manual/configuration.html#servers
server:
applicationConnectors:
- type: http
port: 8087
adminConnectors:
- type: http
port: 9087
# Dropwizard logging configuration, see
# http://www.dropwizard.io/1.2.0/docs/manual/configuration.html#logging
logging:
level: INFO
# Swagger nets us alot of bogus warnings
loggers:
org.reflections: ERROR
# Hystrix configuration for downstream services.
hystrix:
# This property indicates whether the Hystrix Commands should have a timeout.
timeoutEnabled: true
# Determines whether a circuit breaker will be used to track health and to
# short-circuit requests if it trips.
circuitBreakerEnabled: true
# Sets the maximum number of concurrent Commands to allow before rejecting.
maxConcurrentRequests: 1337
# Commands that are not finished before this timeout will be rejected.
executionTimeout: 30 seconds
Search Service
File: /opt/ace/search-service.yml
# contentService configures the connection to the content service
contentService:
# uri is the URI to the login service
uri: http://ace-content-service:8081
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# permissionService configures the connection to the permission service
permissionService:
# uri is the URI to the login service
uri: http://ace-content-service:8081
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# The locations of the ZooKeepers used by solrCloud.
zookeeperHosts:
- ace-solr:9983
# Configures how searches work.
search:
# Each collection can have its own configuration.
collections:
# This is the collection used for internal searches, e.g. in the GUI.
internal:
# If a query doesn't include a view, this view is used by default. For
# the internal index we default to the latest version.
defaultView: aceLatest
# This is the collection for content that should be visible to everyone
public:
# Public index defaults to public view.
defaultView: acePublic
# Views in this list are available to anonymous users, bypassing the
# permission system.
noAuthViews:
- acePublic
# Maximum number of requests to the content service per incoming request.
requestScaling: 50
# Authentication subsystem configuration.
auth:
# Location of the file containing the secret key for tokens. Don't change
# this setting, use Docker secrets instead.
keyFile: file:/opt/ace/token.properties
# Dropwizard server configuration, see
# http://www.dropwizard.io/1.2.0/docs/manual/configuration.html#servers
server:
applicationConnectors:
- type: http
port: 8086
adminConnectors:
- type: http
port: 9086
# Dropwizard server configuration, see
# http://www.dropwizard.io/1.2.0/docs/manual/configuration.html#servers
logging:
level: INFO
# Swagger nets us alot of bogus warnings
loggers:
org.reflections: ERROR
# Hystrix configuration for downstream services.
hystrix:
# This property indicates whether the Hystrix Commands should have a timeout.
timeoutEnabled: true
# Determines whether a circuit breaker will be used to track health and to
# short-circuit requests if it trips.
circuitBreakerEnabled: true
# Sets the maximum number of concurrent Commands to allow before rejecting.
maxConcurrentRequests: 1337
# Commands that are not finished before this timeout will be rejected.
executionTimeout: 30 seconds