Service Configuration
The ACE Services are configured by service .yml files. Each service has got two configuration files — one with the defaults, and one with project overrides.
To override a certain property, create a .yml file and map it to /opt/ace/xxx-service.yml in
your docker container (for example by adding a configs entry to your docker-compose.yml).
services:
...
content-service:
configs:
- source: content-service
target: /opt/ace/content-service.yml
...
configs:
content-service:
file: <path-to-config>/content-service.yml
...
Connections to other ACE services
There is communication between services in ace, and it is possible to customise some settings for this, to better suit the deployment size and use case.
The following are the services used from other services in ACE:
- Content Service; Used from most other ACE serivices.
- File Service; Used from the File Delivery Service, Content Service for content Import and the Image Service.
- Search Service; Used from the Content Service in Callbacks.
Common settings for these connections:
someService:
# uri is the URI to the file service
uri: http://ace-file-service:8082
# maxConnections is how many connections are allowed to the backend service
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
Normaly the defaults should work fine, but if you i.e have many content services, you can increase the number of connections to it from other services.
Authentication secret key
Most ACE services require authentication, implemented by the use of
JSON Web Tokens.
In order to generate and validate such tokens, the authentication properties issuer and key must be configured.
These are supplied in a token.properties file:
# token.properties
ace.token.issuer=<your organization here>
ace.token.key=<secret key>
Minimum key length
The secret key must be at least 32 characters (256 bits). Keys shorter than this will cause
services to fail at startup with a WeakKeyException. This is enforced by the JWT library (JJWT).
The key should be a full-entropy random string. There are a number of ways to generate it, like using a password generator. Alternatively you can use OpenSSL:
openssl rand -hex 32
The token properties file is read from /opt/ace/token.properties. To keep it safe, we recommend supplying it as a
Docker secret.
services:
...
content-service:
secrets:
- source: token
target: /opt/ace/token.properties
...
secrets:
token:
file: <path-to-token>/token.properties
...
Content Service
File: /opt/ace/content-service.yml
##
# Content Service default configuration
##
# contentService contains configuration for the content service itself.
contentService:
# noAuthViews is a list of views that can be accessed without
# authenticating. Any content version on one of these views is available to
# anyone who can connect to the service.
noAuthViews:
- acePublic
# allowedViews can restrict the service to only resolve and serve versions
# that are on this list of views, even for authenticated users. The empty
# list allows all views. Other services may expect this not to be set.
allowedViews: []
# configContent names the configuration content used for the service. This
# defines all the variants and callbacks and so on.
configContent: aceCallbacksConfig
# opTimeout is the timeout for content operations. A content operation often
# involves multiple operations against Couchbase, this is the timeout for
# all those operations.
opTimeout: 30 seconds
# kafka contains settings for event listener that keeps the in-memory content
# cache up to date.
kafka:
# brokers contains a list of the Kafka brokers to connect to.
brokers:
- ace-kafka:9092
# The name of the internal changelist topic. This must match what is set in
# Kafka Connect.
topic: aceEvents
# groupId is the Kafka consumer group name for this application's use of the
# changelist. Sharing with other content service instances is fine because
# we don't care about what happened while we were down.
groupId: ace-cache
# batchSize controls how many messages are processed in a batch.
batchSize: 5000
# bufferSize is the size of the network buffer used to read from Kafka.
bufferSize: 64K
# fetchSize is the maximum number of bytes to read in one Kafka request.
fetchSize: 256K
# reconnectDelay is how long to wait before reconnecting to Kafka after a
# failure.
reconnectDelay: 1s
# partition is the Kafka partition number used by this indexer. Currently
# only partition 0 is supported.
partition: 0
# soTimeout sets the SO_TIMEOUT for the connection to Kafka.
soTimeout: 100s
reindexFeeder:
# The list of Kafka brokers to use for publishing the re-indexing events.
brokers:
- ace-kafka:9092
# The name of the topic to publish the re-indexing events to.
topic: aceReindexEvents
# The number of times to retry sending each batch of Kafka records in case of failures.
maxRetries: 5
# How long to wait before sending a batch of Kafka records to the topic. A value
# larger than 0 allows for greater batching.
lingerTime: 1ms
eventPublisher:
# Whether publishing of content events is enabled or not.
enabled: true
# Kafka settings
kafka:
# The list of Kafka brokers to use for publishing content events.
brokers:
- ace-kafka:9092
# The name of the topic to publish the content events to.
topic: aceContentEvents
# The number of times to retry sending each batch of Kafka records in case of failures.
maxRetries: 1
# How long to wait before sending a batch of Kafka records to the topic. A value
# larger than 0 allows for greater batching.
lingerTime: 10ms
# The maximum amount of time to wait before timing out when there is a problem
# handing over an event message to the Kafka message producer.
maxBlockTime: 200ms
eventConsumer:
# Whether consumption of events (and implicitly also the subscription
# service endpoint) is enabled or not.
enabled: true
# How often to consume events from the content events topic
consumeInterval: 250 millisecond
# The maximum number of records to handle in every batch
maxPollRecords: 100
# The maximum time to wait for Kafka to respond with records in a consume attempt
kafkaPollTimeout: 10 second
# Kafka settings
kafka:
# The list of Kafka brokers to consume content events from
brokers:
- ace-kafka:9092
# The name of the topic to consume the content events from.
topic: aceContentEvents
# The Kafka consumer group name prefix for the event consumer.
groupId: message-service
# auth configures the authentication subsystem.
auth:
# Location of the file containing the secret key for tokens. Don't change
# this setting, use Docker secrets instead.
keyFile: file:/opt/ace/token.properties
# Properties that can be used to override the settings from
# token.properties or to use when secure file mounting is not available.
issuer: ${ACE_TOKEN_ISSUER:-}
key: ${ACE_TOKEN_KEY:-}
# login configures the login system, which checks user's passwords against
# LDAP before authenticating them.
login:
# The session timeout for the authentication token issued by ACE
sessionDuration: 12 hours
# Whether Cognito is enabled or not. Regardless, password file functionality
# is still in play in parallel.
cognitoEnabled: ${COGNITO_ENABLED:-false}
cognito:
# Region of the user pool (eu-west-1 etc)
region: ${COGNITO_REGION:-}
# App client id
clientId: ${COGNITO_CLIENT_ID:-}
# Cognito pool id
poolId: ${COGNITO_POOL_ID:-}
# jwk token source for the configured user pool
jwkSource: ${COGNITO_JWK_SOURCE:-}
# Location of the file containing the Cognito IAM credentials. Don't change
# this setting, use Docker secrets instead.
credentialsFile: file:/opt/ace/cognito-credentials.json
# IAM access key to be used for user verification with cognito
accessKey: ${COGNITO_ACCESS_KEY:-}
# IAM secret key to be used for user verification with cognito
secretKey: ${COGNITO_SECRET_KEY:-}
# A list of approved user search attributes that will be used when searching for
# users in AWS Cognito. An empty list of search attributes will disable free text search.
searchAttributes:
- username
# Whether LDAP is enabled or not. Regardless, password file functionality
# is still in play in parallel.
ldapEnabled: false
# Configures the connection to the LDAP server.
ldap:
# userObjectClass is the name of the LDAP object class that represents
# users.
userObjectClass: inetOrgPerson
# loginNameAttribute is the LDAP attribute where a user's login name is
# stored.
loginNameAttribute: uid
# A list of approved user attributes that will be used when searching for
# users in LDAP. An empty list of search attributes will disable free text search.
searchAttributes:
- uid
# userSearchBaseDN is used as a base for all user searches
userSearchBaseDN:
# an LDAP query that will be applied as a filter to user authentication
# and listing operations
userFilter:
# Provider URL for the LDAP server
providerUrl: ldaps://ace-ldap:636
# Location of the file containing the LDAP credentials. Don't change
# this setting, use Docker secrets instead.
credentialsFile: file:/opt/ace/ldap-credentials.json
# Properties that can be used to override the settings from ldap-credentials.json
# or to use when secure file mounting is not available.
securityPrincipal: ${LDAP_SECURITY_PRINCIPAL:-}
securityCredentials: ${LDAP_SECURITY_CREDENTIALS:-}
# The maximum number of milliseconds to wait for the connection to the LDAP server.
connectionTimeout: 20000
# The maximum number of milliseconds to wait for the result of an LDAP user search operation.
searchTimeout: 15000
# couchbase configures how the content service uses Couchbase.
couchbase:
# Location of the file containing the Couchbase credentials. Don't change
# this setting, use Docker secrets instead.
credentialsFile: file:/opt/ace/couch-credentials.json
# Properties that can be used to override the settings from credentialsFile
# or to use when secure file mounting is not available.
bucket: ${COUCHBASE_BUCKET:-}
username: ${COUCHBASE_USERNAME:-}
password: ${COUCHBASE_PASSWORD:-}
# waitToPersist controls whether the server waits for Couchbase to persist
# before responding to the client. Turning this off improves performance but
# risks losing acknowledged writes when a node crashes. This should not be
# turned off in production, but in development mode it is disabled.
# NOTE: Set COUCHBASE_WAIT_TO_PERSIST=false in development for better performance
waitToPersist: ${COUCHBASE_WAIT_TO_PERSIST:-true}
# opTimeout is the timeout for content operations. A content operation often
# involves multiple operations against Couchbase, this is the timeout for
# all those operations. DEPRECATED. Use contentService.opTimeout setting instead.
opTimeout: 30 seconds
# metricsFrequency controls how often Couchbase events are recorded to the
# Prometheus endpoint.
metricsFrequency: 10 seconds
# The following settings are standard Couchbase client settings, see
# http://docs.couchbase.com/sdk-api/couchbase-java-client-2.3.6/
# connectionString lists the Couchbase nodes to connect to on startup. This
# doesn't need to be exhaustive because Couchbase tells us which nodes are
# in the cluster, but at least one of these nodes must be available on
# startup.
connectionString: couchbase://ace-couch
#kvTimeout:
#connectTimeout:
#analyticsTimeout:
#managementTimeout:
# This timeout may be need to tuned if N1QL queries times out. Especially relevant
# for heavier queries like those for re-indexing and content statistics.
queryTimeout: 10 minutes
#socketConnectTimeout:
#searchTimeout:
#viewTimeout:
#maxRequestLifetime:
#kvConnections:
#queryConnections:
#searchConnections:
#viewConnections:
fileService:
# uri is the URI to the file service
uri: http://ace-file-service:8080
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# client name
clientName: ContentService
searchService:
# uri is the URI to the search service
uri: http://ace-search-service:8080
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 10
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# client name
clientName: ContentService
# Dropwizard server configuration, see
# https://www.dropwizard.io/en/release-5.0.x/manual/configuration.html#servers
server:
applicationConnectors:
- type: http
port: 8080
useForwardedHeaders: true
# Allow legacy URI characters (like pipe '|') that are decoded by nginx.
# Required for the localization endpoint which uses pipe-separated contexts.
# See: https://eclipse.dev/jetty/documentation/jetty-12/programming-guide/index.html#pg-server-compliance-uri
uriCompliance: UNSAFE
# Administrative port - not used in current version of ACE
adminConnectors:
- type: http
port: 9080
gzip:
enabled: false
# The following request log filter factory will prevent any request with a status
# in range [100, 299] (inclusive) from being logged in the request log.
# requestLog:
# appenders:
# - type: console
# filterFactories:
# - type: status-filter-factory
# rules:
# - from: 100
# to: 299
# action: DENY
# Dropwizard logging configuration, see
# https://www.dropwizard.io/en/release-5.0.x/manual/configuration.html#logging
logging:
level: INFO
appenders:
- type: console
threshold: TRACE
# Hystrix configuration for downstream services.
hystrix:
# This property indicates whether the Hystrix Commands should have a timeout.
timeoutEnabled: true
# Determines whether a circuit breaker will be used to track health and to
# short-circuit requests if it trips.
circuitBreakerEnabled: true
# Maximum concurrent Hystrix commands per command key (using SEMAPHORE isolation).
# This limits how many requests can be in-flight to downstream services.
# The connection pool (maxConnections) provides additional per-client limiting.
maxConcurrentRequests: 100
# Commands that are not finished before this timeout will be rejected.
executionTimeout: 30 seconds
# Configuration of the in-memory content caches
cache:
# How often the cache will attempt to evict stale content
evictionInterval: 250 millisecond
# Maximum number of aliases to keep in cache
aliasCacheMaxSize: 64000
# Maximum number of content info to keep in cache
contentInfoCacheMaxSize: 32000
# Maximum number of content versions to keep in cache
versionCacheMaxSize: 64000
# Controls content overview statistics
statistics:
# Enable content overview statistics
enabled: false
# Minimum time between content statistics generation
minimumInterval: 5 minutes
# Maximum number of concurrent interactions with the ACE content storage
maxContentStorageConcurrency: 6
# Sql connection configuration
sql:
# JDBC connection URL.
# Should be on the form "jdbc:mysql" for standard MySQL. For AWS RDS MySQL, the URL
# should instead be on the form "jdbc:aws-wrapper:mysql" in order to activate AWS-specific
# functionality such as IAM authentication and failover handling.
# Example: jdbc:aws-wrapper:mysql://ace-mysql:3306/ace.
url: ${SQL_URL:-}
# JDBC username.
username: ${SQL_USERNAME:-}
# JDBC password.
password: ${SQL_PASSWORD:-}
# Maximum time (in ms) to wait for a database
# connection before throwing an error.
connectionTimeout: 30000
# Maximum size of the database connection pool.
maximumPoolSize: 10
# Maximum number of versions of a content to keep.
maxVersions: 32
# AWS (RDS) specific settings
aws:
# Whether to use IAM authentication when connecting to the database. Any
# password set will be ignored when this is set to true.
useIamAuth: false
# Type of content storage; either COUCHBASE or SQL
contentStorageType: COUCHBASE
bootstrapper:
# Whether to automatically bootstrap (create when not existing) core system
# configuration content on startup of the Content Service.
enabled: true
File Delivery Service
File: /opt/ace/file-delivery-service.yml
##
# File Delivery Service default configuration
##
# timeout is how long a request is allowed to take before it aborts.
timeout: 30 seconds
# auth configures the authentication subsystem.
auth:
# Location of the file containing the secret key for tokens. Don't change
# this setting, use Docker secrets instead.
keyFile: file:/opt/ace/token.properties
# Properties that can be used to override the settings from token.properties
# or to use when secure file mounting is not available.
issuer: ${ACE_TOKEN_ISSUER:-}
key: ${ACE_TOKEN_KEY:-}
# fileService configures the connection to the file service
fileService:
# uri is the URI to the file service
uri: http://ace-file-service:8080
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# client name
clientName: FileDeliveryService
# contentService configures the connection to the content service
contentService:
# uri is the URI to the content service
uri: http://ace-content-service:8080
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# client name
clientName: FileDeliveryService
# Dropwizard server configuration, see
# https://www.dropwizard.io/en/release-5.0.x/manual/configuration.html#servers
server:
applicationConnectors:
- type: http
port: 8080
useForwardedHeaders: true
# Administrative port - not used in current version of ACE
adminConnectors:
- type: http
port: 9080
# Dropwizard logging configuration, see
# https://www.dropwizard.io/en/release-5.0.x/manual/configuration.html#logging
logging:
level: INFO
# Hystrix configuration for downstream services.
hystrix:
# This property indicates whether the Hystrix Commands should have a timeout.
timeoutEnabled: true
# Determines whether a circuit breaker will be used to track health and to
# short-circuit requests if it trips.
circuitBreakerEnabled: true
# Maximum concurrent Hystrix commands per command key (using SEMAPHORE isolation).
# This limits how many requests can be in-flight to downstream services.
# The connection pool (maxConnections) provides additional per-client limiting.
maxConcurrentRequests: 100
# Commands that are not finished before this timeout will be rejected.
executionTimeout: 30 seconds
File: /opt/ace/file-service.yml
File Service
##
# File Service default configuration
##
# S3 configuration for Amazon S3 compatible object storage. This is where files are stored.
s3:
bucket: ${BUCKET}
# region to store files in.
region: ${REGION}
# serviceEndpoint lets you use a non-AWS S3 implementation.
serviceEndpoint: ${S3_ENDPOINT}
# Properties that can be used to override the settings from access_key and secret_key files
# or to use when secure file mounting is not available.
secretKey: ${S3_SECRET_KEY:-}
accessKey: ${S3_ACCESS_KEY:-}
# Configures the Amazon S3 client, see
# http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/ClientConfiguration.html
client:
# Timeout when establishing a connection.
connectionTimeout: 10s
# Maximum number of connections to open.
maxConnections: 50
# socketTimeout: 50s
connectionMaxIdleTime: 60s
# Connections are automatically closed after this amount of time. If not set connections are
# only closed if left idle.
connectionTTL: 60s
# Timeout for individual HTTP requests to S3. If not set there is no timeout.
# requestTimeout: 60s
# Timeout for a complete request to S3. If not set there is no timeout.
# clientExecutionTimeout: 60s
# Whether to use gzip compression for data transport.
gzip: false
# How many times to retry a request on a retryable error from S3.
maxErrorRetry: 0
# The maximum size of files uploaded to the File Service.
maxFileUploadSize: 100M
# auth configures the authentication subsystem.
auth:
# Location of the file containing the secret key for tokens. Don't change
# this setting, use Docker secrets instead.
keyFile: file:/opt/ace/token.properties
# Properties that can be used to override the settings from token.properties
# or to use when secure file mounting is not available.
issuer: ${ACE_TOKEN_ISSUER:-}
key: ${ACE_TOKEN_KEY:-}
# Dropwizard server configuration, see
# https://www.dropwizard.io/en/release-5.0.x/manual/configuration.html#servers
server:
applicationConnectors:
- type: http
port: 8080
useForwardedHeaders: true
# Administrative port - not used in current version of ACE
adminConnectors:
- type: http
port: 9080
# Dropwizard logging configuration, see
# https://www.dropwizard.io/en/release-5.0.x/manual/configuration.html#logging
logging:
level: INFO
# Hystrix configuration for downstream services.
hystrix:
# This property indicates whether the Hystrix Commands should have a timeout.
timeoutEnabled: true
# Determines whether a circuit breaker will be used to track health and to
# short-circuit requests if it trips.
circuitBreakerEnabled: true
# Maximum concurrent Hystrix commands per command key (using SEMAPHORE isolation).
# This limits how many requests can be in-flight to downstream services.
# The connection pool (maxConnections) provides additional per-client limiting.
maxConcurrentRequests: 100
# Commands that are not finished before this timeout will be rejected.
executionTimeout: 30 seconds
The file service needs access and secret keys for S3 provided to it in order to function. There are a couple of ways to provide these. In order of precedence:
- As environment variables
S3_SECRET_KEYandS3_ACCESS_KEYfor the container. - As configuration options in the file service YML configuration. Please see above for example.
- As mounted files
/run/secrets/access_keyand/run/secrets/secret_keyin the container. - Any of the other ways the AWS Java SDK reads the credentials. See AWS SDK documentation for more information.
Image Service
File: /opt/ace/image-service.yml
##
# Image Service default configuration
##
# fileService configures the connection to the file service
fileService:
# uri is the URI to the file service
uri: http://ace-file-service:8080
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# client name
clientName: ImageService
# auth configures the authentication subsystem.
auth:
# Location of the file containing the secret key for tokens. Don't change
# this setting, use Docker secrets instead.
keyFile: file:/opt/ace/token.properties
# Properties that can be used to override the settings from token.properties
# or to use when secure file mounting is not available.
issuer: ${ACE_TOKEN_ISSUER:-}
key: ${ACE_TOKEN_KEY:-}
# contentService configures the connection to the content service
contentService:
# uri is the URI to the content service
uri: http://ace-content-service:8080
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# client name
clientName: ImageService
# view is the view to get the image content from
view: acePublic
# timeout in seconds for reading and processing images.
timeout: 30s
# cache time in seconds for 303 redirects (max-age and expires)
# a redirectMaxAge of 0 means no cache headers
redirectMaxAge: 0s
# exec configures the executor where scaling image scaling runs.
exec:
# queueSize is the maximum length of the image scaling request queue. If
# more requests are queued we reject the requests with errors.
queueSize: 12
# coreSize is the number of threads to run scaling on before queueing up
# requests.
coreSize : 12
# maxSize is the maximum number of threads to run scaling on, when the queue
# is full.
maxSize : 12
# Dropwizard server configuration, see
# https://www.dropwizard.io/en/release-5.0.x/manual/configuration.html#servers
server:
applicationConnectors:
- type: http
port: 8080
useForwardedHeaders: true
# Administrative port - not used in current version of ACE
adminConnectors:
- type: http
port: 9080
# Dropwizard logging configuration, see
# https://www.dropwizard.io/en/release-5.0.x/manual/configuration.html#logging
logging:
level: INFO
# Hystrix configuration for downstream services.
hystrix:
# This property indicates whether the Hystrix Commands should have a timeout.
timeoutEnabled: true
# Determines whether a circuit breaker will be used to track health and to
# short-circuit requests if it trips.
circuitBreakerEnabled: true
# Maximum concurrent Hystrix commands per command key (using SEMAPHORE isolation).
# This limits how many requests can be in-flight to downstream services.
# The connection pool (maxConnections) provides additional per-client limiting.
maxConcurrentRequests: 100
# Commands that are not finished before this timeout will be rejected.
executionTimeout: 30 seconds
watermarking:
# Whether watermarking functionality is enabled or not for the image service.
enabled: false
# A list of content property paths on format <aspectName>/<propertyName> where
# to look for a boolean indicating whether watermarking is enabled for the specific
# content or not. The value of the first property that exists will be used.
enabledContentPropertyPaths: []
# Location of the watermark image, expressed as a URI.
# watermarkImage: file:///opt/ace/watermark.png
# The watermark image placement (location) on the source image.
watermarkPlacement: BOTTOM_LEFT
# The maximum size of the rendered watermark (width / height) overlay as a
# percentage (%) of the source image size.
maximumWatermarkSize: 100
Indexer
File: /opt/ace/indexer.yml
##
# Indexer default configuration
##
# kafka contains settings for event listener that keeps the in-memory content
# cache up to date.
kafka:
# brokers contains a list of the Kafka brokers to connect to.
brokers:
- ace-kafka:9092
# The name of the internal changelist topic. This must match what is set in
# Kafka Connect.
topic: aceEvents
# groupId is the Kafka consumer group name for this application's use of the
# changelist. For an indexer, this must be unique to that indexer.
groupId: ace-indexer
# batchSize controls how many messages are processed in a batch.
batchSize: 32
# bufferSize is the size of the network buffer used to read from Kafka.
bufferSize: 64K
# fetchSize is the maximum number of bytes to read in one Kafka request.
fetchSize: 256K
# reconnectDelay is how long to wait before reconnecting to Kafka after a
# failure.
reconnectDelay: 1s
# partition is the Kafka partition number used by this indexer. Currently
# only partition 0 is supported.
partition: 0
# soTimeout sets the SO_TIMEOUT for the connection to Kafka.
soTimeout: 100s
# auth configures the authentication subsystem.
auth:
# Location of the file containing the secret key for tokens. Don't change
# this setting, use Docker secrets instead.
keyFile: file:/opt/ace/token.properties
# Properties that can be used to override the settings from token.properties
# or to use when secure file mounting is not available.
issuer: ${ACE_TOKEN_ISSUER:-}
key: ${ACE_TOKEN_KEY:-}
# Solr configuration
solr:
# Either CLOUD (Solr Cloud) or STANDALONE (one or more Solr endpoints).
# NOTE: mode CLOUD has to be set together with a non-empty list of
# Zookeeper hosts and STANDALONE with a non-empty list of Solr urls.
mode: CLOUD
# The locations of the Zookeeper nodes to use for the Solr client.
zookeeperHosts:
- ace-solr:9983
# The locations of the Solr endpoints to use for the Solr client.
# A load-balanced Solr client (LBHttpSolrClient) will be used if
# multiple Solr urls are supplied.
#
# WARNING: never configure the ACE Indexer in Solr mode STANDALONE with
# multiple Solr urls; indexing will not work since the load-balanaced
# Solr client can not forward update requests to the current collection leader.
solrUrls: []
# Solr authentication settings
authentication:
# Indicates whether Solr (basic) authentication is enabled.
# Please see https://solr.apache.org/guide/8_11/basic-authentication-plugin.html.
enabled: false
# Solr authentication credentials
credentials:
# The username to use for Solr authentication
username: ${SOLR_USERNAME:-}
# The password to use for Solr authentication
password: ${SOLR_PASSWORD:-}
# Indicates whether old revisions should be ignored or still indexed
onlyIndexNewerRevisions: true
# Controls if the indexer should ignore missing content versions or not.
# By default a missing version error is seen as fatal and will block the
# indexer from proceeding, but in controlled cases you might want to turn
# this option on.
ignoreMissingVersions: false
# When one of the services the indexer uses responds with an error, we use an
# exponential backoff for retries.
retryDelay:
# The minimum time to wait before retries
minimum: 200ms
# The maximum time to wait
maximum: 10s
# The factor the wait time is multiplied with
backoffFactor: 1.5
# contentService configures the connection to the content service
contentService:
# uri is the URI to the content service
uri: http://ace-content-service:8080
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# client name
clientName: Indexer
# collections
collections:
- collectionName: internal
views: [aceLatest]
variant: aceIndexing
# fetchFrequency is how long we wait between checking for new content to
# index.
fetchFrequency: 200 ms
# Dropwizard server configuration, see
# https://www.dropwizard.io/en/release-5.0.x/manual/configuration.html#servers
server:
applicationConnectors:
- type: http
port: 8080
useForwardedHeaders: true
# Administrative port - not used in current version of ACE
adminConnectors:
- type: http
port: 9080
gzip:
enabled: false
# Dropwizard logging configuration, see
# https://www.dropwizard.io/en/release-5.0.x/manual/configuration.html#logging
logging:
level: INFO
appenders:
- type: console
threshold: TRACE
# Hystrix configuration for downstream services.
hystrix:
# This property indicates whether the Hystrix Commands should have a timeout.
timeoutEnabled: true
# Determines whether a circuit breaker will be used to track health and to
# short-circuit requests if it trips.
circuitBreakerEnabled: true
# Maximum concurrent Hystrix commands per command key (using SEMAPHORE isolation).
# This limits how many requests can be in-flight to downstream services.
# The connection pool (maxConnections) provides additional per-client limiting.
# Set higher than other services because the indexer processes Kafka batches in
# parallel - worst case concurrent requests is approximately batchSize x 2 (64),
# with headroom for bursts.
maxConcurrentRequests: 100
# Commands that are not finished before this timeout will be rejected.
executionTimeout: 30 seconds
Taxonomy Service
File: /opt/ace/taxonomy-service.yml
##
# Taxonomy Service default configuration
##
# collection is the Solr core to use to find categorization in the search service
collection: internal
# auth configures the authentication subsystem.
auth:
# Location of the file containing the secret key for tokens. Don't change
# this setting, use Docker secrets instead.
keyFile: file:/opt/ace/token.properties
# Properties that can be used to override the settings from token.properties
# or to use when secure file mounting is not available.
issuer: ${ACE_TOKEN_ISSUER:-}
key: ${ACE_TOKEN_KEY:-}
# searchService configures the connection to the search service
searchService:
# uri is the URI to the file service
uri: http://ace-search-service:8080
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# client name
clientName: TaxonomyService
# contentService configures the connection to the content service
contentService:
# uri is the URI to the content service
uri: http://ace-content-service:8080
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# client name
clientName: TaxonomyService
# Dropwizard server configuration, see
# https://www.dropwizard.io/en/release-5.0.x/manual/configuration.html#servers
server:
applicationConnectors:
- type: http
port: 8080
useForwardedHeaders: true
# Administrative port - not used in current version of ACE
adminConnectors:
- type: http
port: 9080
# Dropwizard logging configuration, see
# https://www.dropwizard.io/en/release-5.0.x/manual/configuration.html#logging
logging:
level: INFO
# Hystrix configuration for downstream services.
hystrix:
# This property indicates whether the Hystrix Commands should have a timeout.
timeoutEnabled: true
# Determines whether a circuit breaker will be used to track health and to
# short-circuit requests if it trips.
circuitBreakerEnabled: true
# Maximum concurrent Hystrix commands per command key (using SEMAPHORE isolation).
# This limits how many requests can be in-flight to downstream services.
# The connection pool (maxConnections) provides additional per-client limiting.
maxConcurrentRequests: 100
# Commands that are not finished before this timeout will be rejected.
executionTimeout: 30 seconds
Search Service
File: /opt/ace/search-service.yml
##
# Search Service default configuration
##
# contentService configures the connection to the content service
contentService:
# uri is the URI to the content service
uri: http://ace-content-service:8080
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# client name
clientName: SearchService
# permissionService configures the connection to the permission service
permissionService:
# uri is the URI to the login service
uri: http://ace-content-service:8080
# maxConnections is how many connections are allowed to the backend service per ip
maxConnections: 20
# maxTotal restricts the total number of outgoing connections to the backend
# service, 0 means maxConnections x 4
maxTotal: 0
# timeout is the read timeout
timeout: 30 seconds
# connectionTimeout is the connection timeout
connectionTimeout: 30 seconds
# client name
clientName: SearchService
# Solr configuration
solr:
# Either CLOUD (Solr Cloud) or STANDALONE (one or more Solr endpoints).
# NOTE: mode CLOUD has to be set together with a non-empty list of
# Zookeeper hosts and STANDALONE with a non-empty list of Solr urls.
mode: CLOUD
# The locations of the Zookeeper nodes to use for the Solr client.
zookeeperHosts:
- ace-solr:9983
# The locations of the Solr endpoints to use for the Solr client.
# A load-balanced Solr client (LBHttpSolrClient) will be used if
# multiple Solr urls are supplied.
#
# WARNING: never configure the ACE Indexer in Solr mode STANDALONE with
# multiple Solr urls; indexing will not work since the load-balanaced
# Solr client can not forward update requests to the current collection leader.
solrUrls: []
# Solr authentication settings
authentication:
# Indicates whether Solr (basic) authentication is enabled.
# Please see https://solr.apache.org/guide/8_11/basic-authentication-plugin.html.
enabled: false
# Solr authentication credentials
credentials:
# The username to use for Solr authentication
username: ${SOLR_USERNAME:-}
# The password to use for Solr authentication
password: ${SOLR_PASSWORD:-}
# Configures how searches work.
search:
# Each collection can have its own configuration.
collections:
# This is the collection used for internal searches, e.g. in the GUI.
internal:
# This is the Solr collection used by this collection
solrCollection: internal
# If a query doesn't include a view, this view is used by default. For
# the internal index we default to the latest version.
defaultView: aceLatest
# This is the collection for content that should be visible to everyone
public:
# This is the Solr collection used by this collection
# solrCollection: public
# Public index defaults to public view.
defaultView: acePublic
# Views in this list are available to anonymous users, bypassing the
# permission system.
noAuthViews:
- acePublic
# Maximum number of requests to the content service per incoming request.
# This limits parallelism for content inlining. Must not exceed hystrix.maxConcurrentRequests
# to avoid semaphore rejection when multiple searches with inlining run concurrently.
requestScaling: 10
# Authentication subsystem configuration.
auth:
# Location of the file containing the secret key for tokens. Don't change
# this setting, use Docker secrets instead.
keyFile: file:/opt/ace/token.properties
# Properties that can be used to override the settings from token.properties
# or to use when secure file mounting is not available.
issuer: ${ACE_TOKEN_ISSUER:-}
key: ${ACE_TOKEN_KEY:-}
# Dropwizard server configuration, see
# https://www.dropwizard.io/en/release-5.0.x/manual/configuration.html#servers
server:
applicationConnectors:
- type: http
port: 8080
useForwardedHeaders: true
# Administrative port - not used in current version of ACE
adminConnectors:
- type: http
port: 9080
# Dropwizard logging configuration, see
# https://www.dropwizard.io/en/release-5.0.x/manual/configuration.html#logging
logging:
level: INFO
# Hystrix configuration for downstream services.
hystrix:
# This property indicates whether the Hystrix Commands should have a timeout.
timeoutEnabled: true
# Determines whether a circuit breaker will be used to track health and to
# short-circuit requests if it trips.
circuitBreakerEnabled: true
# Maximum concurrent Hystrix commands per command key (using SEMAPHORE isolation).
# This limits how many requests can be in-flight to downstream services.
# The connection pool (maxConnections) provides additional per-client limiting.
maxConcurrentRequests: 100
# Commands that are not finished before this timeout will be rejected.
executionTimeout: 30 seconds