Merge pull request #10125 from hashicorp/docs-autoscaler-409-plugins

autoscaler plugin docs

website/content/docs/autoscaling/internals/index.mdx

@@ -11,3 +11,6 @@ description: >
 
 This section covers the internals of the Nomad Autoscaler and explains the
 technical details of how it functions, its architecture, and sub-systems.
+
+- [Autoscaler plugins](/docs/autoscaling/internals/plugins)
+- [Check calculations](/docs/autoscaling/interals/checks)

website/content/docs/autoscaling/internals/plugins/apm.mdx

+---
+layout: docs
+page_title: APM Plugins
+sidebar_title: APM
+description: Learn how to author a Nomad Autoscaler APM plugin.
+---
+
+# APM Plugins
+
+APM plugins are used by the autoscaler to interact with an external APM system,
+returning metrics that are used by the autoscaler to inform scaling actions.
+
+For a real-world example of a Nomad APM plugin implementation, see the
+[`prometheus` plugin](https://github.com/hashicorp/nomad-autoscaler/tree/main/plugins/builtin/apm/prometheus).
+
+## Authoring APM Plugins
+
+Authoring an APM plugin in Go can be accomplished by implementing the
+[`apm.APM`][apm_plugin] interface, alongside a `main` package to launch the plugin.
+
+The [`no-op` APM plugin][noop_plugin] can be used as a starting point for new APM
+plugins.
+
+## APM Plugin API
+
+The [base plugin][base_plugin] interface must be implemented in addition to the
+following functions.
+
+#### `Query(query string, timeRange sdk.TimeRange) (sdk.TimestampedMetrics, error)`
+
+The `Query` function is called by the agent during policy
+evaluation. The `query` argument is the opaque string from the scaling policy,
+and the `timeRange` indicates the period of time over which the query should be
+made. The response is a series of timestamped metrics; as such, the query semantics
+should be such that the backing APM will return a time series. An example is the
+CPU utilization of a task group, averaged over all current allocations.
+
+#### `QueryMultiple(query string, timeRange sdk.TimeRange) ([]sdk.TimestampedMetrics, error)`
+
+The `QueryMultiple` method is similar to `Query`, except that the interface allows
+multiple time series to be returned. This endpoint is currently only used for
+[Dynamic Application Sizing][das].
+An example would be to return the CPU utilization for all allocations during
+the time range.
+
+[apm_plugin]: https://github.com/hashicorp/nomad-autoscaler/blob/v0.3.0/plugins/apm/apm.go#L11
+[base_plugin]: /docs/autoscaling/internals/plugins/base
+[noop_plugin]: https://github.com/hashicorp/nomad-autoscaler/tree/v0.3.0/plugins/test/noop-apm
+[das]: /docs/autoscaling#dynamic-application-sizing

website/content/docs/autoscaling/internals/plugins/base.mdx

+---
+layout: docs
+page_title: Base Plugin
+sidebar_title: Base
+description: Learn about how to author a Nomad Autoscaler plugin.
+---
+
+# Base Plugin
+
+The base plugin is an interface required of all autoscaler plugins,
+providing a set of common operations.
+
+## Plugin API
+
+#### `PluginInfo() (*PluginInfo, error)`
+
+A `PluginInfo` contains metadata about the plugin. For example,
+the Prometheus APM plugin returns the following;
+
+```go
+PluginInfo{
+    // Name of the plugin
+    Name: "prometheus",
+    // Plugin type: "apm", "strategy", or "target"
+    PluginType: "apm"
+}
+```
+
+#### `SetConfig(config map[string]string) error`
+
+The `SetConfig` function is called when starting an instance of the plugin. It contains the
+configuration for a named instance of the plugin as provided in the autoscaler [agent config][plugin_config].
+
+[plugin_config]: /docs/autoscaling/agent

website/content/docs/autoscaling/internals/plugins/index.mdx

+---
+layout: docs
+page_title: Plugins
+sidebar_title: Plugins
+description: Learn about how external plugins work in the Nomad Autoscaler.
+---
+
+# Plugins
+
+The Nomad Autoscaler uses a plugin framework which allows users to extend its
+functionality. The design of the plugin system is inspired by the
+[plugin system][nomad_plugin_system] used in Nomad for task drivers and
+devices.
+
+The following components are currently pluggable within the Nomad Autoscaler:
+
+- [APMs](/docs/autoscaling/internals/plugins/apm)
+- [Strategies](/docs/autoscaling/internals/plugins/strategy)
+- [Targets](/docs/autoscaling/internals/plugins/target)
+
+In addition, each plugin implements a [base](/docs/autoscaling/internals/plugins/base)
+plugin functionality.
+
+# Architecture
+
+The Nomad Autoscaler plugin framework uses the [go-plugin][goplugin] project to expose
+a language-independent plugin interface. Plugins implement a set of gRPC
+services and methods which the Autoscaler agent manages by running the plugin
+and calling the implemented RPCs. This means that plugins are free to be
+implemented in the author's language of choice.
+
+To make plugin development easier, a set of Go interfaces and structs exist for
+each plugin type that abstract the `go-plugin` and gRPC interfaces. The guides in
+this documentation reference these abstractions for ease of use.
+
+The existing plugins can serve as examples; in addition, no-op external plugins
+are available in the [autoscaler repo][noop_plugins].
+
+[goplugin]: https://github.com/hashicorp/go-plugin
+[nomad_plugin_system]: /docs/internals/plugins
+[noop_plugins]: https://github.com/hashicorp/nomad-autoscaler/tree/main/plugins/test

website/content/docs/autoscaling/internals/plugins/strategy.mdx

+---
+layout: docs
+page_title: Strategy Plugins
+sidebar_title: Strategy
+description: Learn how to author a Nomad Autoscaler Strategy plugin.
+---
+
+# Strategy Plugins
+
+Strategy plugins are used by the autoscaler to implement the logic of scaling strategy.
+They typically consume metrics from the configured APM plugin and the current scaling value
+from the target plugin and compute a new desired value for the scaling target.
+
+For a real-world example of a Nomad strategy plugin implementation, see the
+[`target-value` plugin][target_value].
+
+## Authoring Strategy Plugins
+
+Authoring a strategy plugin in Go can be accomplished by implementing the
+[`strategy.Strategy`][strategy_plugin] interface, alongside a `main` package to launch the plugin.
+
+The [`no-op` Strategy plugin][noop_plugin] can be used as a starting point for new Strategy
+plugins.
+
+## Strategy Plugin API
+
+The [base plugin][base_plugin] interface must be implemented in addition to the
+following functions.
+
+#### `Run(eval *sdk.ScalingCheckEvaluation, count int64) (*sdk.ScalingCheckEvaluation, error)`
+
+The `Run` function is called by the agent during policy
+evaluation. The `eval` argument contains [information](https://github.com/hashicorp/nomad-autoscaler/blob/v0.3.0/sdk/eval_oss.go#L8)
+about the current policy evaluation, including the policy specification and the metrics from the APM. The `count`
+argument includes the current value of the scaling target. The returned struct should include the result
+from applying the strategy.
+
+[strategy_plugin]: https://github.com/hashicorp/nomad-autoscaler/blob/v0.3.0/plugins/strategy/strategy.go#L11
+[base_plugin]: /docs/autoscaling/internals/plugins/base
+[noop_plugin]: https://github.com/hashicorp/nomad-autoscaler/tree/v0.3.0/plugins/test/noop-strategy
+[target_value]: https://github.com/hashicorp/nomad-autoscaler/tree/v0.3.0/plugins/builtin/strategy/target-value

website/content/docs/autoscaling/internals/plugins/target.mdx

+---
+layout: docs
+page_title: Target Plugins
+sidebar_title: Target
+description: Learn how to author a Nomad Autoscaler Target plugin.
+---
+
+# Target Plugins
+
+Target plugins are used by the autoscaler to retrieve information about a
+scaling target and also to perform a scaling action against them.
+
+For a real-world example of a Nomad Target plugin implementation, see the
+[Nomad `group->count` plugin][nomad_group_count_plugin].
+
+## Authoring Target Plugins
+
+Authoring a target plugin in Go can be accomplished by implementing the
+[`target.Target`][target_plugin] interface, alongside a `main` package to
+launch the plugin.
+
+The [`no-op` Target plugin][noop_plugin] can be used as a starting point for new Target
+plugins.
+
+## Target Plugin API
+
+The [base plugin][base_plugin] interface must be implemented in addition to the
+following functions.
+
+#### `Scale(action sdk.ScalingAction, config map[string]string) error`
+
+The `Scale` method is called by the agent during policy evaluation. The `action`
+argument specifies the [details][scaling_action_sdk] about the scaling action
+that should be made against the target. `config` includes the details about the
+scaling target that were provided in the [scaling policy][policy_target].
+
+#### `Status(config map[string]string) (*sdk.TargetStatus, error)`
+
+The `Status` method is called by the agent in order to determine the current
+status of a scaling target. This is performed as part of policy evaluation,
+and the [information][target_status_sdk] returned may be used by the scaling
+strategy to inform the next scaling action. Information returned includes
+current scaling level, readiness, and arbitrary metadata.
+
+[nomad_group_count_plugin]: https://github.com/hashicorp/nomad-autoscaler/tree/v0.3.0/plugins/builtin/target/nomad
+[target_plugin]: https://github.com/hashicorp/nomad-autoscaler/blob/v0.3.0/plugins/target/target.go#L12
+[base_plugin]: /docs/autoscaling/internals/plugins/base
+[noop_plugin]: https://github.com/hashicorp/nomad-autoscaler/tree/v0.3.0/plugins/test/noop-target
+[scaling_action_sdk]: https://github.com/hashicorp/nomad-autoscaler/blob/v0.3.0/sdk/strategy.go#L25
+[policy_target]: /docs/autoscaling/policy#target
+[target_status_sdk]: https://github.com/hashicorp/nomad-autoscaler/blob/v0.3.0/sdk/target.go#L6

website/content/docs/autoscaling/plugins/external/index.mdx

+---
+layout: docs
+page_title: 'Autoscaler Plugins: Community Supported'
+sidebar_title: Community
+description: A list of community-supported Autoscaler Plugins.
+---
+
+# Community Supported
+
+If you have authored an autoscaler plugin that you believe will be useful to the
+broader Nomad community and you are committed to maintaining the plugin, please
+file a PR to add your plugin to this page.
+
+For details on authoring an autoscaler plugin, please refer to the [plugin
+authoring guide][plugin_guide].
+
+Below is a list of community-support plugins available for the Nomad autoscaler.
+
+## Target Plugins
+
+- [OpenStack Senlin][senlin]
+
+[plugin_guide]: /docs/autoscaling/internals/plugins
+[senlin]: https://github.com/dkt26111/nomad-senlin-autoscaler

website/content/docs/autoscaling/plugins/index.mdx

@@ -9,15 +9,17 @@ description: Plugins are used to architect the Nomad Autoscaler into distinct ar
 
 Plugins are an essential part of the Nomad Autoscaler architecture. The Autoscaler
 uses the [go-plugin][go_plugin_github] library to implement an ecosystem of
-different types of plugins. Each plugin type is responsible for a specific task;
-APM plugins retrieve metrics about the workloads being monitored and Strategy
-plugins decide which actions Nomad should execute to keep the policy valid. The
-flexibility of plugins allows the Nomad Autoscaler to be extended to meet specific
-business requirements or technology use cases.
+different types of plugins. Each plugin type is responsible for a specific task:
+APM plugins retrieve metrics about the workloads being monitored; strategy
+plugins decide the scaling action to satisfy the scaling policy; and target
+plugins perform the scaling action. The flexibility of plugins allows the Nomad
+Autoscaler to be extended to meet specific business requirements or technology
+use cases.
 
 The Nomad Autoscaler currently ships with a number of built-in plugins to ease
 the learning curve. Details of these can be found in the side menu, under the
-specific plugin type sections.
+specific plugin type sections. The autoscaler also supports external plugins; see
+this list of [community-supported plugins][community_plugins].
 
 # General Options
 
@@ -63,3 +65,4 @@ targets.
   strongly discouraged.
 
 [go_plugin_github]: https://github.com/hashicorp/go-plugin
+[community_plugins]: /docs/autoscaling/plugins/external

website/content/docs/autoscaling/plugins/target.mdx

@@ -18,6 +18,8 @@ Below is a list of plugins you can use with the Nomad Autoscaler:
 - [AWS AutoScaling Group][aws_asg_target]
 - [Azure Virtual Machine Scale Set][azure_vmss_target]
 
+
+
 ## Nomad Task Group Target
 
 The Nomad task group target indicates the scalable resource is a Nomad job

website/data/docs-navigation.js

@@ -419,11 +419,30 @@ export default [
       'telemetry',
       {
         category: 'plugins',
-        content: ['apm', 'strategy', 'target'],
+        content: [
+          'apm',
+          'strategy',
+          'target',
+          {
+            category: 'external',
+            content: [],
+          }
+        ],
       },
       {
         category: 'internals',
-        content: ['checks'],
+        content: [
+          'checks',
+          {
+            category: 'plugins',
+            content: [
+              'base',
+              'apm',
+              'strategy',
+              'target',
+            ],
+          }
+        ],
       },
     ],
   },