Generate the LXD configuration documentation from source code

LXD Specifications
1
Project LXD
Status Implemented
Author(s) @gabrielmougard
Approver(s) @tomp @ru-fu
Release 5.18
Internal ID LX050

Abstract:

We want to generate a list of configuration options that we have in our documentation from comments in the code. We will use these comments in the code to also generate a YAML file that contains all the metadata configuration, which can be consumed by, for example, the LXD UI.

In our use case, we would want a config variable spec generator that can bundle any key-value data pairs alongside metadata to build a sense of hierarchy and identity

Rationale:

We want to associate a unique entry key (or configuration option) to each lxddoc:generate comment group that will also be displayed in the generated documentation, in a swagger fashion. lxd-metadata (name of the binary tool) can associate metadata key-value pairs (here for example, group and key) to data key-value pairs. As a result, it can generate a YAML tree out of the code documentation and also a Markdown document.

Specification

Design

We’ll introduce a new command line tool under lxd/lxd-metadata that parses a go codebase and generates a YAML document (the source of truth of the documentation) and a .TXT document (used for internal documentation templating systems like Sphinx).

The shape of the generated YAML looks like this:

configs:
   <group_key>:
       - <config_option_key1>:
           <key1>: <value1>
           <key2>: <value2>
		…
       - <config_option_key2>:
           …
   <group_key2>:
       ...

Here is an example parsable comment in the codebase :

// lxddoc:generate(group=cluster, key=scheduler.instance)
//
//  <Possibly a very long documentation on multiple lines with Markdown tables, etc.>
// ---
//  shortdesc: Possible values are all, manual and group. See Automatic placement of instances for more information.
//  condition: container
//  default: `all`
//  type: integer
//  liveupdate: `yes`
//  <ANY_KEY>: <ANY_VALUE>

The metadata keys (group and key) in parentheses are used as group_key and config_option_key respectively. Everything between the metadata and the --- is considered a value of the special longdesc key and can hold any multi line comment (a markdown table, graph, etc.)

What comes after are the key/value pairs. The naming of the key can be anything (except the reserved longdesc keyword). Note that each value needs to be a valid YAML value. Only the longdesc field can be of Markdown format.

The generated .TXT section is as follow:

<!-- config group <group_key1> start -->
'''{config:option} <config_option_key1> <group_key1>
:type: string
:liveupdate: `yes`
:shortdesc: Free form user key/value storage (can be used in search).
:condition: container
:default: -


<an optional longdesc>
'''

'''{config:option} <config_option_key2> <group_key1>
...
'''

...

<!-- config group <group_key1> end –> 
<!-- config group <group_key2> start →
...

Note that within a same group (in YAML or TXT format), the configuration options are alphabetically sorted.

This .TXT format allows us to create tables of options in a clean way (here for example). This content is supposed to be used by the Config options Sphinx extension:

https://github.com/canonical/lxd-sphinx-extensions#config-options

API Changes:

  • We will introduce a 1.0/metadata/configuration endpoint to serve the generated YAML file as text/plain content.

CLI Changes:

  • We add a new tool (only called internally) called lxd-metadata :
$ lxd-metadata -h
   Usage of lxd-metadata:
   -y value
       Path of the generated YAML file
   -t value
       Path of the generated .TXT file (used for `readthedoc`)
   -e value
       Path that will be excluded from the process

Database Changes:

  • None

Upgrade Handling:

  • None