Design Reference

Python

When choosing what to document, AutoAPI aims to document anything that is publicly accessible through the actual package when loaded in Python. For example if a function is imported from a submodule into a package, that function is documented in both the submodule and the package. There are some exceptions to this rule:

• Anything that is imported into a module is not documented. Usually a module is where implementations exist. Therefore an import of something is usually for the usage of the implementation, and not as something to be accessed publicly.

• When the module or package defines an __all__, only the members named in __all__ are documented.

• When a configuration option indicates that private or special members should also be documented.

Furthermore, AutoAPI follows the same docstring inheritance rules as inspect.getdoc(), with some exceptions:

• The docstrings of the following methods are not inherited because they are usually redundant:

  • object.__init__()

  • object.__new__()

  • type.__init__()

  • type.__new__()

.NET

This document talks about the design of a .NET Sphinx integration. This will include a mechanism for generating Javadoc style API references automatically. We will describe decisions that lead to specific implementation details.

Goals

The main goal of this project is to be able to generate a MSDN or Javadoc style API reference from a .Net project in Sphinx.

Primary Goals

• Build MSDN/Javadoc style HTML output for arbitrary .Net code.

• Have specific pages for each Package, Class, and all class-level structures.

  • /api/System/

  • /api/System/String/

  • /api/System/String/Constructors/

Secondary Goals

• Allow for definition of .Net classes inside of normal Sphinx prose docs (classic Sphinx style definition).

• Allow generation of Javadoc style docs from other languages.

Requirements

Introduction

We are working with Sphinx, which has an existing way of doing this. Generally, you define a Domain which describes the various language structure, a Class or Method, for example. Then the user will write RST that uses these definitions, and Sphinx will create output from that markup.

.. py:function:: spam(eggs)

   Spam the foo.

The author of the documentation will have now told Sphinx that the spam function exists in the Python project that is being documented.

Autogenerated Output

Sphinx then built a series of tools to make the generation of this markup easier and more automatic:

• Autodoc

• Autosummary

Autodoc is a Python-only solution that imports the author’s code into memory, and then allows the author to more automatically document full objects. For example, you can document a whole class on a page.

.. autoclass:: Noodle

This will generate output that looks like:

class Noodle

Noodle’s docstring.

There are also options for it to include a full listing of the classes attributes, methods, and other things, automatically.

Warning

Remember, this depends on Noodle being importable by the Python interpreter running Sphinx.

Proposed Architecture

The proposed architecture for this project is as follows:

• A program that will generate a YAML (or JSON) file from a .Net project, representing it’s full API information.

• Read the YAML and generate an appropriate tree structure that will the outputted HTML will look like (YAMLTree)

  • If time allows, we will allow a merging of these objects with multiple YAML files to allow for prose content to be injected into the output

• Take the YAML structure and generate in-memory rst that corresponds to the Sphinx dotnet Domain objects

• dotnet Domain will output HTML based on the doctree generated from the in-memory RST

In diagram form:

Code -> YAML -> YAMLTree -> RST (Dotnet Domain) -> Sphinx -> HTML

YAMLTree

One of the main problems is how to actually structure the outputted HTML pages. The YAML file will likely be ordered, but we need to have a place to define the page structure in the HTML.

This can be done before or after the loading of the content into RST. We decided to do it before loading into RST because that matches standard Sphinx convention. Generally the markup being fed in as RST is considered to be in a file that maps to it’s output location. If we tried to manipulate this structure after loading into the Domain, that could lead to unexpected consequences like wrong indexes and missing references.

File Structure vs. Hierarchy

Specific ID’s should have one specific detail representation. This means that every YAML docid object should only have one place that it is rendered with a .. dn:<method> canonical identifier. All other places it is referenced should be in either:

• A reference

• A toctree (listing)

Sphinx Implementation

The user will run a normal make html as part of the experience. The generation and loading will be done as an extension that can be configured.

There will be Sphinx configuration for how things get built:

autoapi_root = 'api' # Where HTML is generated
autoapi_dirs = ['yaml'] # Directory of YAML sources

We will then loop over all YAML files in the autoapi_dir and parse them. They will then be output into autoapi_root inside the documentation.

Examples

A nice example of Sphinx Python output similar to what we want:

• http://dta.googlecode.com/git/doc/_build/html/index.html

• Src: https://raw.githubusercontent.com/sfcta/dta/master/doc/index.rst

An example domain for Spec:

• https://subversion.xray.aps.anl.gov/bcdaext/specdomain/trunk/src/specdomain/sphinxcontrib/specdomain.py

Other Ideas

Warning

Things in this section might not get implemented.

The .Net domain will not be able to depend on importing code from the users code base. We might be able to implement similar authoring tools with the YAML file. We might be able to output the YAML subtree of an object with autodoc style tools:

.. autodnclass:: System.String
   :members: