Writing Applications🔗
The Pillar Application
class encapsulates common boilerplate when writing applications. Although they are configured to have many "sane" defaults or "automagic" detection of settings, they are fairly configurable.
Specifically Pillar applications handle the following tasks:
- Argument parsing and help info
- Configuration loading and parsing
- Standard logging outputs
- Console logging
- File logging
- Graceful handling of uncaught exceptions.
This makes Pillar applications a good base when writing services, background taks, scheduled (cron) jobs, or other similar programs. That said, Pillar applications are in no way limited to these types of programs, but if you are writing a heavily CLI driven program (like git
) there are better alternatives.
Writing your application🔗
You write applications by subclassing Application
and overriding class variables and methods as needed. For an example see the quickstart sample application.
The only method that you must override is the main
method. This is where you should put the bulk of your program. You can set the exit/return code of your application by simply returning an int
(note: it must be in the range of 0-255
inclusive). If your main method returns None
this will be converted to a 0
(no error) return code.
Sometimes you may wish to modify the setup
method to add your own setup steps. Whilst you can completely replace it, it is recommended that you that you call super().setup()
first so that the default behaviour can run first.
A common reason for overriding setup
rather than doing custom setup at the start of main
is when you have multiple applications that should do the same basic setup. For example you might attach a custom logger that sends messages to the work chat. By using a shared based class you don't need to worry about subclasses remembering to call your custom setup in main
.
# Could be in the same package, or you might have your own
# organisation base package you import into all projects
class BaseApplication(Application):
name = "my-app"
def setup(self, *args, **kwargs):
super().setup(*args, **kwargs)
# Custom setup
return
class ProcessStuff(BaseApplication):
application_name = "my-app"
# ...
class ReportStuff(BaseApplication):
application_name = "my-app-report"
# ...
Argument Parsing🔗
Argument parsing is handled through the standard library argparse
. Whilst there are more advanced packages available they are generally aimed at CLI heavy applications. For applications that are primarily customised through their config argparse
should provide enough functionality.
To add your own arguments you will need to override the get_argument_parser
method. This should provide a ready to use ArgumentParser
. In general you'll want to modify the default parser rather than creating one from scratch. To access it you should call the super().get_argument_parser()
method first:
class MyApplication(Application):
def get_argument_parser(self):
parser = super().get_argument_parser()
# modify the parser
return parser
When running you can access the parsed arguments through self.args
.
The Default Parser🔗
Generation of the default ArgumentParser
can be customised through certain class attributes on your application.
Program Name - This is used when generating the usage: <application_name> <args>
displayed in the help text. By default Pillar will attempt to guess what this name is based on how the application has been called (e.g. python yourscript.py
, python -m your.package.app
). However this detection is pretty limited so is recommended that you set it via the application_name
class attribute.
Description - This is used when generating the help text. It is collected and used verbatim from the docstring of your application class.
Epilog - This is displayed at end of the generated help text. It can be set through the epilog
class attribute.
Version - Version information can be provided through the version
class attribute. If set it will register a --version
argument that will print the version and exit.
The default parser may also generate arguments for the following:
Configuration Files - If enabled uses -c
/--config
and populates self.args.config_paths
.
Log directory - If file logging is enabled uses --log-dir
and populates self.args.log_dir
.
class MyApplication(Application):
"This is my super awesome cli"
application_name = "some-cli"
epilog = "NO WARRANTY IS PROVIDED"
version = "0.0.0@a97ef09"
Configuration Files🔗
In general Pillar applications are designed to be customised more through config files than they are commandline arguments. Configuration is loaded from two locations:
Default Config - The default config is set via the default_config
class attribute. It is expected to be of type Dict[str, Any]
.
If you are instantiating the application multiple times it is safe to use basic types including dict
and list
as these will be deep copied by the config loader. It is unsafe to use custom classes as by default these are not copied (and thus will be shared by all instances of your application).
Arguments - Config files can be passed as individual files via --config
or as a directory of files using --config-dir
to the be loaded using the ConfigLoader
. Config is first loaded from config directories with individual files being loaded in alphabetical/lexical order. Config is then loaded from individual files in the order they were provided.
This can be disabled by setting the config_args_enabled
class attribute to False
. Note: even if the config args are disabled the config loader will still be created (with the default config passed through).
By default providing config via the --config
or --config-dir
arguments is optional, however it can be made mandatory by setting the config_required
attributes to True
.
class MyApplication(Application):
config_required = True
default_config = {
"foo": 1,
"bar": True,
}
When running, the config is available via self.config
.
The Config Loader🔗
The ConfigLoader
class handles the loading, parsing, and merging of configuration files into a single config object.
Merging of configuration is will merge any dictionaries with any existing values from earlier calls being overriden by values set by later calls.
The config loader is available at self.config_loader
. Strictly speaking self.config
is just a reference to dict
located at self.config_loader.config
.
The config loader is exposed in case you want to customise how config files are loaded. For example you may wish to disable config files from arguments and instead load the config from a predetermined location.
class MyDnsService(Application):
"Custom DNS service"
name = "super-dns"
application_name = "dnsd"
config_args_enabled = False
default_config = {
# ...
}
def setup(self, *args, **kwargs):
super().setup(*args, **kwargs)
self.config_loader.load_config("/etc/dnsd/config.yaml")
return
The actual config loader used can be customised by setting the config_loader_class
class attribute on your application.
Logging🔗
By default your application will log on it's name (self.name
). Like application_name
Pillar will attempt to guess this name if it is not provided. You can control this by setting the name
class attribute.
Pillar applications use the LoggingMixin
which provides the following methods to log at the given level:
self.vvdebug
(very verbose debug)self.vdebug
(verbose debug)self.debug
self.info
self.warning
self.error
self.critical
You'll notice two extra debug levels. These are defined in pillar.logging
to allow for finer grained control over your debug messages.
Customising Logging🔗
Much of the default logging behaviour can be customised by setting the logging_manifest
class attribute to your own instance of LoggingManifest
.
Default level - The default log level is INFO
however this can be changed by setting the default_level
. This default level will affect the logging level calculated from increasing the logging verbosity in the arguments. For example if the default level is DEBUG
increasing the verbosity by one will increase it to VDEBUG
. This is calculated using get_log_level
.
Logging additional names - If you wish to log additional names (for example from another library) you can provide them through additional_namespaces
on the logging manifest. These loggers will be attached to the console and file handler.
Console Logging🔗
By default logs to sys.stderr
.
For further options see the LoggingManifest
.
File Logging🔗
Log files will be generated at /var/tmp/{name}.log
. This can be disabled by setting file_enabled
to False
.
To prevent accidentally filling of diskspace these logs are limited in size (file_max_size
) and only a limited number are stored (file_backup_count
).
For further options see the LoggingManifest
.
Initial Logging🔗
The initial logging is created as the first step during setup
. This enables logging of errors encountered during argument parsing, config loading, and later logging setup.
In addition to logging to the console it also logs to a dedicated file located at /var/tmp/{name}.init.log
.
For further options see the LoggingManifest
.
Testing and Debugging Applications🔗
Under Construction 🏗️