Logging

Comby uses slog as the foundation for its logging system. Users are free to utilize the provided logging implementation or integrate their own. The primary advantage of the existing implementation is that it is tailored to comby’s specific requirements.

Additionally, all log entries are accessible through the Admin Dashboard, providing an intuitive interface for monitoring and troubleshooting. This feature simplifies debugging by consolidating relevant information in one convenient location.

Default Logger

The default logger comby.Logger logs both to the console and to a local SQLite database. The SQLite database is created as a file named logs.db. Log entries can be viewed directly through the Admin interface. The following details are logged:

• InstanceId: The ID of the application instance that created the log entry (default is 1).
• TenantUuid: The UUID of the tenant that created the log entry.
• Level: The log level (e.g., INFO, ERROR, DEBUG).
• Pkg: The name of the package that created the log entry.
• Message: The message being logged.
• Time: The timestamp when the message was logged.
• Fields: Additional fields included in the log entry.

comby’s Default Logger is fully compatible with slog and can be used directly within it. This allows users to seamlessly integrate the default logging implementation into their existing slog-based workflows.

import "log/slog"
...
slog.Info("This is an info message")

Alternatively, the Default Logger can be used directly without additional configuration:

import "github.com/gradientzero/comby/v2"
...
comby.Logger.Info("This is an info message")

Attribute Logging

The Default Logger follows the conventions of the slog package, including the use of log levels (INFO, ERROR, DEBUG) and structured logging. This ensures consistency with existing logging practices and simplifies the integration of comby’s logging system into various workflows.

All attributes included in a log message are always logged and stored in the database by default. These attributes can be viewed in the database and are accessible through the Admin Dashboard. The dashboard also provides filtering and sorting options based on instances, tenants, and packages.

To enable these filtering and sorting capabilities, each log entry must include the following attributes:

• instanceId: The ID of the application instance that created the log entry (default is 1).
• tenantUuid: The UUID of the tenant that created the log entry.
• pkg: The name of the package that created the log entry.

slog.Error("my message", "instanceId", 3, "tenantUuid", "tenant123", "pkg", "app1", "anyOther", "anyValue")

or if you prefere to use a more technical form:

slog.Error("my message", 
    comby.LOG_INSTANCE_ID_KEY, 3, 
    comby.LOG_TENANT_UUID_KEY, "tenant123", 
    comby.LOG_PKG_KEY, "app1", 
    "anyOther", "anyValue",
)

It is generally better to create a custom logger for your application that automatically includes the required attributes in every log entry.

var myLogger = slog.With(
    "pkg", "app1/pkg1/sub2",
    "other", "value",
)
...
myLogger.Error("This is an slog.Error message with custom logger")
// pkg and other are automatically included in the log entry

The attributes instanceId and tenantUuid are missing now. These can either be added manually or automatically by using a context.Context that injects the attributes - if they are available in the context. This approach is used within comby itself when processing commands or events.

// context values set within methods...
ctx = context.WithValue(ctx, comby.LOG_INSTANCE_ID_CTX_KEY, 3)
ctx = context.WithValue(ctx, comby.LOG_TENANT_UUID_CTX_KEY, "tenant123")
ctx = context.WithValue(ctx, comby.LOG_PKG_CTX_KEY, "myPkg/123")
...
slog.ErrorContext(ctx, "Msg")

Whichever approach you take, ultimately the values from the context are also converted into slog's fields.

Environment Variables

Comby supports the use of environment variables to configure various aspects of the Logging system. The following environment variables are available:

• COMBY_LOG_LEVEL: Sets the log level for the Default Logger. The default value is INFO. Options include DEBUG, INFO, WARN, and ERROR.
• COMBY_LOG_SQLITE_ENABLED: Enables or disables logging to internal database (SQlite). The default value is true. If disabled, logs are only written to the console.
• COMBY_LOG_SQLITE_PATH: Specifies the path to the log database file (SQlite). The default value is logs.db. If the file does not exist, it will be created automatically.

Note: If COMBY_LOG_SQLITE_ENABLED is set to false, the COMBY_LOG_SQLITE_PATH variable is ignored. Otherwise, the specified path is used to create the log database file. The internal SQLiteLogger writes logs non-blockingly, which means not all logs are immediately visible in the database. This is a trade-off between performance and consistency. Incoming log entries are first stored in an internal channel and then processed sequentially and written to the database. This ensures that logging does not block the application's flow.

Custom LogStore

Comby provides a default implementation of the LogStore interface that retrieves logs from a local SQLite database - if COMBY_LOG_SQLITE_ENABLED was not disabled. However, users can create their own custom LogStore implementation to retreieve logs from a different location. This allows for greater flexibility and customization based on specific requirements.

type LogStore interface {
	// Init initializes the log store with the provided options.
	Init(ctx context.Context, opts ...LogStoreOption) error

	// Get retrieves an log by its unique identifier.
	Get(ctx context.Context, opts ...LogStoreGetOption) (*LogStoreModel, error)

	// List retrieves a list of logs based on the provided options.
	List(ctx context.Context, opts ...LogStoreListOption) ([]*LogStoreModel, int64, error)

	// UniqueList retrieves a list of unique values based on a specific field.
	UniqueList(ctx context.Context, opts ...LogStoreUniqueListOption) ([]string, int64, error)
	
	// Total returns the total number of logs in the store.
	Total(ctx context.Context) int64

	// Close closes the log store connection.
	Close(ctx context.Context) error

	// Options returns the configuration options of the log store.
	Options() LogStoreOptions

	// String returns a string representation of the log store.
	String() string

	// Info provides detailed information about the log store.
	Info(ctx context.Context) (*LogStoreInfoModel, error)

	// Reset clears all logs from the store.
	Reset(ctx context.Context) error
}

The user could simply redirect console output to an external service and implement a custom LogStore that retrieves logs from the service and makes them available for viewing in the Comby Admin Dashboard. Then, the user could disable the internal SQLite logging by setting COMBY_LOG_SQLITE_ENABLED to false and pass the custom LogStore to the facade as an option.

import "github.com/gradientzero/comby/v2"
// ...
customLogStore := MyCustomLogStore()
fc, _ := comby.NewFacade(
    comby.FacadeWithLogStore(customLogStore),
)