Part 6: Readmodels

INFO

Readmodels are a part of the CQRS pattern, which the comby framework implements. While it's not mandatory to use them, they are a powerful tool for improving performance. Alternatively, one could use the AggregateRepository to read the desired models, but this approach is less performant and would require additional processing to transform the data into the desired response format. Readmodels, on the other hand, are optimized for this purpose. They can be directly utilized and tailored for specific requests, providing a more efficient and streamlined solution. Further information about Readmodels can be found in the documentation Readmodel

What is a Readmodel?

A Readmodel serves as a domain event handler that consumes and processes received events. It interprets these events to build a projection, which is a tailored representation of the data. This projection can take various forms, such as a table in a database, a local file, or an in-memory model.

The Readmodel's projection is optimized for querying, allowing immediate responses to be returned. This eliminates the need to request data from the write side and aggregate information dynamically, significantly improving efficiency and responsiveness.

comby offers robust support for implementing Readmodels, catering to a variety of use cases by providing both predefined and customizable options.

Predefined Readmodels

The predefined Readmodels — also referred to as Projections — provide a straightforward way to transform existing Aggregates into simplified representations. They are easy to create and offer a quick and efficient means to reflect the current state of the domain in a Readmodel. Alongside these predefined Readmodels, the comby framework includes matching QueryHandlers, designed to retrieve data directly from these projections. As we want to show you how to create a custom readmodel in this document, refer to Projection for more details.

Custom Readmodel

Like most key components in comby, a Readmodel is built upon a base class called BaseReadmodel. This class provides a solid foundation for implementing Readmodels, including essential functions and structures needed for processing domain events and restoring state from the store.

Here is the structure of our custom Readmodel:

// simple/domain/order/readmodel/readmodel.go
package readmodel

import (
	"context"
	"sync"

	"github.com/gradientzero/comby/v2"
)

type CustomOrderReadmodel struct {
	*comby.BaseReadmodel
	mapByTenantUuid sync.Map
	mapByOrderUuid  sync.Map
}

func NewCustomReadmodel(EventRepository *comby.EventRepository) *CustomOrderReadmodel {
	rm := &CustomOrderReadmodel{}
	rm.BaseReadmodel = comby.NewBaseReadmodel(EventRepository)
	rm.Domain = "Order"
	rm.Name = "CustomOrderReadmodel"
	rm.mapByTenantUuid = sync.Map{}
	rm.mapByOrderUuid = sync.Map{}

	// register domain event handlers
	comby.AddDomainEventHandler(rm, rm.TenantCreatedEvent)
	comby.AddDomainEventHandler(rm, rm.TenantRemovedEvent)
	comby.AddDomainEventHandler(rm, rm.TenantUpdatedEvent)
	comby.AddDomainEventHandler(rm, rm.OrderPlacedEvent)
	comby.AddDomainEventHandler(rm, rm.OrderCancelledEvent)
	comby.AddDomainEventHandler(rm, rm.OrderDeletedEvent)
	return rm
}

func (rm *CustomOrderReadmodel) RestoreState(ctx context.Context, restoreFromZero bool) (comby.RestorationDoneCh, error) {
	rm.mapByTenantUuid = sync.Map{}
	rm.mapByOrderUuid = sync.Map{}
	return rm.BaseReadmodel.RestoreState(ctx, restoreFromZero)
}

Similar to aggregates, we define our domain event handlers to specify which domain events the Readmodel should process. Additionally, we extend the base RestoreState method to reset or initialize any additional variables required during state restoration.

Our Readmodel utilizes two in-memory maps to store the data, effectively serving as custom storage for our custom model. This storage is dynamically built and updated based on the incoming events.

Custom Models

Our Readmodel is designed to maintain and build two distinct models from the events it processes. The primary model CustomOrderModel represents the orders within the system. The secondary model OrderTenantModel serves as a helper model, linking each order to its corresponding tenant.

Since we are working in a multi-tenant application, tenant information is critical to ensure that queries only return orders belonging to the relevant tenant. This separation allows to exclude orders from other tenants during queries. We could also use the TenantUuid only, but best practices suggest storing the tenant information directly in the model to avoid additional lookups.

// simple/domain/order/readmodel/model.go
package readmodel

type CustomOrderModel struct {
	Tenant       *OrderTenantModel `json:"tenant"`
	OrderUuid    string            `json:"orderUuid"`
	Sku          string            `json:"sku"`
	Quantity     int64             `json:"quantity"`
	CustomStatus string            `json:"customStatus"`
	Version      int64             `json:"version,omitempty"`
	CreatedAt    int64             `json:"createdAt,omitempty"`
	UpdatedAt    int64             `json:"updatedAt,omitempty"`
}

type OrderTenantModel struct {
	TenantUuid string `json:"tenantUuid"`
	Name       string `json:"name,omitempty"`
}

Domain Event Handling

Once our models are defined, the next step is implementing event handling. We define handlers for the domain events that our Readmodel needs to process. In our case, these include events from the external Tenant domain and events from our own Order domain.

Processing events from external domains is perfectly fine and aligns with the purpose of a Readmodel, as long as you do not access the Readmodel of the external domain directly. Doing so would violate the principle of domain separation. There may be exceptions, but in general, you should avoid accessing the Readmodel of another domain.

In practice, this means the Tenant domain fully manages all tenant-related information internally, while our Order domain only stores minimal information—such as the tenant's name. This is usually sufficient for tasks like displaying orders in a table. If additional tenant data is needed, we can adapt our model and add additional handlers to process more external domain events accordingly.

Let's start with handling the external domain events:

Tenants

First, we define the domain event handlers in a separate file. This is a matter of preference, but grouping handlers in their own file helps maintain clarity and organization. Next, we import the external aggregate to access its domain events. As a reminder, domain events are defined within the respective aggregates of their domains.

We then extend our Readmodel by adding three methods to handle the external domain events. In our case, these methods are TenantCreatedEvent, TenantRemovedEvent, and TenantUpdatedEvent. While the method names can be chosen freely, they should clearly reflect the events they process. The crucial part is the method arguments — they must match the the domain events being handled.

The context, the complete Event and the already extracted and typed DomainEvent contained in the complete Event are always passed.

// simple/domain/order/readmodel/handle.tenant.go
package readmodel

import (
	"context"

	"github.com/gradientzero/comby/v2"
	tenantAggregate "github.com/gradientzero/comby/v2/domain/tenant/aggregate"
)

func (rm *CustomOrderReadmodel) TenantCreatedEvent(ctx context.Context, evt comby.Event, domainEvt *tenantAggregate.TenantCreatedEvent) error {
	m, _ := rm.getTenant(evt.GetAggregateUuid())
	if m == nil {
		m = &OrderTenantModel{TenantUuid: evt.GetAggregateUuid()}
	}
	m.Name = domainEvt.Name
	return rm.setTenant(m)
}

func (rm *CustomOrderReadmodel) TenantRemovedEvent(ctx context.Context, evt comby.Event, domainEvt *tenantAggregate.TenantRemovedEvent) error {
	m, _ := rm.getTenant(evt.GetAggregateUuid())
	if m != nil {
		rm.removeTenant(m)
	}
	return nil
}

func (rm *CustomOrderReadmodel) TenantUpdatedEvent(ctx context.Context, evt comby.Event, domainEvt *tenantAggregate.TenantUpdatedEvent) error {
	m, _ := rm.getTenant(evt.GetAggregateUuid())
	if m == nil {
		m = &OrderTenantModel{TenantUuid: evt.GetAggregateUuid()}
	}
	m.Name = domainEvt.Name
	return rm.setTenant(m)
}

Regardless of coding style (there are always ways to make things more elegant), we use helper methods to access the models. These methods (getTenant, setTenant and removeTenant) are defined in the readmodel.tenant.go file. Essentially, the helper methods act as wrappers around the mapByTenantUuid variable, where we store our tenants separately.

While this approach is somewhat a matter of preference, it’s often practical to store an instance of a tenant in a variable and pass it as a pointer to other models. This ensures that we always work with the most up-to-date version of the tenant, maintaining consistency and reducing the risk of stale data.

Orders

We can now process the domain events of our own domain aggregate. We start by implementing the event handlers for the Order domain. As with the external events, we define these handlers in a separate file for better organization. Additionally, we import the corresponding aggregate to gain access to our own domain events.

The custom Readmodel — one of potentially many custom Readmodels we can create to represent different data structures — focuses on handling just three domain events: OrderPlacedEvent, OrderCancelledEvent, and OrderDeletedEvent. Again, the methods names are chosen freely, but the method arguments must match the domain events being handled.

// simple/domain/order/readmodel/handle.order.go
package readmodel

import (
	"context"
	"fmt"

	"comby.io/examples/simple/domain/order/aggregate"
	"github.com/gradientzero/comby/v2"
)

func (rm *CustomOrderReadmodel) OrderPlacedEvent(ctx context.Context, evt comby.Event, domainEvt *aggregate.OrderPlacedEvent) error {
	m, _ := rm.getOrder(evt.GetAggregateUuid())
	if m == nil {
		m = &CustomOrderModel{OrderUuid: evt.GetAggregateUuid()}
	}
	if m != nil {
		tenantModel, _ := rm.getTenant(evt.GetTenantUuid())
		if tenantModel != nil {
			m.Tenant = tenantModel
		}
		m.Sku = domainEvt.Item.Sku
		m.Quantity = domainEvt.Item.Quantity
		m.CustomStatus = "custom-placed"
		m.Version = evt.GetVersion()
		m.CreatedAt = evt.GetCreatedAt()
		m.UpdatedAt = evt.GetCreatedAt()
		return rm.setOrder(m)
	}
	return fmt.Errorf("error on creating Item")
}

func (rm *CustomOrderReadmodel) OrderCancelledEvent(ctx context.Context, evt comby.Event, domainEvt *aggregate.OrderCancelledEvent) error {
	if m, err := rm.getOrder(evt.GetAggregateUuid()); m != nil {
		m.CustomStatus = "custom-cancelled-maybe-next-time"
		m.Version = evt.GetVersion()
		m.UpdatedAt = evt.GetCreatedAt()
		return rm.setOrder(m)
	} else {
		return err
	}
}

func (rm *CustomOrderReadmodel) OrderDeletedEvent(ctx context.Context, evt comby.Event, domainEvt *aggregate.OrderDeletedEvent) error {
	if m, err := rm.getOrder(evt.GetAggregateUuid()); m != nil {
		return rm.removeOrder(m)
	} else {
		return err
	}
}

Similar to the tenant models, we use helper methods to access our custom order models. These methods (getOrder, setOrder, and removeOrder) are defined in the readmodel.model.go file. Essentially, these helper methods act as wrappers around the mapByOrderUuid variable, where we store our custom order models separately.

The key difference here is that we also utilize information directly from the Event itself. A domain event contains all domain-specific core data we need, but it does not include metadata. That's the reason we always define both arguments in the handlers: Event and DomainEvent to have access to all information.

This metadata in Event provides details such as TenantUuid, AggregateUuid, CreatedAt, Version and more. Since this information is present in every event we can keep the domain events more concise and easier to manage.

Model Helpers

These helper functions operate in-memory using a map. However, it’s also possible to replace the in-memory map with a database or another storage medium, depending on the application's requirements and scalability needs.

// simple/domain/order/readmodel/readmodel.tenant.go
package readmodel

func (rm *CustomOrderReadmodel) getTenant(tenantUuid string) (*OrderTenantModel, error) {
	if val, found := rm.mapByTenantUuid.Load(tenantUuid); found {
		m, ok := val.(*OrderTenantModel)
		if ok {
			return m, nil
		}
	}
	return nil, nil
}

func (rm *CustomOrderReadmodel) setTenant(m *OrderTenantModel) error {
	rm.mapByTenantUuid.Store(m.TenantUuid, m)
	return nil
}

func (rm *CustomOrderReadmodel) removeTenant(m *OrderTenantModel) error {
	if _, found := rm.mapByTenantUuid.Load(m.TenantUuid); found {
		rm.mapByTenantUuid.Delete(m.TenantUuid)
	}
	return nil
}

For completeness, we also provide the helper methods for the order models below:

// simple/domain/order/readmodel/readmodel.model.go
package readmodel

func (rm *CustomOrderReadmodel) getOrder(orderUuid string) (*CustomOrderModel, error) {
	if val, found := rm.mapByOrderUuid.Load(orderUuid); found {
		m, ok := val.(*CustomOrderModel)
		if ok {
			return m, nil
		}
	}
	return nil, nil
}

func (rm *CustomOrderReadmodel) setOrder(m *CustomOrderModel) error {
	rm.mapByOrderUuid.Store(m.OrderUuid, m)
	return nil
}

func (rm *CustomOrderReadmodel) removeOrder(m *CustomOrderModel) error {
	if _, found := rm.mapByOrderUuid.Load(m.OrderUuid); found {
		rm.mapByOrderUuid.Delete(m.OrderUuid)
	}
	return nil
}

Query Helpers

Up to this point, we have created a custom Readmodel that processes domain events, generates its own models, and stores them in-memory. Now, we focus on the custom methods needed to handle queries.

These are utility methods that can be freely defined based on the requirements of the queries. For example, one query might request a list of orders, while another might request details for a specific order. The helper methods are responsible for extracting the relevant data from the models and returning it in the desired format, making it easy to respond to various query types efficiently.

In our example, we define and expose two methods for handling queries: GetModelList and GetModel. These methods access the in-memory maps and return the requested data. This is a sample implementation that is used in comby for some domains but can also serve as a foundation for custom implementations. These methods provide a straightforward approach to query data from the Readmodel, which can be extended or modified as needed to suit specific application requirements.

A key principle when working with Readmodels is to never return the original model to a query but instead provide a copy. This practice is crucial for maintaining the integrity of the Readmodel and ensuring that no unintended modifications are made. This affects in-memory implementations, but can also be relevant for other Readmodel storages.

// simple/domain/order/readmodel/readmodel.query.go
package readmodel

import (
	"math"
	"sort"
	"strings"

	"github.com/gradientzero/comby/v2"
)

func (rm *CustomOrderReadmodel) GetModel(orderUuid string) (*CustomOrderModel, error) {
	m, err := rm.getOrder(orderUuid)
	if err != nil {
		return nil, err
	}
	// in-memory: return a copy of the model
	if cpy, ok := comby.DeepCopy[*CustomOrderModel](m); ok {
		return cpy, nil
	}
	// optinally: warn user
	return m, nil
}

type ListOptions struct {
	TenantUuid string
	Page       int64
	PageSize   int64
	OrderBy    string
}

func ListWithTenantUuid(tenantUuid string) func(*ListOptions) {
	return func(modelListOptions *ListOptions) {
		modelListOptions.TenantUuid = tenantUuid
	}
}

func ListWithPage(page int64) func(*ListOptions) {
	return func(modelListOptions *ListOptions) {
		modelListOptions.Page = page
	}
}

func ListWithPageSize(pageSize int64) func(*ListOptions) {
	return func(modelListOptions *ListOptions) {
		modelListOptions.PageSize = pageSize
	}
}

func ListWithOrderBy(orderBy string) func(*ListOptions) {
	return func(modelListOptions *ListOptions) {
		modelListOptions.OrderBy = orderBy
	}
}
func (rm *CustomOrderReadmodel) GetModelList(options ...func(*ListOptions)) ([]*CustomOrderModel, int64, error) {
	modelList, total, err := rm.getModelList(options...)
	if err != nil {
		return nil, total, err
	}
	for i, m := range modelList {
		// in-memory: return a copy of the model
		if cpy, ok := comby.DeepCopy[*CustomOrderModel](m); ok {
			modelList[i] = cpy
		}
		// optinally: warn user
	}
	return modelList, total, err
}

func (rm *CustomOrderReadmodel) getModelList(options ...func(*ListOptions)) ([]*CustomOrderModel, int64, error) {
	listOptions := &ListOptions{
		TenantUuid: "",
		Page:       0,
		PageSize:   math.MaxInt64,
		OrderBy:    "createdAt",
	}
	for _, option := range options {
		option(listOptions)
	}
	var modelList []*CustomOrderModel
	rm.mapByOrderUuid.Range(func(key, value interface{}) bool {
		m, ok := value.(*CustomOrderModel)
		if ok {
			validTenant := len(listOptions.TenantUuid) == 0
			if !validTenant {
				validTenant = m.Tenant.TenantUuid == listOptions.TenantUuid
			}
			if validTenant {
				modelList = append(modelList, m)
			}
		}
		return true
	})
	total := len(modelList)
	modelList = _orderBy(modelList, listOptions.OrderBy)
	modelList = _paginate(modelList, listOptions.Page, listOptions.PageSize)
	return modelList, int64(total), nil
}

// helper
func _orderBy(modelList []*CustomOrderModel, orderBy string) []*CustomOrderModel {
	if orderBy == "status" {
		sort.Slice(modelList, func(i, j int) bool {
			return strings.ToLower(modelList[i].CustomStatus) < strings.ToLower(modelList[j].CustomStatus)
		})
	}
	if orderBy == "-status" {
		sort.Slice(modelList, func(i, j int) bool {
			return strings.ToLower(modelList[i].CustomStatus) > strings.ToLower(modelList[j].CustomStatus)
		})
	}
	if len(orderBy) == 0 || orderBy == "createdAt" { // asc
		sort.Slice(modelList, func(i, j int) bool {
			return modelList[i].CreatedAt < modelList[j].CreatedAt
		})
	}
	if orderBy == "-createdAt" { // desc
		sort.Slice(modelList, func(i, j int) bool {
			return modelList[i].CreatedAt > modelList[j].CreatedAt
		})
	}
	return modelList
}

func _paginate(modelList []*CustomOrderModel, page, pageSize int64) []*CustomOrderModel {
	if page > 0 && pageSize > 0 {
		from := (page * pageSize) - pageSize
		to := from + pageSize
		if from >= int64(len(modelList)) {
			return modelList[0:0]
		}
		if to >= int64(len(modelList)) {
			return modelList[from:]
		}
		return modelList[from:to]
	}
	return modelList
}

Our custom Readmodel is now complete and ready to process domain events, create models, and provide data for queries. The implementation is tailored to the specific requirements of our use case.

Register in Facade

Finally, we need to register our custom Readmodel in the facade. This allows the Readmodel to receive domain events and process them accordingly. By registering our CustomReadmodel in the facade, we ensure that it is actively listening for events and updating its models as needed.

// simple/domain/order/register.go
func Register(ctx context.Context, fc *comby.Facade) error {
    // ...
    rm := readmodel.NewCustomReadmodel(eventRepository)
    if err := comby.RegisterEventHandler(fc, rm); err != nil {
        return err
    }
    // ...
}