About this talk

If you are orchestrating microservices, use queues, webhooks, cron jobs, run periodic processes, distribute work or elect leader, call lambda functions, have background processes... you are most likely implementing a workflow in disguise. Explore the concept that nearly all non-trivial code can be viewed as a workflow. This talk introduces Temporal, an open-source workflow orchestration platform, which allows developers to write workflows as if their servers were running indefinitely.

Examine Temporal's ability to facilitate workflows that span extended periods, from milliseconds to years, without compromising reliability. The presentation will highlight Temporal's built-in features for resiliency, automatic retries, and load balancing, which significantly reduce the complexity of distributed system development.

This technical presentation aims to demonstrate how Temporal's approach to workflow management can significantly enhance the reliability and scalability of distributed systems, while also addressing the practical considerations of its implementation.

Key takeaways from this talk:

How to "easily" implement reliable and self-healing code
Multi-Language Support (Go, Java, PHP, and TypeScript)
Auditability
Testability
When NOT to use Temporal, common pitfalls and issues

The Illusion of Immortality

Temporal's Magic for Reliable Workflows

Tomasz Nurkiewicz

Software eng. at Monday.com

nurkiewicz.com, @[email protected]

It's your first day at work...

Implement this

                    
                        sendRegistrationLink(user);
                        sleep(86400);
                        if(!isRegistered(user)) {
                            sendReminderLink(user);
                        }

Demo time⁰¹

Problems

App can crash
sleep takes resources
Untestable

The illusion of a server running forever

Meet Temporal workflow engine

^*Do not confuse with JavaScript Temporal API

Not your grandpas workflow engine

Microsoft BizTalk Server
BPMN
Airflow

Everything is a workflow

Any non-trivial code on the planet

Several steps
Branches, conditions, loop
Long-running
Distributed
Asynchronous
Retryable

Examples - implicit workflows

Microservice orchestration
Queues
Webhooks
Cron jobs
Periodic processes
Work distribution
Leader election
...

Resiliency

Demo time⁰²

Problems

One, heavy task on a single node
...that can fail
...but no resume/retry mechanism

Supported languages

Go
Java
PHP
Python
TypeScript
.NET
Ruby

Under the hood

Server vs. worker vs. client

Temporal server

Event log and work queue

Database (SQL or Cassandra)
Elasticsearch
UI

State rehydration

Demo time

Worker node

Polls the server. Does the work

Client

Scalability

                    
                        for(user of getPayingUsers()) {
                            const invoice = prepareInvoice(user);
                            const url = storeOnS3(invoice);
                            sendEmail(user.email, url);
                        }

Demo time⁰³

Testing

First class support
Replaying process locally from event log!

The bad

There's no magic, just leaky abstractions

The server is a single point of failure

Sensitive database

All arguments and return values are stored

Versioning is tricky

Tracking progress is not that easy

Activities should be idempotent

Watch out for non-determinism

new Date() and Math.random() are monkey-patched!

Verdict

Impressive... And scary. But impressive!

Materials

Alternatives

Thank you!

nurkiewicz.com, @[email protected]

nurkiewicz.com/slides/temporal-workflow